Technology debates are often a tug-of-war between open and closed systems.
On one side, open allows interoperability, customization, and integration with third-party software or hardware. Champions highlight how openness promotes transparency, accountability, competition, and significant innovation. On the other side, defenders of closed argue that they are more stable and secure and better protect their owners’ property interests.
Much of AI’s creation and evolution have happened thanks to open-source development and diffusion. Numerous widely-adopted AI open-source projects provide development frameworks and libraries such as PyTorch, TensorFlow, and MXNet, and many companies – including Hugging Face, Stability AI, Nomic AI, and Meta – have released open-source AI models or enable open-source development.
Google and OpenAI have traditionally stood on the side of openness. Both have published AI research and open-source tools. Google, for example, originally developed TensorFlow in-house and later released it as an open-source software library for building AI.
But both have become more closed and protective of their models: “If you believe, as we do,” that “AI – AGI [i.e., artificial general intelligence that essentially performs any intellectual task that a human can] is going to be extremely, unbelievably potent, then it just does not make sense to open source,” says Ilya Sutskever, OpenAI’s chief scientist and co-founder. Google’s Zoubin Ghahramani echoes this sentiment: “We want to think more carefully about giving away details or open sourcing code.”
As suggested by these moves from open to closed, AI policy answers will not always be fully open or fully closed. Instead, policymakers will, often appropriately, draw an AI regulatory line somewhere between the two boundaries of the open-closed spectrum.
The debate over finding that balance is most advanced in Europe, where policymakers are moving forward with the AI Act. Recently, the European Parliament proposed amendments to the Act that suggest a hybrid approach to open vs. closed. Concerned about the growing capacity of tools such as ChatGPT, the Parliament decided that “foundation models” may be open source but only in a rule-bound way. Foundation models are AI models – including large language models (LLMs) – that are “trained on broad data at scale, designed for generality of output [i.e., able to perform many different functions], and can be adapted to a wide range of distinctive tasks.”
The proposed parliamentary language would impose significant compliance requirements on open-source developers of foundation models, including the obligation to achieve “performance, predictability, interpretability, corrigibility, security, and cybersecurity throughout [their] lifecycle.”
Realistically, only an organized and well-funded – and perhaps European – open-source project could meet these obligations. Yet open-source projects could still build foundation models, at least in theory.
The proposed restriction attempts to thread the needle between the EU’s interest in supporting open-source AI – important, among other things, to the EU’s efforts to achieve strategic autonomy – and ensuring trustworthy AI development through legislative guardrails.
Whether or not this particular middle-ground outcome represents the right policy answer, it is an approach that uses the open vs. closed framework to break the binary. By doing so, it prompts an important discussion about deep policy concerns. Can semi-closed open-source ecosystems work? Will the European Parliament’s proposed guardrails for open-source foundation model development harm EU competitiveness? Will allowing open-source development of foundation models at all put powerful AI in dangerous hands and create a threat to the EU and beyond?
The last question is one that policymakers are grappling with by potentially drawing a line between open and closed. In testimony before the US Senate earlier this month, Sam Altman, OpenAI’s CEO, stated that AI regulation should take effect above a capability threshold and suggested that open source should not be permitted above that threshold. Below that threshold, however, open source could continue. As Altman later tweeted: “We shouldn’t mess with models below the threshold. [O]pen source models and small startups are obviously important.”
In a possible move back toward open, OpenAI reportedly plans to open source one of its LLMs in the near future. The size and nature of that model – along with Meta’s LLaMA and others – will help develop the rationale for a future potential threshold. Below that level, open source would continue unfettered. Above it, AI models would be subject to restrictions, including licensing and testing requirements and potentially export controls.
The distinction between AI below the “capability threshold” and AI above that threshold will be difficult to draw. Already the conversation seems to be shifting from AGI to concerns about superintelligence (i.e., AI that is more capable than AGI). The decision is also complicated by related issues, such as how and whether AI systems should align with human values and goals.
Yet the threshold question prompts policymakers and other stakeholders to ask which parts of the AI ecosystem should be open and which are more appropriate to restrict. It suggests a granular and precise policy outcome: a largely open AI ecosystem that exists alongside a closely regulated system for foundation models above a certain capacity threshold. This would be similar to the performance threshold that divides advanced integrated circuits from standard ones in the Biden administration’s restrictions on chip sales to China.
The open vs. closed framework remains imperfect. In the AI context, both sides have a legitimate claim to being a better model for safety and security. As noted by Stability AI, an open-source AI company, open models and datasets can help ensure robust oversight; third parties with access can anticipate emerging risks and implement mitigations. But the nature of open-source licensing (which generally makes source code available to all) opens the door to entities or individuals who wish to cause harm or are simply not concerned about or resourced for risk mitigation.
Still, the open vs. closed framework helps policy analysis because it results in nuanced and targeted solutions. Even for AI that is not fully open source, third parties such as academics and regulators should be allowed to review datasets, weights, and other critical components of AI models by third parties to help ensure the models’ safety.
Navigating the spectrum between open and closed will be important to AI policy development. Finding the right balance will help promote innovation while managing the very serious short- and long-term risks posed by the technology.
The European Parliament’s approach and OpenAI’s threshold model attempt to find an appropriate balance between open and closed. Neither is perfect. But both contribute to a much-needed debate to leverage the best of open and closed systems to ensure that AI evolves in a way that benefits and doesn’t harm humanity.
Pablo Chavez is an Adjunct Senior Fellow with the Center for a New American Security’s Technology and National Security Program and a technology policy expert. He has held public policy leadership positions at Google, LinkedIn, and Microsoft and has served as a senior staffer in the U.S. Senate.
Bandwidth is CEPA’s online journal dedicated to advancing transatlantic cooperation on tech policy. All opinions are those of the author and do not necessarily represent the position or views of the institutions they represent or the Center for European Policy Analysis.
