Not long after Anthropic released its groundbreaking Mythos model, word spread that the White House was considering an executive order to impose AI vetting — though the administration later walked this back. As the administration contemplates AI vetting, there are precedents upon which it can draw. The fundamental goal of this vetting process should be “no surprises.” But there are questions involved that need to be answered first.
The main dilemma with AI vetting is the tendency to exaggerate risk in much of the discussion of AI. This bias explains why public opinion has been so negative. A potential goal of “no surprises” makes sense if vetting does not create obstacles to AI development or investment, provides a degree of transparency sufficient to address public concerns, and is based on voluntary procedures. This is sufficient for what needs to be done. The greatest risk for AI is that the US trips over itself in building the economy required for the future.
Is the vetting process review-only or does it require approval? Previous comments from the Administration suggest review alone.
Is vetting mandatory or voluntary? Does the government identify products that it wants to review or is it left to the companies’ discretion? Precedent and experience suggest that a voluntary submission model, based on federal objectives and guidelines and accompanied by after-the-fact penalties for failing to submit, is probably best and the least obstructionist process.
Does vetting create “Safe Harbor,” offering companies that submit their models for review protection from future liability? This is a powerful incentive to participate in voluntary reviews but always raises questions about whether too much protection has been provided.
There will have to be decisions on transparency. Will the results of the vetting process — or, at a minimum, the names of the products being submitted for review — be made public? This is probably a place where erring on the side of greater transparency makes sense. Guidelines on what can be shared publicly or with potential victims need to be agreed with the private sector.
Should there be performance thresholds on what kind of AI models should be reviewed? There are many AI products and services already in existence. The easiest approach is to rely on a voluntary system of self-submission, where companies are required to decide whether their model requires vetting. This has to be reinforced, of course, by penalties for a failure to submit if this is later judged to have been necessary.
Procedures for reviews that identify issues will need to be agreed upon. We cannot assume that every instance will follow the pattern of Mythos and the cybersecurity concerns it raised.
Can a company just ignore the results of the vetting process? Is there a dispute resolution process? And under what legal authorities can a company be required to make changes that address security concerns? There will be a temptation, at least in some communities,to second guess and seek to impose limits on AI performance. The US should avoid this in favor of unrestricted development — subject to review. Unrestricted development, accompanied by vetting, is likely sufficient to manage actual risk.
If this is to be more formal and less ad hoc, topics like review timelines, thresholds, safe harbor, and revisions, should be the subject of negotiation with industry partners. An informal process based on existing federal authority (and a vetting process may identify where Congress might consider expanding authorities) offers greater flexibility and ease of adjustment than a regulatory process. The US is not ready to have vetting captured by lawyers.
There are other precedents for vetting. The National Institute of Standards and Technology’s (NIST) Center for AI Standards and Innovation could serve as the basis for a new vetting system. NIST has a long tradition of working closely with the NSA on encryption and cybersecurity. NIST is not a regulatory agency, which engenders greater private sector trust, but neither is the NSA. A White House group would need to delegate actual vetting to an agency with technical expertise, under its direction.
The Administration referred to the UK’s AI Security Institute and related efforts as precedents. Its use of red-teaming and the creation of compute thresholds offers advantages. The UK’s effort appears to be going well and could be a model — or even a partner — for the US.
An informal vetting process will not satisfy those who worry (excessively) about the risks from AI. But these risks remain hypothetical and should be a secondary priority to protecting the US’s ability to innovate in AI.
James Lewis is a Distinguished Fellow at CEPA’s Tech Policy program.
Bandwidth is CEPA’s online journal dedicated to advancing transatlantic cooperation on tech policy. All opinions expressed on Bandwidth are those of the author alone and may not represent those of the institutions they represent or the Center for European Policy Analysis. CEPA maintains a strict intellectual independence policy across all its projects and publications.
Tech 2030
A Roadmap for Europe-US Tech Cooperation
