A growing chorus of AI experts have recently raised concerns about risks from advanced AI models. In June 2024, nine current and former OpenAI employees wrote an open letter criticizing their employer for “recklessly racing” to build artificial general intelligence, or AGI. One of these individuals, Daniel Kokotaljo, spoke out notwithstanding that OpenAI initially conditioned his equity, worth approximately $1.7 million, on compliance with a non-disparagement agreement. Another whistleblower, Suchir Balaji, died by suicide only months after he accused OpenAI of violating US copyright law. And an anonymous survey from 2024 showed that many employees of other leading AI labs — including DeepMind, Meta, and Anthropic — also express worry about their employers’ approach to AI development. 

Such employees have unique visibility into the risks that emerging AI poses. But unless these employees are adequately protected against retaliation, they may be reluctant to speak out. Fortunately, federal and state lawmakers have sought to empower them with more robust whistleblower protections. California Governor Gavin Newsom recently signed into law the Transparency in Frontier Artificial Intelligence Act (TFAIA), which protects certain whistleblowers who disclose information about potential “catastrophic risks” from AI. In Congress, Senator Chuck Grassley has introduced the AI Whistleblower Protection Act (AIWPA), which would protect whistleblowers who report on a broad range of “AI violations” or “AI security vulnerabilities.” Both the TFAIA and the AIWPA (if enacted) meaningfully expand existing statutory and common law whistleblower protections, which generally only cover employees who report violations of law. 

While many debates on AI policy splinter along party lines, whistleblower protections generally enjoy bipartisan support. As one illustration, over thirty Senators from both parties belong to the Senate Whistleblower Protection Caucus. Nevertheless, designing effective whistleblower legislation is challenging. It requires legislators to determine the type of workers protected, the scope of permissible disclosures, and the channels to which the whistleblower can report. Legislators must also balance significant trade-offs between protecting employees who disclose meaningful information and safeguarding companies’ legitimate interests in trade secrets. 

While the TFAIA is a step in the right direction, it does not go far enough in protecting whistleblowers from AI labs. The AIWPA is superior because it protects a larger group of would-be whistleblowers for a wider range of disclosures, while also safeguarding employers’ legitimate interests. As such, Congress should prioritize its enactment. 

The TFAIA’s Whistleblower Protections

The TFAIA’s whistleblower provisions are primarily intended to cover some of the world’s most powerful and well-resourced AI labs. Many of the law’s protections apply only to certain employees of “frontier developers,” that is, AI labs developing models using a very high amount of computing power. In some cases, the law applies only to “large frontier developers,” namely those frontier developers that exceed $500 million in annual revenue. According to some estimates, only a small handful of labs — such as OpenAI — meet these computing and revenue thresholds, though more likely will soon. 

Moreover, the law only covers a relatively narrow subset of employees at these labs. Specifically, the law covers all California-based employees who are “responsible for assessing, managing, or addressing risk of critical safety incidents.” It is unclear, however, which employees fit this description. As one critic of the law has noted, however, the TFAIA clearly excludes “numerous highly qualified whistleblowers…such as contractors, consultants, external auditors, and company employees whose work does not directly address the risk of critical safety incidents.” 

The scope of permissible disclosures is also relatively narrow. The TFAIA protects covered employees when they have “reasonable cause to believe” that either 1) their employer has violated other provisions of the TFAIA or 2) their employer’s “activities pose a specific and substantial danger to the public health or safety resulting from a catastrophic risk.”

These categories of protected disclosures are narrow because the law has a very limited definition of “catastrophic risk.” A catastrophic risk refers to “a foreseeable and material risk” that the employer’s model will “materially contribute to” the death or serious injury of more than 50 people, or alternatively, cause over $1 billion in property damage or loss. Moreover, something only qualifies as a “catastrophic risk” under the statute if such death, injury, or damage results from a limited class of behaviors, such as an AI model “providing expert-level assistance in the creation or release of a chemical, biological, radiological, or nuclear weapon,” engaging in certain behaviors “with no meaningful human oversight,” or “evading control of the frontier developer or user.” 

Notwithstanding these narrow provisions, the law is relatively broad in other key respects. For one, it covers not only internal “rule[s], regulation[s], and polic[ies]” that would prohibit whistleblowing, but also “contracts.” Importantly, this means that covered employers cannot eviscerate whistleblower protections by asking employees to sign broad non-disclosure or non-disparagement agreements. The TFAIA also broadly prevents “retaliation” against covered whistleblowers and provides several channels — including certain federal officials, the Attorney General, and certain coworkers — to which whistleblowers can disclose.

Why the AIWPA is Still Needed

The AIWPA differs from the TFAIA in two key respects. 

First, the AIWPA covers a much broader swath of individuals. The AIWPA covers all current and former employees and independent contractors; unlike the TFAIA, it is not limited to “frontier developers,” nor is it limited to AI labs in general. 

Second, the AIWPA has a much wider scope of permissible disclosures than the TFAIA. The AIWPA prohibits a wide variety of employer behavior — such as discharging, demoting, suspending or otherwise discriminating when an individual lawfully provides “information regarding an AI security vulnerability or AI violation” or any conduct that the individual “reasonably believes constitutes an AI security vulnerability or AI violation.”

The AIWPA defines these two key terms — “AI security violation” and “AI violation” — very broadly. The former refers to security lapses that might allow an AI model to be easily stolen, while the latter refers to either 1) “any violation of Federal law… occurring during AI development, deployment, or use,” or 2) “any failure to appropriately respond to a substantial and specific danger” to “public safety, public health, or national security.” 

The scope of permissible disclosures under the AIWPA is therefore much broader than those under the TFAIA. This matters because, in some important cases, an employee’s disclosure won’t meet the TFAIA’s high bar. Imagine, for instance, that an employee had disclosed what we now know: that some AI models can induce emotional attachments from users and contribute to psychotic episodes. This would likely not meet TFAIA’s stringent standard for a “catastrophic risk,” but it is certainly a “substantial and specific danger” to “public health” under the AIWPA’s more capacious definition. 

Conclusion

Whistleblower laws are not the only legal protections that might benefit employees who disclose AI risks. For example, if an employer fired an employee who disclosed a serious risk, an employee might bring a successful claim for wrongful termination in violation of public policy. But to meaningfully incentivize whistleblowing, the law must assure employees that they won’t be fired in the first place.

The TFAIA offers an important foundation, but it is insufficient. Many would-be whistleblowers may still be deterred because of uncertainty about whether their disclosures are permitted under the law. To better protect these individuals, Congress should enact the AIWPA.