SACRAMENTO – Senator Scott Wiener (D-San Francisco) announced amendments to expand Senate Bill (SB) 53 into a first-in-the-nation transparency requirement for the largest AI companies. The new provisions draw on the recommendations of a working group led by some of the world’s leading AI experts and convened by Governor Newsom. Building on the report’s “trust, but verify” approach, the amended bill requires the largest AI companies to publicly disclose their safety and security protocols and report the most critical safety incidents to the California Attorney General. The requirements codify voluntary agreements made by leading AI developers to boost trust and accountability and establish a level playing field for AI development.
SB 53 retains provisions — called “CalCompute” — that advance a bold industrial strategy to boost AI development and democratize access to the most advanced AI models and tools. CalCompute will be a public cloud compute cluster housed at the University of California that provides free and low-cost access to compute for startups and academic researchers. CalCompute builds on Senator Wiener’s recent legislation to boost semiconductor and other advanced manufacturing in California by streamlining permit approvals for advanced manufacturing plants, and his work to protect democratic access to the internet by authoring the nation’s strongest net neutrality law.
SB 53 also retains its protections of whistleblowers at AI labs who disclose significant risks.
Weeks ago, the U.S. Senate voted 99-1 to remove provisions of President Trump’s “Big Beautiful Bill” that would have prevented states from enacting AI regulations. By boosting transparency, SB 53 builds on this vote for accountability.
“As AI continues its remarkable advancement, it’s critical that lawmakers work with our top AI minds to craft policies that support AI’s huge potential benefits while guarding against material risks,” said Senator Wiener. “Building on the Working Group Report’s recommendations, SB 53 strikes the right balance between boosting innovation and establishing guardrails to support trust, fairness, and accountability in the most remarkable new technology in years. The bill continues to be a work in progress, and I look forward to working with all stakeholders in the coming weeks to refine this proposal into the most scientific and fair law it can be.”
As AI advances, risks and benefits grow
Recent advances in AI have delivered breakthrough benefits across several industries, from accelerating drug discovery and medical diagnostics to improving climate modeling and wildfire prediction. AI systems are revolutionizing education, increasing agricultural productivity, and helping solve complex scientific challenges.
However, the world’s most advanced AI companies and researchers acknowledge that as their models become more powerful, they also pose increasing risks of catastrophic damage. The Working Group report states:
Evidence that foundation models contribute to both chemical, biological, radiological, and nuclear (CBRN) weapons risks for novices and loss of control concerns has grown, even since the release of the draft of this report in March 2025. Frontier AI companies’ [including OpenAI and Anthropic] own reporting reveals concerning capability jumps across threat categories.
To address these risks, AI developers like Meta, Google, OpenAI, and Anthropic have entered voluntary commitments to conduct safety testing and establish robust safety and security protocols. Several California-based frontier AI developers have designed industry-leading safety practices including safety evaluations and cybersecurity protections. SB 53 codifies these voluntary commitments to establish a level playing field and ensure greater accountability across the industry.
Background on the report
Governor Newsom convened the Joint California Policy Working Group on AI Frontier Models in September 2024, following his veto of Senator Wiener’s SB 1047, tasking the group to “help California develop workable guardrails for deploying GenAI, focusing on developing an empirical, science-based trajectory analysis of frontier models and their capabilities and attendant risks.”
The Working Group is led by experts including the “godmother of AI” Dr. Fei-Fei Li, Co-Director of the Stanford Institute for Human-Centered Artificial Intelligence; Dr. Mariano-Florentino Cuéllar, President of the Carnegie Endowment for International Peace; and Dr. Jennifer Tour Chayes, Dean of the UC Berkeley College of Computing, Data Science, and Society.
On June 17, the Working Group released their Final Report. While the report does not endorse specific legislation, it promotes a “trust, but verify” framework to establish guardrails that reduce material risks while supporting continued innovation.
SB 53 balances AI risk with benefits
Drawing on recommendations of the Working Group Report, SB 53:
- Establishes transparency into large companies’ safety and security protocols and risk evaluations. Companies will be required to publish their safety and security protocols and risk evaluations in redacted form to protect intellectual property.
- Mandates reporting of critical safety incidents (e.g., model-enabled CBRN threats, major cyber-attacks, or loss of model control) within 15 days to the Attorney General.
- Protects employees and contractors who reveal evidence of critical risk or violations of the act by AI developers.
The bill's provisions apply only to a small number of well-resourced companies, and only to the most advanced models. The Attorney General has the power to update the thresholds governing which companies are covered under the bill to ensure the requirements keep up with rapid advancements in the field, but must cover only well-resourced companies at the frontier of AI development.
Under SB 53, the Attorney General imposes civil penalties for violations of the act. SB 53 does not impose any new liability for harms caused by AI systems.
In addition, SB 53 creates CalCompute, a research cluster to support startups and researchers developing large-scale AI. The bill helps California secure its global leadership as states like New York establish their own AI research clusters.
SB 53 is sponsored by the Encode AI, Economic Security Action California, and the Secure AI Project.
SB 53 is supported by a broad coalition of researchers, industry leaders, and civil society advocates:
“California has long been the birthplace of major tech innovations. SB 53 will help keep it that way by ensuring AI developers responsibly build frontier AI models,” said Sneha Revanur, president and founder of Encode AI, a co-sponsor of the bill. “This bill reflects a common-sense consensus on AI development, promoting transparency around companies’ safety and security practices.”
“At Elicit, we build AI systems that help researchers make evidence-based decisions by analyzing thousands of academic papers,” said Andreas Stuhlmüller, CEO of Elicit. “This work has taught me that transparency is essential for AI systems that people rely on for critical decisions. SB53's requirements for safety protocols and transparency reports are exactly what we need as AI becomes more powerful and widespread. As someone who's spent years thinking about how AI can augment human reasoning, I believe this legislation will accelerate responsible innovation by creating clear standards that make future technology more trustworthy.”
"I have devoted my life to advancing the field of AI, but in recent years it has become clear that the risks it poses could threaten us all,” said Geoffrey Hinton, University of Toronto Professor Emeritus, Turing Award winner, Nobel laureate, and a “godfather of AI.” “Greater transparency requirements into how companies are addressing safety concerns from the most powerful technology of our time is an important step towards addressing those risks."
"SB 53 is a smart, targeted step forward on AI safety, security, and transparency,” said Bruce Reed, Head of AI at Common Sense Media. "We thank Senator Wiener for reinforcing California’s strong commitment to innovation and accountability.”
"AI can bring tremendous benefits, but only if we steer it wisely. Recent evidence shows that frontier AI systems can resort to deceptive behavior like blackmail and cheating to avoid being shut down or fulfill other objectives,” said Yoshua Bengio, Full Professor at Université de Montréal, Co-President and Scientific Director of LawZero, Turing Award winner and a “godfather of AI.” “These risks must be taken with the utmost seriousness alongside other existing and emerging threats. By advancing SB 53, California is uniquely positioned to continue supporting cutting-edge AI while proactively taking a step towards addressing these severe and potentially irreversible harms."
“Including safety and transparency protections recommended by Gov. Newsom’s AI commission in SB 53 is an opportunity for California to be on the right side of history and advance commonsense AI regulations while our national leaders dither,” said Teri Olle, Director of Economic Security California Action, a co-sponsor of the bill. “In addition to making sure AI is safe, the bill would create a public option for cloud computing – the critical infrastructure necessary to fuel innovation and research. CalCompute would democratize access to this powerful resource that is currently enjoyed by a tiny handful of wealthy tech companies, and ensure that AI benefits the public. With inaction from the federal government – and on the heels of the defeat of the proposed 10-year moratorium on AI regulations – California should act now and get this done.”
“The California Report on Frontier AI Policy underscored the growing consensus for the importance of transparency into the safety practices of the largest AI developers,” said Thomas Woodside, Co-Founder and Senior Policy Advisor, Secure AI Project, a co-sponsor of the bill. “SB 53 ensures exactly that: visibility into how AI developers are keeping their AI systems secure and Californians safe.”
“Reasonable people can disagree about many aspects of AI policy, but one thing is clear: reporting requirements and whistleblower protections like those in SB 53 are sensible steps to provide transparency, inform the public, and deter egregious practices without interfering with innovation,” said Steve Newman, Technical co-founder of eight technology startups, including Writely – which became Google Docs, and co-creator of Spectre, one of the most influential video games of the 1990s.
###
English (US) ·