According to Sam Altman, CEO of OpenAI, the company is collaborating with the U.S. AI Safety Institute, a federal government organization that evaluates and manages risks associated with AI platforms, on a deal to get early access to its upcoming major generative AI model for safety testing.
Altman made the announcement late on Thursday night in a post on X, although it lacked specifics. Nonetheless, it seems to be meant to refute the claim that OpenAI has placed less emphasis on AI safety in favor of developing more potent, capable generative AI technologies, as seen by this agreement and another one reached in June with the UK’s AI safety committee.
A team focusing on creating safeguards to stop “superintelligent” AI systems from going rogue was essentially disbanded by OpenAI in May. Both of the team’s co-leads, Ilya Sutskever, who founded Safe Superintelligence Inc. and Jan Leike, who now oversees safety research at AI startup Anthropic, resigned from OpenAI as a result of reports, that the company had neglected the team’s safety research in favor of introducing new products.
A growing number of opponents forced OpenAI to declare that it would abolish its harsh non-disparagement restrictions, which tacitly discouraged whistleblowing, establish a safety commission, and commit 20% of its computing power to safety research. (The dissolved safety team never got the 20% of OpenAI’s compute that was promised to them for their work.) Altman restated the 20% commitment and confirmed that OpenAI terminated the non-disparagement terms for both new hires and current employees in May.
But other observers were not impressed by the actions, especially after OpenAI filled the safety committee with only business insiders, including Altman, and more recently moved a senior AI safety executive to a different organization.
In a recent letter to Altman, five senators—among them Hawaii Democrat Brian Schatz—posed concerns over OpenAI’s practices. Responding to the letter today, OpenAI Chief Strategy Officer Jason Kwon stated that the company “[is] dedicated to implementing rigorous safety protocols at every stage of our process.”
The timing of OpenAI’s agreement with the U.S. AI Safety Institute appears suspicious, given the company’s support earlier this week for the Future of Innovation Act, a proposed Senate bill that would establish the Safety Institute as an executive body that sets standards and guidelines for AI models. The moves could be interpreted as an attempt at regulatory capture — or, at the very least, an effort by OpenAI to exercise influence over AI policymaking at the federal level.
Not for nothing, Altman is a member of the Artificial Intelligence Safety and Security Board of the Department of Homeland Security in the United States. This board makes recommendations for the “safe and secure development and deployment of AI” throughout the vital infrastructures of the country. Additionally, OpenAI has significantly upped its federal lobbying budget this year, spending $800,000 in the first half of 2024 as opposed to $260,000 in 2023.
A group of businesses that includes Anthropic as well as major tech companies like Google, Microsoft, Meta, Apple, Amazon, and Nvidia seek advice from the U.S. AI Safety Institute, which is based under the National Institute of Standards and Technology of the Commerce Department. Establishing criteria for AI red-teaming, capability assessments, risk management, safety and security, and watermarking synthetic content are among the tasks assigned to the industry group by President Joe Biden’s October AI executive order.
