OpenAI has reduced the time and resources spent on testing its powerful AI models for safety, raising concerns that the company may be releasing technologies too quickly and without sufficient safeguards.
Shortened Review Cycles Raise Concerns
OpenAI and its partners now have just a few days to conduct risk and performance assessments of AI models, compared to the months previously allocated. According to the Financial Times, which cited eight people familiar with the matter, the company’s internal review mechanisms have become less thorough, and fewer resources are being devoted to identifying and mitigating threats. As OpenAI competes in an increasingly crowded market and seeks to maintain its $300 billion valuation, it is under pressure to release new models faster. While the capabilities of its large language models grow, so does their potential for misuse—yet demand is rising, pushing OpenAI to accelerate deployment.
Though there are no global standards for testing AI safety, the European “AI Act,” coming into effect this year, will require developers to test their most powerful models. Previously, some companies—including OpenAI—voluntarily agreed to allow UK and US authorities to involve third-party researchers in safety assessments. With the release of OpenAI’s new model o3 expected next week, the window for testing will be less than a week, although the launch date may change. We’ll keep you updated as more information becomes available.
Reduced Transparency and Incomplete Reports
The company has never previously allocated so little time to safety. When GPT-4 was released in 2023, testing took around six months. One participant involved in the evaluations said some dangerous capabilities were discovered only two months into testing. OpenAI has committed to creating special versions of its systems for safety testing—such as determining whether the model could help make a biological virus more contagious—a process requiring specialized data and significant fine-tuning resources.
However, this commitment is only partially fulfilled. Older and less powerful models are fine-tuned, while advanced ones are sometimes ignored. For instance, the security report for the o3-mini model, released in January, includes references to the earlier GPT-4o, and some models like o1 and o3-mini lack published testing data entirely. OpenAI has responded by stating that its evaluation methods have become more efficient through automation, reducing timelines without compromising quality. While there’s no industry-wide standard for approaches like fine-tuning, the company maintains that its methods are effective and applied with full transparency.
Testing on Incomplete Models
Another issue is that safety assessments often focus on “checkpoints,” or earlier versions of models, rather than the final public releases, adds NIXSolutions. These checkpoints are later updated to improve performance and introduce new features. OpenAI’s reports often link to “near-final” versions and claim that these are “mostly identical” to what users eventually receive.
We’ll keep you updated as more integrations and safety measures develop alongside new model releases.