10-01-2025 
|
Getting your Trinity Audio player ready...
|
OpenAI, one of the world’s leading artificial intelligence (AI) developers, has introduced a new evaluation tool designed to help track the performance of AI models on real-world tasks. It found that human-produced work still outperforms AI in quality when rated against the leading AI models, but the difference is increasingly narrow.
On September 25, OpenAI announced the evaluation tool ‘GDPval,’ which examined 44 occupations selected from the top nine industries contributing to the United States gross domestic product (GDP). Based on the results, human work was still rated higher in quality in at least 50% of the tasks compared to the various AI models.
The best performing model, Claude Opus 4.1, won or drew in 47% of tasks when compared to the human output. Meanwhile, GPT‑5 “excelled in particular on accuracy,” winning or drawing in 38.8% of tasks.
The evaluation compared the output of industry experts across 220 tasks—representing the types of “day-to-day work” where AI can meaningfully assist professionals—with the output of several leading AI models, namely GPT‑4o, o4-mini, OpenAI o3, GPT‑5, Claude Opus 4.1, Gemini 2.5 Pro, and Grok 4. The quality of the output was then vetted by professionals with over 14 years of experience in the fields being tested.
The results offer some reassurance to those concerned about losing their jobs to AI, concluding that humans still have the edge in quality, for now. However, considering the rapid pace of AI’s evolution since its inception, it is clearly catching up.
The evaluation also revealed that AI was already far outstripping humans in other criteria. Specifically, frontier AI models can complete GDPval tasks “roughly 100x faster and 100x cheaper than industry experts,” indicating that, if-and-when the technology does catch up, businesses may be left with an obvious cost-cutting choice.
“As AI becomes more capable, it will likely cause changes in the job market,” said OpenAI. “Early GDPval results show that models can already take on some repetitive, well-specified tasks faster and at lower cost than experts.”
However, the company qualified this by adding that “most jobs are more than just a collection of tasks that can be written down. GDPval highlights where AI can handle routine tasks so people can spend more time on the creative, judgment-heavy parts of work.”
This, suggested OpenAI, could be the future of the workplace. Rather than AI simply replacing human jobs, it can be used to complement and enhance human work.
“When AI complements workers in this way it can translate into significant economic growth,” said the company. “Our goal is to keep everyone on the “up elevator” of AI by democratizing access to these tools, supporting workers through change, and building systems that reward broad contribution.”After examining the evaluation results, OpenAI said that they incrementally trained an internal, experimental version of GPT‑5 to assess whether they could improve its performance. They found this process did produce better results for the model on the same evaluation, thus “creating a pathway for further potential improvement.”
According to OpenAI, GDPval represents the first version of this evaluation, which will be ongoing. It also noted that GDPval builds on previous evaluations, from Massive Multitask Language Understanding (MMLU) using exam-style questions across dozens of subjects, to more “applied evaluations” such as SWE-Bench (software engineering bug-fixing tasks) and Paper-Bench (scientific reasoning and critique on research papers).
“People often speculate about AI’s broader impact on society, but the clearest way to understand its potential is by looking at what models are already capable of doing,” said OpenAI. “Previous AI evaluations like challenging academic tests and competitive coding challenges have been essential in pushing the boundaries of model reasoning capabilities, but they often fall short of the kind of tasks that many people handle in their everyday work.”
In contrast, GDPval models’ performance on tasks drawn “directly from the real-world knowledge work of experienced professionals across a wide range of occupations and sectors, providing a clearer picture on how models perform on economically valuable tasks,” said the company.
OpenAI said it plans to expand GDPval to include more occupations, industries, and task types, with increased interactivity, as well as more tasks involving navigating ambiguity. The long-term goal is to better measure the progress of AI on “diverse knowledge” work.
“GDPval is an early step that doesn’t reflect the full nuance of many economic tasks,” said OpenAI. “Future versions will extend to more interactive workflows and context-rich tasks to better reflect the complexity of real-world knowledge work.”
In order for artificial intelligence (AI) to work right within the law and thrive in the face of growing challenges, it needs to integrate an enterprise blockchain system that ensures data input quality and ownership—allowing it to keep data safe while also guaranteeing the immutability of data. Check out CoinGeek’s coverage on this emerging tech to learn more why Enterprise blockchain will be the backbone of AI.
Watch: Can blockchain keep AI in check?
Recommended for you
