One new benchmark wey dem build from real, paid expert jobs don just grade twenty frontier AI models, and di best among dem score 12 percent.
Fit AI agent do di work wey your business dey pay experts to do? How honest scoreboard for dat kain claim go look? And wetin you suppose ask vendor before you sign?
As private AI model developer and AI security consultant wey dey work with SMEs and big companies for Sydney, dem dey ask me dis questions every week. Di answers matter for budget, staffing and risk.
Dis article go explain wetin di GDPx benchmark measure, wetin im numbers prove and wetin e no prove, and di practical lessons for any leader wey dey evaluate AI agents dis quarter.
Dis week, di research company SemiAnalysis publish GDPx, one benchmark of 206 tasks across 11 domains wey dem choose because na so pipo dey take earn dia living: chip design, cybersecurity, legal work, quantitative trading, biology, software engineering and more.
Fourteen industry partners supply di tasks, and dem grade twenty frontier models. Di benchmark dey special no be because of how hard e be, but because of wetin e take build: work wey pipo dey collect money to do.
Di leader, Claude Opus 5.5, manage only 12 percent. Gemini 4 Carbon score 10.9. GPT-6 Astra score 9.6.
However, scoreboard na starting point, no be verdict. Dis numbers dey measure one firm tasks under dat same firm rubric, and dem dey describe today models, no be next year own.
GDPx no perfect. E use one firm tasks and one firm rubric, nobody don reproduce am independently yet, and di firm dey sell research give di same industry wey e dey grade.
Di lesson na to hold di number loose. But di direction correct, because e dey measure models on work wey dey pay, no be trivia wey dey impress.
Di leader average about $51 for each completed task. Di runner-up average about $16 for score wey dey within one point of am.
If your agentic strategy assume say capability and cost dey move together, dis table dey talk say no be so. Di cheapest road to result don dey as strategic as di result itself, and dat analysis suppose dey for procurement, no be for marketing page.
All di demo wey you go see dis quarter go claim say dem ready for expert work. One public, named benchmark just put di ceiling for 12 percent.
Dat no mean say agents useless. E mean say di honest deployment zone today na narrow, well-instrumented tasks with human review, no be di autonomous departments wey dem dey sell for stage.
First, di demo era don finish. When industry begin dey argue about scoreboards, demo don stop to dey settle arguments.
Second, cost and capability no dey move together. Put cost per completed task inside procurement, beside di score.
Third, read every agent pitch against one public number. Di honest deployment zone today dey narrow, well-instrumented and human-reviewed.
Finally, ask every AI vendor one question dis quarter: show me your score for independent, task-based benchmark for work wey be like my own, and your cost per completed task. If dem no fit answer, you don learn something wey pass demo value.
A 12 percent ceiling no be reason to waka comot from AI agents. Na reason to buy dem with evidence instead of excitement.
Artificial intelligence go earn im place with results wey pesin fit measure, no be stage performance. For di end, na di mathematics of cost per completed task, honestly counted, go decide where dis technology truly belong.
SemiAnalysis, "GDPx: The General Agentic Benchmark" - https://gdpx.semianalysis.com
SemiAnalysis, research notes and industry analysis - https://semianalysis.com
Think am mathematically. Engineer am well-well. Lead with strategy. Build am responsibly. Scale am globally. Use artificial intelligence create real business value.
