AI companies keep telling us their agents can "do real work." This week, a research firm actually tested that claim the hard way — and the best AI model in the world completed about one task in eight. The benchmark is called GDPx, it comes from SemiAnalysis, and if you use AI tools for anything that matters, its results should change how you work with them.
What GDPx does differently
Most AI benchmarks are quizzes: short questions, quick answers. GDPx, published on 2 November 2026 by SemiAnalysis researchers Max Kan, Andrew Megalaa and Dylan Patel, instead drops an AI agent into a simulated professional workplace — real files, real tools, real mess — and gives it up to eight hours to finish an actual job. There are 206 tasks across 11 high-value domains: chip design, cybersecurity, biology, finance, legal work, quantitative trading, hardware design, materials science, software engineering, alignment, and even recursive self-improvement. The environments were built with 14 specialist partner firms; the finance tasks, for example, reconstruct a real $880 million company acquisition across roughly 4,000 files. Each model gets three attempts per task, and the headline score is its first-try success rate, with all domains weighted equally.
The leaderboard
From the live leaderboard at gdpx.semianalysis.com:
- Claude Opus 5.5 (Anthropic) — 12.0%, at an average cost of $51.41 per task
- Gemini 4 Carbon (Google) — 10.9%, at $16.29 per task
- GPT-6 Astra (OpenAI) — 9.6%, at $53.99 per task
- Claude Fable 5.1 (Anthropic) — 8.5%, at $67.16 per task
- GPT-6.1 Sol (OpenAI) — 8.4%, at just $9.80 per task
- DeepSeek 4.1 Flash (DeepSeek, open) — 5.4%, at $0.94 per task
Two domains humbled everyone: in finance and legal, every model scored 0% under strict pass/fail grading. And SemiAnalysis is upfront about one distortion — the top two American models lost points in cybersecurity, biology and alignment tasks partly because their own safety systems refused or cut short the work.
Read it fairly: the caveats
GDPx is deliberately brutal. These are expert-level tasks chosen because they are hard, graded strictly, by one research firm, using mostly private tasks that outsiders cannot fully reproduce, with only three attempts per task. A 12% score does not mean an agent fails 88% of ordinary office work — drafting, summarising, coding help and research are far easier than reconstructing a billion-dollar acquisition. What GDPx measures is the distance between "impressive assistant" and "autonomous professional," and today that distance is still large.
What this means when you choose AI tools
- Buy an assistant, not an employee. Plan every agent workflow with a human checking the output. GDPx is your evidence that unsupervised agents are not there yet — in finance and legal, not even close.
- Judge cost per finished task, not per token. Gemini 4 Carbon scored nearly as well as the leader at a third of the cost per task, and GPT-6.1 Sol is close behind at under $10. The cheapest model per message is often not the cheapest per result.
- Cheap models earn their place. DeepSeek's open model scored 5.4% at 94 cents a task. For bulk, low-stakes work, that trade can be the smart one.
- Test on your own work. A benchmark's tasks are not your tasks. Give any agent a real job from your week, grade it yourself, and then decide what it may do alone.
- Watch this board. Scores this low will not stay low. The leaderboard is public and free — checking it every few months is the easiest way to see whether "agents" are becoming real.
The headline sounds like bad news for AI. It is actually the most useful kind of news: a map of where the technology really stands, drawn by people who graded the work instead of the marketing.
Sources: SemiAnalysis, "GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains" (2 November 2026) and the live GDPx leaderboard at gdpx.semianalysis.com, checked 4 November 2026; TechTimes' coverage of the launch.
