Anthropic’s Claude Fable 5.2 just posted the highest score yet on the Remote Labor Index, a benchmark that tests whether AI agents can finish real paid freelance projects. The model completed 41.3% of the jobs at a level a client would accept, roughly double the next-best system. A year ago, the top agent on the same test managed just 2.5%.
What the Remote Labor Index Actually Measures
Most AI benchmarks check a single skill, like answering a math question or fixing one bug. The Remote Labor Index, built by Scale Labs and the Center for AI Safety, takes a different path. It hands agents complete projects that real freelancers were once paid to do on Upwork.
The full set contains 240 projects collected from 358 verified freelancers across 23 work categories. These are not small tasks. The average project took a human almost 29 hours and paid about $632, while the median was 11.5 hours and $200.
Agents work inside a virtual workspace with familiar professional software such as Blender, GIMP and Audacity. They also get AI tools for making images, speech and video. Scoring is done by trained human reviewers, who open the files and compare them against the work the original freelancer delivered.
How a “pass” is decided
The main score is the automation rate. A project only counts if reviewers judge the AI’s work at least as good as the human standard, meaning a reasonable client would accept it. Three independent evaluators vote on each project, and Scale reports 94.4% agreement among them.
Fable 5.2’s Remote Labor Index Score Versus Rivals
On the updated leaderboard, Fable 5.2 sits alone at the top with a 41.30% automation rate. Its own predecessor, Fable 5.1, scored 17.92%, so the new version finishes more than twice as many jobs. Here is how the top of the board looks right now.
| Rank | Model | Automation rate |
|---|---|---|
| 1 | Claude Fable 5.2 | 41.30% |
| 2 | Claude Opus 5.5 | 21.25% |
| 3 | GPT 6 Astra | 20.83% |
| 4 | Claude Fable 5.1 | 17.92% |
| 5 | Gemini Argon 4 | 17.08% |
| 6 | Claude Fable 5 | 15.83% |
The pace of change is the real story. When the leaderboard launched, the best agent, Manus, cleared only 2.5% of projects. Older models like GPT-5.2 still score under 3% today, which shows how fast the newest frontier systems have pulled away.
Where the new model made its biggest gains
According to Scale Labs, the biggest jumps came in a few creative and technical areas that used to stump agents:
- Editing audio and music tracks
- Designing architecture and interior spaces
- Building 3D models
- Handling long, complex projects that earlier top models often failed
Scale also says nearly 50 of the projects Fable 5.2 completed had never been solved by any agent before. Industrial product design and animation remain weak spots.
Why Agents Still Fail Most Jobs
Even at 41%, the model still falls short on most of the work. The benchmark’s own analysis lists the common ways agents miss the mark. The biggest problem is amateur-quality output, followed by incomplete deliverables like truncated videos or missing files.
Other failures include corrupt or wrong file formats and designs that do not stay consistent, such as a house that changes shape between 3D views. Reviewers also found that AI judges get fooled by work that looks polished but is technically broken. That is why the test still relies on human experts, which makes it slow and costly to run.
There are limits to keep in mind too. The benchmark skips work that needs client calls, physical labor or long-term results like SEO. So it captures a slice of remote work, not the whole job market.
What This Means for You
If you hire freelancers or do freelance work yourself, this result is worth watching. It does not mean AI can replace a skilled pro, but it does show agents are getting better at full projects, not just snippets. Here are a few practical takeaways:
- Try agents on well-defined creative chores first, like cleaning up audio or drafting simple 3D assets, then review the output closely.
- Always open and check the actual files, since polished-looking results can hide broken parts.
- Keep humans on client-facing, judgment-heavy and long-running work, which the test does not cover.
- Compare scores across tools before paying for a premium model, because the gap between first and second place is now large.
If you are new to Anthropic’s lineup, our guide to Claude AI basics is a good starting point. We also covered the Fable 5.2 launch earlier and how the model designs working electronics. For another look at agents struggling with real tasks, see how leading models handled AI shopping tests.
A jump from 2.5% to 41% in about a year shows agents moving from demos to real deliverables, but human review still decides what counts as done.
You can follow more updates like this on our daily AI news page.
Sources
Primary source: Scale Labs Remote Labor Index leaderboard. Additional reporting from The Decoder.
Conclusion
Claude Fable 5.2 has set a new high on the Remote Labor Index by completing 41.3% of real freelance projects to a client-ready standard. That is about twice the score of any rival and a huge leap from the 2.5% top score when the test began. Most jobs still end in failure, and the benchmark leaves out many kinds of work. Still, the trend is clear: AI agents are getting much better at finishing whole projects, and that is worth tracking closely.
