At Robust Data Solutions we put autopilots with human oversight into companies' software. That's why we were so interested in the study that Mercor, a US company that measures how AI performs on professional tasks, published on October 1: on four test month-end closes, the AI did a perfect job, while certified accountants averaged 37%. That's true. But the same study, read in full, explains why AI can't work on its own yet.
Data checked as of October 22, 2026, against the Mercor study and its public APEX-Accounting leaderboard.
What Mercor measured
12 certified accountants in the United States, with an average of 5.4 years of experience, completed four simplified month-end close tasks: budget versus actuals for a nonprofit, a hotel's December revenue, the variance for an event venue and the leases of three companies in a group.
| Accountants alone | AI alone (Claude Opus 5) | |
|---|---|---|
| Criteria met | 37% on average (from 0% to about 90%) | 100% on all 20 attempts |
| Time per task | Mostly between 30 and 180 minutes | Under 10 minutes |
| Cost per criterion met | $10.35, at the median wage of a US accountant | $0.21, at list prices (an upper bound) |
AI comes out about 49 times cheaper. And that's counting only salary: what an hour of work really costs a company is considerably more.
From almost zero to almost everything in two years
The most striking thing about the study isn't the snapshot: it's the movie. Mercor ran the same tasks on the models available at each point:
- In mid-2024, GPT-4o, the best available at the time, scored close to 0%.
- In spring 2025, o3 was already beating the accountants' average (37 %).
- GPT-5 reached about 69%.
- Today, Claude Opus 5 scores 100% on all 20 attempts, and other leading models are very close.
In about two years, AI has gone from being unable to do a test month-end close to doing it better than the average professional. And the curve hasn't stopped. That's why the sensible move isn't to wait and see, but to start preparing now how you'll work with it: what the AI does, what a person reviews and with what data.
What the headline leaves out
Mercor puts it bluntly: these results don't mean accountants are replaceable. And it explains why:
- The tasks were simplified, and the accountants were given a map of the files so they wouldn't waste time searching.
- There was no one to ask: no coworkers, no client, none of the context of someone who knows the company. One of them summed it up: “In reality I can usually ask my client questions as I work.”
- The tasks were designed to make the AI fail, with traps that build on each other: one number that slips through sinks the score.
And then there's the full exam. In APEX-Accounting, with 100 tasks across 10 simulated companies and no file map, the best model meets 71.5% of the criteria, but gets fewer than one in four tasks perfect on the first try. Mercor sums it up this way: AI agents can't yet reliably close the books.
It's a study by Mercor itself, with 12 people in the United States. And the AI's cost doesn't include anyone to review its work.
What is an autopilot with human oversight?
The conclusion isn't “AI is useless” or “AI does it all.” It's that AI prepares very fast and very well, and someone has to review it before it counts. We call it Robust Autopilots: autopilots that handle the repetitive work on their own, always with a person in charge.
Like an airplane's, it holds the course it has been set and, if something unexpected happens, it alerts you and hands back control. In a company, that means five things:
- It only does what it has been programmed to do. It doesn't improvise.
- If something isn't anticipated, it stops and alerts the right person.
- Every step is logged: who, what and when.
- Permissions per person: if you can't do something in your software, you can't do it through AI either.
- What matters goes through a person. Paying, sending a batch of emails or deleting: you see what it's going to do and you approve it. Routine tasks you've already approved, like an appointment reminder, go out on their own, are logged and can be stopped.
In the study, the accountants who worked with the AI took about 15 times longer than the AI alone, without improving the score: if the AI is already 100% right, reviewing only adds time. In real work it isn't always right, which is why we design the review: the AI leaves the work ready and flags anything doubtful; the person approves what's correct in one go and fixes the rest.
Finding the data is half the job
Even with the file map, one accountant said “there was enough info to do almost any task, but that means I had to sort through all of it.” In a real company, the data is spread across the accounting software, the ERP, email and folders.
That map is what we build: a custom smart connector for each program, which knows where every piece of data is and what rules your company works by. And all of them under a single AI, with each person's permissions.
A chain of hearing care centers, with accounting on autopilot
- Sales: hundreds of invoices a month flow from the clinical software into the accounting system without being retyped. If five out of fifty prepared journal entries are doubtful, a person approves the forty-five correct ones and fixes the other five in the same panel.
- Purchase invoices: they're read automatically and go through the review queue before being posted, with the original document attached.
- VAT and collections: before each quarter, VAT reconciled with any anomalies flagged; with the bank statement, the system proposes which payments match and a person approves them.
- A control center: what has run, who approved it and what's still pending, item by item.
We do the same in-house: supplier invoices that reach us by WhatsApp or email are prepared in the accounting system, with the document attached, and don't count until a person approves them.
The AI prepares, the person signs
For a tax or accounting advisory firm, this isn't a threat: it's extra capacity for the busy seasons. The AI prepares each close with anything pending flagged, and the person in charge reviews and files. We cover it in AI for Tax and Accounting Advisory Firms.
Our AI never decides on its own: it proposes, and a person always makes the final decision. That’s what we call Human in the Loop. And every project goes through a regulatory compliance review with our partner Forrellat Consultors, a Data Protection Officer registered with the Spanish Data Protection Agency (AEPD).
More in AI agents with a person in charge. We’ve published the Robust Autopilots page.
If your company has a close, a reconciliation or a process that repeats every month, tell us how you handle it and we'll show you what it would look like with an autopilot and a person in charge.
Frequently Asked Questions
Can AI already do the accounting on its own?
No. In the Mercor study it did four simplified closing tasks perfectly, with the information organized and clear instructions. In Mercor's full test, the best model gets fewer than one in four tasks perfect on the first try, and Mercor itself says AI agents can't yet reliably close the books. What it already does very well is prepare: search, cross-check, calculate and leave the work ready for a person to review.
What exactly is an autopilot with human oversight?
An automated process that only does what it has been programmed to do, stops and alerts someone when something isn't anticipated, logs every step and leaves anything important pending a person's approval. We call them Robust Autopilots: autopilots that handle the repetitive work on their own, always with a person in charge.
What always goes through a person?
Anything sensitive. No sensitive action—money, contracts, customer data—runs without confirmation: paying, sending a batch of emails or deleting is approved first. Routine tasks the company has already approved, like an appointment reminder, go out on their own, but they're logged and can be stopped at any time. Your company decides what goes on each side, and it can be changed from the panel.
What if the person reviewing makes a mistake?
It can happen, and the study shows it: in three of the four AI-assisted sessions that didn't come out perfect, the AI had the correct figure at some point, but it didn't make it into the final result. In two, the AI changed it and the accountant went along with it; in the other, the accountant overrode the AI's correct advice. That's why the review has to be designed: the AI flags anything doubtful, items are reviewed one by one and everything is logged, so you can see afterwards who decided what and correct it.
Is the Mercor study reliable?
It's a serious, well-explained study, but a small one: 12 accountants, in the United States, with four simplified tasks, and run by the same company that created the test. Mercor says it hesitated to publish it, for fear it would be misinterpreted. The most useful part is the trend: with the same tasks, AI has gone from close to 0% in mid-2024 to 100%, or very close, today.
Will AI replace my tax advisor or my accountant?
That's not what the study says, nor what we do. Mercor expects accountants' work to shift toward what AI doesn't do: dealing with the client, asking the right questions and making decisions when there are no clear rules. With an autopilot, the AI prepares and the person reviews, decides and signs. It's more capacity for the same team, especially in the busy seasons.
Do I have to change my accounting software?
No. We add, we don't replace. Your ERP is still your ERP and your email is still your email. If you ever stopped working with us, you'd keep invoicing and reading your email exactly as you do today. We connect all your software, today's and whatever comes next. If any of it is very old, we bring it up to date first: we assess that in the diagnosis, before we start.