Hidden Deadlines and Professional Judgment: A Guide to Surge AI's DAYJOB: Tax Benchmark

AI agents are increasingly marketed as assistants that can finish professional work, but a new evaluation from Surge AI suggests that tax-related judgment is still a hard frontier. The example at the center of DAYJOB: Tax involves Anna, who had moved out of a condo in August 2024 and wanted to weigh two dates, October 2026 and January 2027, because either one could support up to $250,000 in excluded gains under the usual rule that owners must live in the home for at least two of the five years. The hidden risk was timing: if eligibility ended around the end of August 2027, a sale delayed beyond August 2027 could cost her the break. In five attempts, Astra never identified that deadline; Opus identified it in all five attempts.

For readers, the story is not just about tax law. It is about whether an AI can notice a consequential issue when a human never names it.

Why DAYJOB: Tax was built

DAYJOB: Tax asks whether AI agents can move beyond narrow answers. It places them in assignments that require specialized expertise, complex materials, and decisions that are not fully specified in initial instructions. The DAYJOB series previously covered Healthcare and Finance; this edition covers tax preparation, compliance, and advisory work.

It contains 50 environments across eight areas of tax practice. In each setting, the agent gets a client request and supporting records, then must create tax returns, workpapers, technical reviews, or client advice.

The challenge is not just calculation. Agents must decide what deserves investigation, resolve conflicting evidence, recognize when information is insufficient, and carry those decisions into the final deliverable while demonstrating tax knowledge, retrieval, and instruction following.

What the scores show

The leaderboard is far from solved. Claude Sonnet 5.5 tops DAYJOB: Tax with a 10.8% pass rate, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%.

Scoring is strict: at a 95% rubric threshold, pass@1 is reported. A trial passes only if it meets at least 95% of criteria for calculations, supporting evidence, consequential issues, and the finished deliverable.

Cost and performance together

The report also compares results against average cost per trial. Claude Sonnet 5.5 and Opus 5.5 sit at the high-performance end of the measured Pareto frontier.

The eight areas covered

DAYJOB: Tax spreads the work across eight areas, each with its own rules, records, and expertise:

• Individual and family tax
• Small-business tax
• Partnerships
• Property and investments
• Estates, gifts, and trusts
• International tax
• Corporate transactions
• Tax review and disputes

Some tasks require a complete filing package with supporting workpapers. Others require investigating transactions, reviewing previous filings, or advising on decisions the client has not yet made.

Inside the task files: receipts, logs, and loose ends

The benchmark uses 1,789 workspace files in total, with a median of 30 per assignment and as many as 123. The file types include 1,487 PDFs, 117 Excel workbooks, 38 Word documents, plus emails, images, CSVs, JSON records, and text files.

Workspaces hold filed and draft tax returns, K-1s, W-2s, IRS notices, general ledgers, payroll records, contracts, and correspondence. They also include the rough, less standardized evidence that professionals often must use.

In one example, the task includes a photo of a handwritten receipt for merchandise purchased for resale. The agent must read descriptions and amounts, match them to spending records, and determine their effect on inventory and cost of goods sold.

Another task involves an industrial silo-services company whose client assumes it operates only in Alabama. The field dispatch log instead shows crews working across five states, and job locations, crew sizes, and days on site bear on multistate tax obligations.

Some environments cross borders. One reconciles Polish employment and property records with cryptocurrency exchange exports, wallet histories, and historical purchase lots.

Who made the benchmark

DAYJOB tasks are not pulled from a generic dataset. Tax experts start from real professional workflows and build realistic assignments, supporting records, expected outcomes, and grading criteria. Each task receives a three-layer review for correctness, completeness, and solvability.

The expert group has backgrounds in corporate taxation, partnerships, individual returns, trusts, and multistate compliance, including:

• Corporate and private-equity taxation: A tax attorney with over 10 years of experience at PwC, Deloitte, and Exxon covers tax planning, compliance, audits, and controversy.
• Partnership and corporate taxation: A Baker Tilly Tax Director with experience at PwC, Moss Adams, and Windes specializes in complex partnership allocations, consolidated corporate returns, REITs, and high-net-worth clients.
• Real estate and individual taxation: A tax professional with experience at Deloitte and EY specializes in real-estate partnerships, individual taxation, and trusts.

The retirement-account assumption problem

A separating couple asked an AI agent how to divide their Roth IRA and 401(k). Opus 5.5 told them they were not legally married and recommended keeping accounts separate, with a cash payment to even balances.

You aren't legally married, so neither account can be split without triggering taxes and penalties.

Their own records created doubt. They lived in Florida, where new common-law marriages generally are not recognized, but had previously lived together in Texas and held a ceremony in Colorado. Texas recognizes informal marriages under some conditions, and a valid marriage established elsewhere may remain valid in Florida.

That legal status could change the tax impact of splitting retirement accounts.

Opus relied on the unmarried assumption across all five attempts, even while acknowledging uncertainty. Astra saw the need to verify marital status in four of five attempts, and one response said:

We first need to confirm your legal marital status.

The clients asked only how to divide the accounts, but the agent needed to establish whether they were married before recommending action. The evidence for that check was already inside the files.

How this differs from earlier tax AI tests

Earlier tax evaluations usually start with a defined problem: a legal question, a return to calculate, or missing taxpayer information to collect. Those tasks matter. But professional tax work often begins with figuring out the problem, because instructions are incomplete, evidence conflicts, and consequences may not be named in the request.

DAYJOB: Tax asks whether an agent can own the end-to-end task: spot overlooked issues, weigh conflicting evidence, and decide what should be investigated.

Benchmark comparison

Benchmark	Core question	What it evaluates	What DAYJOB also asks
TaxBench	Is a difficult tax question answered correctly?	TaxBench evaluates 114 tax questions across multiple specialties, each with a defined problem and grading rubric.	It asks whether the client asked the right question. Anna's two dates both satisfy the residence requirement for a tax exclusion, but does the agent warn her when eligibility could expire?
TaxCalcBench	Is an accurate tax return calculated?	TaxCalcBench uses realistic tax documents and taxpayer scenarios to evaluate tax-return accuracy against expected filing values.	It asks whether underlying records support the treatment. If a ledger references an attached receipt but the receipt is missing, does the agent flag missing evidence or treat reimbursement as verified?
TaxLab	Can missing information be gathered and return filed correctly?	TaxLab tests whether agents can question a simulated taxpayer, obtain missing information, and complete a federal individual tax return.	It extends responsibility across wider tax profession: partnership loss analysis, estate return, review of disputed tax position, or strategy letter.

The core distinction is responsibility. DAYJOB tests whether agents can discover consequential issues, use professional judgment, and bring findings to a completed filing, review, or recommendation.

What readers should take away

The results are not a simple failure to celebrate. They show that agents can complete steps but still miss the hidden deadline, hidden assumption, or hidden missing evidence. For now, agents haven't yet passed the audit.

The DAYJOB: Tax benchmark page lists the full leaderboard, with every model evaluated.

FAQ for everyday tech readers

What is DAYJOB: Tax?

It is the tax edition of Surge AI's DAYJOB benchmark family. It tests AI agents on messy professional tax assignments, not just direct Q&A.

Why does Anna matter?

She represents a user with a seemingly narrow request. An agent can compare October 2026 and January 2027, but still fail to warn that the rule requiring at least two of the five years could expire around the end of August 2027.

What does the leaderboard mean?

It means no model is close to dependable professional execution. Claude Sonnet 5.5 leads at 10.8%, and other top models are below it. Passing requires 95% rubric compliance.

Can users trust AI for taxes now?

The benchmark suggests not for complex professional judgment. It may assist with routine steps, but consequential issues remain risky.

Common misconceptions

• A benchmark only tests math. In DAYJOB: Tax, consequential issues and professional judgment carry weight.
• A correct answer to the client's literal request is enough. Anna's case shows that the right response may include a deadline the client did not ask about.
• Missing documents are just inconvenience. TaxCalcBench-style ledger and receipt issues show that evidence quality matters.
• Marital status is only personal context. In the retirement-plan case, it changes the tax consequences.

Action suggestions

1. Use AI tax agents for drafting and retrieval, not for unreviewed final filing.
2. Ask the model to identify assumptions, deadlines, and missing evidence before giving advice.
3. Request file references for every conclusion.
4. When records involve property sales, retirement divisions, multistate work, or cross-border crypto holdings, treat the first answer as incomplete.
5. Have a qualified tax professional review major decisions.

This summary is provided by MyTechDailyNews, edited by Sarah Chen for Tech Daily.