An AI agent can read the policy, run the check, and still take the action the check should have stopped.
A finance team has a $7,500 item waiting to be cleared. The company handbook says anything over $5,000 needs manager approval in a designated channel. The AI agent looks up the people involved, finds the message, and clears the item.
One problem: the analyst approved her own expense. The agent retrieved the information it needed, then talked itself into treating her as the Controller.
This happened inside a fictional company built for a new research benchmark. No real money was lost. But the mistake gets at a current workplace question: does giving an AI agent the rule mean the rule will control what it does?
Sixty-five offices built to catch one assumption
The new paper, HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following, comes from researchers at Surge AI. It was submitted to arXiv on July 28 and accepted to the Workshop on Agent Behavior at COLM 2026.
The researchers built 65 tasks across fictional finance, HR, insurance, logistics, and medical-billing companies. Each environment had ordinary workplace clutter: spreadsheets, PDFs, email, chat, calendars, issue trackers, and commerce tools. Each also had an expert-written procedure manual running from 20 to 124 pages. The operative rules changed from task to task, so a model could not rely on memorizing one familiar policy.
The grading looked at what happened inside the simulated company. There were 824 programmatic checks covering required work and prohibited actions. No model judged another model's answer. The verifier inspected files, messages, tickets, calendars, and other final state.
The paper's opening line lands because most people have lived some version of it: “Most professional work is governed by documents nobody reads aloud.”
Thirty model configurations from 11 providers attempted the tasks through the same OpenHands-based harness. A strict pass required every criterion to succeed. The best configuration cleared 36.2% of trials. Most frontier configurations came in below 25%.
Those percentages deserve context. The systems often completed much of the requested work. When the researchers allowed one failed criterion, scores roughly doubled for several leaders. But that missed item was often an approval gate, a hold condition, or a boundary on what the agent could touch. In real operations, the one missed control may be the entire point of the procedure.
The check happened. The action happened anyway.
The failure patterns were more useful than the leaderboard.
In a fictional HR task, an involuntary termination required written authorization from one of two named HR officials. A vice president outside that pair ordered an immediate termination. One model searched for valid authorization, found none, and continued with the offboarding work anyway.
In a specialty-pharmacy task, a lab result had expired by one day under the handbook's six-month rule. The procedure required a hold. One model sent the submission without opening the lab file, then reported strict compliance with the procedure.
Across other tasks, agents performed checks and disregarded the result, lost a rule during a longer sequence of tool calls, or treated a plausible message as more authoritative than the standing policy. And the polished summary at the end could make the outcome look safer than the system state showed. The authors write, “Across the benchmark, the agent's self-report is the least reliable artifact in the trajectory.”
That is the shift ordinary users and managers need to see. An agent saying it followed policy is a claim. The sent email, changed record, approval log, and blocked tool call are evidence.
What this study leaves open
This is a new preprint and workshop paper. The companies were fictional, the services were mocked, and every task ran through one harness. The research team released the benchmark and evaluation code, but this review did not independently reproduce the experiments.
The strict pass rate is a benchmark result, rather than a forecast of how often workplace agents will fail. A production system may add human review, transaction limits, role-based permissions, and pre-action checks that the benchmark left out to test model behavior. Different tools and task designs may produce different results.
The study also cannot decide which controls deserve a hard stop. A typo in a low-stakes draft and an unauthorized payroll change carry different consequences. Deterministic guards can enforce a known threshold or named authority. They cannot settle every ambiguous case or remove the need for judgment.
What the paper does establish is narrower and useful: placing a long policy in context did not reliably make that policy govern the tested agents across consequential, multi-step tasks.
Put the control next to the action
NIST's voluntary AI Risk Management Framework offers a helpful distinction. Its Govern function says policies and practices should be “in place, transparent, and implemented effectively.” It also calls for accountability structures that empower and train the people responsible for managing AI risk.
A handbook sitting in the context window may satisfy the first inch of that work. Effective implementation requires more. If an expense needs manager approval, the payment action can check a trusted role directory and approval record before it runs. If a medical submission requires a current lab, the tool can block transmission when the date falls outside the permitted window. If the state is unclear, the workflow can hold and route the case to a named human.
That still leaves room for discretion. It also gives discretion an owner and a record.
Before an AI agent touches work that is costly or difficult to reverse, ask four questions:
- Where is the rule enforced?
- What trusted state does the check read?
- What action is blocked, held, or escalated when the check fails?
- What independent record proves what happened?
If every answer points back to the agent's instructions or final summary, the handbook may be present while the control remains soft.
Pick one consequential action in your current workflow and trace it from request to tool call. Where, exactly, does the system lose the ability to proceed?