What happened
OpenAI announced a research collaboration with Ironclad on October 6, 2026, using contracting tasks to train and evaluate computer-use agents. Its published evaluation covers 11 tasks, scored against 8 to 50 criteria each. GPT-6 Astra achieved a mean rubric score of 55.0%, compared with 41.6% for GPT-5.6 Sol. The models used different reasoning settings.
OpenAI reports estimated attempt times of 19.2 and 37.0 minutes, respectively. Those are simulated estimates, not measured customer savings. The company says training and evaluation used synthetic tasks drawn from filtered public SEC contracts, rather than nonpublic customer contracts. The announcement and its footnotes provide the scope of those claims.
What actually changed
The useful development is a more specific target for agent evaluation: translating business requirements into a functioning process. This remains a research result. A 55% rubric score is not a 55% end-to-end success rate, and it should not be presented as production readiness.
Why it matters
Our interpretation is that evaluation should follow the structure of the work. In a contracting process, one missed approval condition could matter more than several correctly completed cosmetic steps. Buyers should ask whether scores reflect that difference.
Consider a hypothetical software-purchasing workflow. An agent builds the intake form, adds a finance approval, and produces a clean final record. If the approval fires at the wrong spending threshold, a visually convincing result can still violate the business requirement. A completion metric should therefore test both the configuration and its behavior on either side of the threshold.
The same distinction affects the business case. Faster execution has value only after the organization includes review, correction, and exception handling. A team that saves ten minutes of setup but spends twenty minutes finding a subtle policy error has not improved the process.
Who should care
Legal operations, procurement, and enterprise automation teams should use this announcement to sharpen their pilot requirements. Before delegating a workflow, define the decisions the agent may make, the changes requiring approval, the evidence it must retain, and the conditions that trigger escalation.
A useful pilot should include ordinary cases and deliberately awkward ones: conflicting instructions, missing records, inaccessible controls, changed thresholds, and interrupted sessions. Ask the agent to demonstrate that the final configuration works, rather than merely describe the steps it took.
What to watch next
Look for evaluations that report critical failures separately from average scores, explain how tasks were selected, and distinguish estimated execution time from observed deployment outcomes. Independent testing across customers and interfaces would help establish how far the result travels.
The next meaningful question is whether a team can predict when the agent will fail and contain the consequences. Until that is demonstrated, the practical stance is a bounded pilot with accountable review, supported by evidence rather than a polished demo.
Sources & Further Reading
Editorial disclosure: AI assisted research, drafting, and production. Reported developments are attributed to the sources above; the practical implications are The Agentic Current’s analysis.