- UC Berkeley researchers and Anthropic engineers released the “Reliable Agency” framework on May 21, 2026, to address chronic miscalibration in AI agents.
- Applying the Brier Score to agentic action-selection steps reduced catastrophic execution failures by roughly 35% compared to standard logit-based sampling, according to the paper.
- The paper argues ISO 42001 AI management standard compliance will likely require strictly proper scoring rules to generate a verifiable audit trail of model confidence, a requirement that auditing firms including Deloitte and KPMG are already beginning to enforce in practice.
AI agents fail not because they lack capability but because they are overconfident. A joint team from UC Berkeley and Anthropic released a paper on May 21, 2026, arguing that autonomous systems go wrong when their internal probability estimates bear little relationship to actual outcomes, deleting database records or sending incorrect financial transfers because the model’s confidence and its actual reliability have come apart. The fix they propose is borrowed from forecasting science, not from model scaling.
The Brier Score for Action Success
Most calibration failures are quiet until they are not. An agent proposes a tool call, sounds authoritative, executes, and the error only surfaces after something breaks. The “Reliable Agency” paper targets exactly that gap by introducing a “Calibration Head” into Claude 4.5’s self-correction loop. When the agent proposes an action, the Calibration Head predicts the probability that the action will succeed. The Brier Score, the mean squared error between that predicted probability and the actual binary outcome, then measures how honest those predictions are over time. A persistently high score means the agent is consistently misjudging its own reliability. Applied to action-selection steps, the paper reports roughly a 35% reduction in catastrophic execution failures compared to standard logit-based sampling.
Logarithmic Rules for Token-Level Confidence
Large language models are trained using cross-entropy loss, which is itself a form of logarithmic scoring. The calibration should, in theory, carry through to deployment. The Berkeley paper’s uncomfortable finding is that it does not. During reinforcement learning from human feedback, models are rewarded for sounding helpful and certain, which compresses their probability distributions toward the extremes, overwriting the calibration learned during pre-training. The result is reasoning loops where the model produces confident-sounding output while effectively guessing.
Spherical Scoring for Directional Planning
For multi-step planning, neither the Brier Score nor the logarithmic rule is quite the right tool. Both can be destabilised by extreme outlier predictions, a problem that compounds when an agent is navigating a 10 or 20-step task, where a single badly scored step can distort the overall picture. The Berkeley framework addresses this with spherical scoring rules, which are bounded and less sensitive to those extremes, making them better suited to what the researchers call “Trajectory Calibration.” Agents optimised with spherical scoring showed more efficient navigation, avoiding the looping behaviour seen in earlier models, according to the paper.
Resource Forecasting with CRPS
Temporal blindness is one of the more practically expensive failure modes in enterprise agents: a system starts a task with no estimate of whether it will take 10 seconds or 10 minutes. For automated workflows tied to cloud budgets or SLA commitments, that uncertainty compounds quickly. The Berkeley paper introduces the Continuous Ranked Probability Score (CRPS), a proper scoring rule designed for continuous variables rather than discrete actions, as the mechanism for fixing this. Without CRPS calibration, the paper argues, agents are prone to optimism bias in cost estimates. With it, they flag high-uncertainty cost scenarios and pause for human approval rather than proceeding.
Scoring Rules as Liability Shields
The researchers propose that Brier and CRPS scores should be included in standardised agentic logs, noting relevance to ongoing AI standardisation work by the ISO/IEC JTC 1/SC 42 committee on artificial intelligence. According to the paper, auditing firms including Deloitte and KPMG are already requesting calibration data when reviewing enterprise AI deployments. The paper draws an analogy to telematics data in automotive insurance: as the industry moves toward agentic liability coverage, calibration scores may become the basis for calculating premiums.
The “Reliable Agency” framework will not make agents infallible. What it offers instead is something arguably more useful for production deployment: agents that are honest about when they are likely to fail. That shift from overconfident execution to calibrated uncertainty is what the Berkeley-Anthropic team frames as the actual prerequisite for high-stakes automation, and it is a harder engineering problem than simply improving accuracy scores. This connects to a broader pattern worth watching: as our coverage of NYU’s analysis of METR’s AI capability benchmarks shows, how AI systems report their own performance is becoming as contested as the performance itself. For more coverage of AI research and breakthroughs, visit our AI Research section.



