New Paper Quantifies AI Agent Intelligence in Bits via Compression

There is no agreed way to measure how intelligent an AI agent is. Task benchmarks are inconsistent, performance scores don’t travel across architectures and the field has largely settled for “good enough on this test.” A paper published this week proposes a different approach: treat the agent as an information compressor and measure its intelligence in bits. The framework, if it holds up empirically, could give researchers and developers their first genuinely portable metric for agent capability.

Framing Agents as Information Compressors

The paper’s central argument runs against how AI agents are typically described. Most evaluation approaches treat an agent as an explorer: retrieve information, compare options, evaluate outputs. That framing makes performance hard to measure consistently, because exploration looks different across tasks.

The alternative proposed here is to think of an agent as a compressor. Just as a compression algorithm reduces data while preserving what matters, an intelligent agent converts raw environmental information and interaction history into concise, actionable representations. The more efficiently it does that, the more intelligent it is, expressed not as a score on a task but as a quantity in bits.

The reframing has a practical motivation. In long-running agent sessions, conversation history grows without bound, pushing against context window limits, compounding token costs and degrading reasoning quality through what researchers call “context drift.” Intelligence, by this framework’s logic, scales not by knowing more but by compressing successful operations into reusable structures. An agent that retains critical information while shedding redundancy is, by this measure, demonstrably smarter, and cheaper to run.

Information-Theoretic Foundations

The quantification relies on established information theory. Information is measured in bits: the theoretical minimum size of a compressed representation of a given signal. Highly compressible data contains fewer bits of surprise; diverse, unpredictable data contains more. The framework applies mutual information, a measure of how much two signals share, to ask how much task-relevant information survives an agent’s compression step.

Stanford University researchers Shizhe He and Avanika Narayan treated the agent’s compression model as a communication channel and asked exactly that question. Their finding: compression quality predicts downstream performance, and information density (bits per token) is a better predictor than raw token count. That moves the result from an interesting theoretical correlation to something closer to a methodology for measuring performance directly.

Three Coding Mechanisms

The paper operationalises the framework using three coding techniques drawn from data compression: arithmetic coding, seed coding and fallback coding. It evaluates the measure across five settings, though implementation details remain sparse in available material.

Arithmetic coding assigns shorter codes to more probable sequences. Applied to agent behaviour, it measures how predictably an agent reaches a goal. An agent that takes decisive, consistent steps through a complex task produces a highly compressible action sequence, fewer bits to encode the successful trajectory means higher measured intelligence. An agent that fumbles toward an answer leaves a more expensive trail.

Seed coding relates to knowledge representation. An intelligent agent compresses raw data into a compact, reusable knowledge base, a “seed” from which it can bootstrap reasoning across varied tasks. The bit-count of that seed captures the minimum information required to reconstruct the agent’s problem-solving capability. Smaller seed, broader coverage: higher intelligence under this metric.

Fallback coding addresses adaptation. When an agent hits an edge case its standard pathways can’t handle, it has to recover. The efficiency of that recovery, how few new bits it needs to process or infer to get back on track, is a measure of adaptability. An agent that recovers cleanly from failure with minimal additional input demonstrates something that task-success metrics tend to miss entirely.

Together, the three mechanisms aim to capture perception, planning, action and adaptation in a single quantitative framework.

Practical Implications for Agent Design

The measurement framework is most useful if it actually guides design decisions, not just ranks existing systems. There is already practical work pointing in the same direction.

Anthropic‘s compact-2026-01-12 API offers automatic context compaction in production, a signal that managing context intelligently, rather than simply expanding context windows, is becoming a design priority across major providers. Context compression approaches using anchored iterative summarisation consistently outperform full-reconstruction methods, according to the paper.

The clearest real-world test is Murakkab, an intelligent workflow system developed by researchers at MIT and Microsoft. Designed to streamline agentic workflows, it used roughly 35% of the computation and less than 25% of the cost of comparable methods on video Q&A and code generation tasks, with approximately a 2% accuracy drop in some cases. That efficiency was achieved by optimising for operational compression rather than raw performance, precisely what the bit-quantification framework is designed to incentivise.

The potential benchmarking value is real. Current agentic evaluations struggle with non-determinism, limited reproducibility and benchmark overfitting. A metric grounded in information theory is task-agnostic by design, which could make it far more portable across the varied environments where agents are now deployed. Whether it becomes a standard is a question of adoption, not theory, but the theoretical grounding is there. For teams grappling with agent reliability at scale, this connects directly to a pattern seen across recent deployments: a significant share of enterprises have rolled back AI agents because of the context and efficiency failures this framework targets.

Open Questions and Research Directions

The hard problems are ahead. Defining “information” precisely in a real-world agentic environment, where inputs are messy, goals shift and tool outputs are unpredictable, is not straightforward. Attributing specific bit-counts to distinct aspects of agent behaviour (planning versus tool use versus memory) will require methodology that the paper only begins to sketch.

Three directions seem most pressing. Decomposing total intelligence bits into sub-components, planning bits, tool-use bits, learning bits, would give developers something actionable rather than a single summary figure. Real-time quantification, measuring an agent’s compression efficiency as it runs rather than post-hoc, would make the metric useful for systems that learn during deployment. And cross-domain validation across multi-agent systems and tasks requiring human-agent collaboration remains open, where the compression dynamics are considerably more complex.

The conceptual link between intelligence and compression has a long history in AI and cognitive science, algorithmic information theory has been pointing in this direction for decades. What’s new here is the attempt to make it operational for modern, multi-step, tool-using agent systems, with specific coding methodologies attached. Whether the three-mechanism approach holds up empirically across diverse settings is the next question. For more coverage of AI research and breakthroughs, visit our AI Research section.