- Per-token AI inference costs for models with capabilities comparable to GPT-4 have fallen from around $30 per million tokens in 2023 to under $1 per million tokens in 2026.
- Open-source LLMs, including Zhipu AI‘s GLM-5.1 released this week, now rival or exceed leading proprietary models on coding benchmarks, with costs limited to self-hosting infrastructure.
- Companies such as Pinterest are already reporting substantial savings by blending open-source and proprietary AI models, with some estimates pointing to cost reductions of up to 90%.
The per-token cost for GPT-4-level AI performance has collapsed from around $30 per million tokens in 2023 to under $1 today, and this week’s release of a 744-billion-parameter open-source model that reportedly outperforms leading proprietary systems on coding benchmarks suggests the floor hasn’t been reached yet. For product teams and developers, the question is no longer whether they can afford to build with AI it’s what to build first.
The cost collapse, by the numbers
The price drop isn’t a modest improvement. It represents roughly a tenfold annual reduction in cost for equivalent performance. Zhipu AI this week released GLM-5.1, a mixture-of-experts model under an MIT licence.
What cheap inference actually unlocks
Lower costs don’t just make existing applications cheaper they make previously unviable categories worth attempting. Here are the areas where the economics now make sense:
- Hyper-personalised content generation: With cheaper inference, developers can build agents that generate tailored marketing copy, educational materials or creative writing at scale for individual users. A learning platform that dynamically rewrites explanations to match a student’s comprehension level, for instance, would have been prohibitively expensive to run at scale two years ago.
- Advanced coding assistants and agents: Models like GLM-5.1 make it feasible to build specialised coding tools that go beyond autocomplete autonomously debugging codebases, refactoring legacy systems or generating functional modules from natural language prompts. AWS launched Neuron Agentic Development capabilities on May 1, 2026, providing open-source agents and skills for AI coding assistants on its Trainium and Inferentia hardware. For a practical look at how AI coding assistants compare, see our recent breakdown.
- Low-cost data analysis and reporting: Affordable LLMs can now power tools that ingest raw data, identify trends and generate readable reports for small businesses that can’t justify dedicated analysts. The barrier was always cost per query that barrier is largely gone.
- Specialised language processing microservices: Rather than monolithic AI solutions, developers can build tightly scoped services for niche tasks: extracting specific legal clauses from contracts, summarising medical research for a particular subspecialty, or running sentiment analysis tuned to industry-specific language. Open-source models make fine-tuning for these use cases both practical and affordable.
- Interactive educational platforms: AI tutors capable of real-time feedback, adaptive simulations and language learning tools that adjust to a user’s progress are now within reach for teams without enterprise-scale budgets.
- Sophisticated customer service automation: The latest open-source models handle complex multi-turn conversations and can draw on large knowledge bases with far more reliability than earlier chatbot generations. Pinterest reportedly achieved cost savings of up to 90% by blending open-source and proprietary models in its customer-facing features, according to recent reports.
The strategic case for building now
Reduced inference costs compress time-to-market and lower the financial risk of iteration. Developers can run more experiments, pivot more freely and reach viability faster than at any previous point in the AI stack’s history. Free API credits from major providers add further runway: in 2026, new developers can access over $200 in free AI API credits across more than 15 providers, including OpenAI, Anthropic, xAI, Together AI and Google Gemini, which reportedly offers unlimited free-tier access without a credit card.
There’s a counterintuitive dynamic worth understanding: overall enterprise spending on AI inference is rising even as per-token costs fall. The reason is volume. Agentic AI workflows where a single user task triggers chains of model calls rather than a single prompt consume far more tokens per session than conventional generative AI. Unit costs are down; total demand is accelerating. Building now means establishing market position before costs stabilise and competition for the best inference capacity intensifies. For a broader look at how agentic workflows are reshaping enterprise AI, see our recent roundup. The window is real, but it won’t stay open indefinitely.
Navigating what comes next
Future inference costs will be shaped by next-generation GPU architectures, continued gains in model efficiency and ongoing competition among cloud providers. Gartner has forecast that LLMs could become substantially more cost-efficient by 2030 than early models from 2022, driven by improved hardware and model design. But cheaper unit costs don’t automatically translate to lower enterprise bills demand for frontier capabilities has consistently grown to absorb efficiency gains.
The practical response is architectural flexibility: build systems that can swap models and providers without major rewrites, and lean into open-source options for cost control, customisation and reduced vendor lock-in. Inference now accounts for a large share of enterprise AI budgets, so understanding inference economics isn’t a technical footnote it’s a core business decision. The developers who treat it that way, focusing on niche applications where AI delivers clear, measurable value, are best positioned for what comes next. Stay up to date with the latest AI developments at Auton AI News.
