- Etched’s first-pass silicon success on TSMC’s N4P process, paired with over $1 billion in signed contracts, and the OpenAI-Broadcom “Jalapeño” debut put real competitive pressure on Nvidia‘s roughly 80% share of the AI accelerator market for the first time at the hardware layer.
- Inference now accounts for approximately two-thirds of all AI compute, up from around one-third in 2023, shifting procurement priorities away from training-optimised GPUs toward chips built specifically for serving models at scale.
- Total cost of ownership for AI hardware runs three to four times the purchase price over three years: power alone for a 100-GPU deployment runs roughly $150,000 annually, and annual maintenance contracts typically add 15% to 25% of hardware value on top of that.
Etched just demonstrated working silicon on TSMC’s N4P process and walked away with over $1 billion in signed customer contracts, all before most enterprises had heard of the company. That debut, arriving the same week OpenAI and Broadcom unveiled their custom “Jalapeño” inference chip, makes 2025 the first year the challenge to Nvidia’s accelerator dominance has come from actual shipping hardware rather than roadmap slides. For enterprises planning AI infrastructure through 2034, the procurement calculus is changing fast.
Phase 1: Assessing the Current AI Accelerator Landscape
Before projecting where the market goes, it helps to be clear-eyed about where it stands. That means benchmarking the current leaders on market share and scrutinising performance-per-watt figures that will drive real operational costs.
Step 1: Benchmark Current Market Leaders and Their Share
Nvidia holds roughly 80% of the AI accelerator market, a position built less on raw transistor counts than on the CUDA software ecosystem that simplifies model training and deployment. For most enterprises, Nvidia hardware remains the path of least resistance. It is also the most expensive, and supply constraints have kept prices elevated.
The competitive picture is shifting. AMD‘s MI300 AI accelerator series was projected to generate over $2 billion in revenue in 2024, according to the company’s guidance at the time, and the MI355X is now in the field. Intel’s Gaudi chips target cost-sensitive buyers, with Intel claiming pricing roughly 50% below Nvidia’s H100. Cloud providers are doing their own thing entirely: Google’s TPUs, AWS’s Inferentia and Trainium, and Meta’s custom ASICs are all purpose-built for their own inference workloads, pulling significant compute off the merchant silicon market. The arrival of Etched’s transformer-specific chip and the OpenAI-Broadcom Jalapeño indicates the field is fragmenting further, with specialised inference silicon now coming from outside the traditional chip industry.
Step 2: Analyze Performance Metrics and Power Efficiency
Raw FLOPS figures matter less than workload-specific throughput. For LLM inference, the relevant metrics are FP8 and FP16/BF16 performance, INT8 throughput, memory bandwidth and memory capacity, and queries-per-watt under sustained load. A chip that wins a synthetic benchmark but throttles under continuous inference traffic is not a good buy.
Power draw numbers are worth taking seriously. A single Nvidia H100 pulls up to 700W; the B200 draws 1,000W; AMD’s MI355X is rated at 1,400W. Higher thermal density means more cooling infrastructure. Liquid cooling systems can add $50,000 to $200,000 per rack in upfront cost, and a 100-GPU deployment can run roughly $150,000 annually in combined power and cooling expenses. Qualcomm’s Cloud AI 100 chip has shown approximately 227 server queries per watt in certain benchmarks, compared to roughly 108 queries per watt for the H100 in the same tests, though independent replication of vendor benchmark figures is always worth requesting before making procurement decisions.
Phase 2: Projecting Future Demands and Technology Trends to 2034
Anticipating AI workload growth and the evolution of chip architecture is the harder part of the planning exercise. The workload mix is shifting faster than most procurement cycles can track.
Step 1: Forecast AI Workload Growth and Diversification
Market size projections for AI accelerators vary widely depending on scope and methodology. Figures in circulation range from roughly $43 billion in 2026 growing to around $309 billion by 2034, to broader definitions that put the 2034 figure closer to $560 billion. The spread reflects genuinely different views on how much edge AI and agentic workloads contribute to the total. Treat any specific CAGR figure as directional rather than precise.
What is more concrete is the composition shift. Inference now accounts for approximately two-thirds of all AI compute, up from around one-third in 2023, a ratio shift that changes what chips enterprises should be buying. Agentic AI workloads are particularly compute-hungry: they can burn five to 30 times more tokens per task than a standard chatbot interaction, which compounds the infrastructure load quickly at scale. Edge AI is also pulling demand toward low-power, low-latency silicon for autonomous vehicles, industrial IoT and robotics, a segment the major GPU vendors are not optimally positioned for. This is where the OpenAI-Broadcom Jalapeño and chips like Etched’s are targeting their wedge.
Step 2: Identify Emerging Architectures and Interconnect Standards
The GPU is not going away, but it is no longer the only serious option. ASICs deliver better efficiency for fixed workloads at the cost of flexibility. Transformer-optimised chips, which tune matrix-multiplication pathways specifically for attention mechanisms, are showing real latency advantages in large-sequence processing. Heterogeneous compute, mixing CPUs, GPUs, FPGAs and ASICs across a single workload, is becoming the architecture of choice for enterprises running diverse AI pipelines.
Interconnect is where the physics get hard. As model sizes grow, moving data between chips becomes the bottleneck. Nvidia’s NVLink hits approximately 14.4 Tbps per GPU with Blackwell; Google’s TPU Pod interconnects use 2D and 3D torus network topologies to keep latency low at scale. The Compute Express Link (CXL) standard is gaining adoption as a memory interconnect layer for AI tensor node chiplets, gradually displacing DDR and PCIe memory hierarchies in high-density deployments. Co-packaged optics (CPO), which integrates optical interfaces directly into chip packages, is being developed to push bandwidth density further. None of these are fully mature yet, but they will matter for infrastructure decisions made in the next 18 months.
Phase 3: Strategic Selection and Investment for Long-Term Value
Hardware performance is only part of the decision. TCO modelling and ecosystem depth can easily outweigh a benchmark lead when you are committing to infrastructure for three to five years.
Step 1: Evaluate Total Cost of Ownership (TCO)
The purchase price of an AI accelerator is the smallest number in the TCO model. A single Nvidia H100 can cost upwards of $30,000 at list price, but that is before power, cooling, maintenance and facilities. Running 2,000 enterprise GPUs can cost roughly $2 million annually in electricity alone, around $1,000 per GPU per year. Cooling adds another 20% to 50% on top of that. Annual maintenance contracts for AI hardware typically run 15% to 25% of the hardware’s purchase price, meaning a $3 million hardware investment can carry $450,000 to $750,000 per year in maintenance costs before a single model is served. Specialist data centre space adds further, typically costing $200 to $300 per kilowatt per month.
The practical consequence is that enterprises focused narrowly on chip acquisition cost tend to hit budget problems within the first year. A complete TCO model, covering upfront capital expenditure for hardware, networking and storage alongside annual operating costs for power, facilities, personnel and maintenance, is the right starting point for any serious infrastructure decision. Firms like SemiAnalysis publish detailed TCO frameworks that break this down per workload type, which is worth referencing before committing to a specific platform.
Step 2: Assess Ecosystem Maturity and Software Support
Nvidia’s staying power comes from CUDA as much as from the silicon. A mature software ecosystem means less integration work, broader framework support and a larger pool of engineers who already know the tooling. AMD’s ROCm and Intel’s oneAPI are both improving, but neither has reached parity with CUDA in terms of library coverage and community depth. That gap is real and carries a cost, particularly for teams without dedicated ML infrastructure engineers.
The broader point, attributed to the AI processor market in a June 2026 enterprise hardware update from ServerMonkey, is that “AI has become a systems problem involving compute, memory, software, networking, power, and economics.” The implication is that a chip vendor’s long-term software roadmap, its partnerships with framework maintainers, and its integration with cloud deployment tooling (Cadence’s collaboration with Google Cloud is one example) matter as much as peak FLOPS. Vendor lock-in is a real risk: NVIDIA AI Enterprise’s GPU-accelerated container bundles are convenient for standardised deployment, but they also tie operational workflows tightly to one vendor’s stack. Enterprises building infrastructure for a decade should model the exit cost of each platform alongside the entry cost.
The Etched and Jalapeño debuts are the clearest signal yet that specialised inference silicon is graduating from research project to procurement-grade option. Enterprises that restrict their chip evaluations to the established GPU vendors will likely miss both cost and performance advantages as inference-optimised hardware matures. The right approach now is to maintain flexibility: run TCO models that include power and maintenance, benchmark against actual inference workloads rather than synthetic figures, and treat software ecosystem depth as a hard requirement rather than a nice-to-have. For more coverage of AI chips and infrastructure, visit our AI Hardware section.
