- French startup ZML launched its free, closed-source inference server ZML/LLMD on July 8, 2026, supporting open-source LLMs across five major chip architectures from a single codebase.
- ZML/LLMD targets Nvidia’s CUDA lock-in directly, enabling deployment across heterogeneous hardware without maintaining separate optimised stacks for each platform.
- Built with Zig, MLIR and OpenXLA, ZML/LLMD automatically generates hardware-specific kernels, though independent benchmarking across all five architectures has not yet confirmed performance parity with native vendor stacks.
A 20-person Paris startup is taking direct aim at Nvidia’s software moat. ZML released ZML/LLMD on July 8, 2026: a free but closed-source inference server that runs open-source LLMs across Nvidia, AMD, Google TPU, Apple Metal and Intel Arc hardware from a single codebase. For any team managing mixed hardware environments, that is a real engineering problem with a concrete proposed solution.
The inference cost problem
Inference, not training, is where production compute costs accumulate. Every prompt served, image generated and recommendation delivered draws on inference capacity that is expensive, fragmented and increasingly hard to manage across different hardware types.
The core problem, as ZML frames it, is proprietary stacks. To get peak performance from an Nvidia H100, you use CUDA, cuDNN and TensorRT. An AMD MI300X requires a different stack. Apple Metal is different again. The result, according to ZML, is teams maintaining separate, hand-tuned codebases for each platform, with performance regressions and operational overhead that often eat the cost savings they were chasing. A company running H100s for training, MI300X for batch inference and Apple Silicon for edge deployments ends up managing three distinct software environments, not one.
ZML/LLMD is built to collapse that to one.
How ZML/LLMD achieves cross-platform performance
The technical approach starts with an unusual language choice: Zig, not Python. Zig is a systems language that operates close to the hardware, giving ZML’s compiler more direct access to memory layout and instruction scheduling than a Python-based stack typically allows. On top of that, ZML uses MLIR (Multi-Level Intermediate Representation) and OpenXLA to compile model operations into hardware-specific instructions automatically.
The alternative is hand-tuned kernels, one per chip architecture. That works well for one platform; it becomes unmanageable at five. ZML’s compiler-driven approach generates those kernels automatically, which is how a 20-engineer team can credibly claim simultaneous support across Nvidia, AMD, Google, Apple and Intel hardware. Automated kernel generation rarely matches a hand-tuned kernel at its absolute peak on a single platform, but across a mixed fleet it can come out ahead on total throughput and cost.
The server supports open-source models including Llama and Mistral. ZML says performance is competitive with native vendor stacks in some configurations, though independent benchmarking across all five architectures has not yet confirmed that picture.
What this means for the hardware market
Nvidia’s competitive position has always rested on two things: GPU performance and CUDA. The GPU performance gap with AMD has narrowed; CUDA is where the lock-in persists. A credible, free runtime that extracts strong inference performance from AMD, Intel and Google hardware without requiring a CUDA rewrite directly reduces that lock-in’s value.
For AMD, Intel, Google and Apple, ZML/LLMD is worth watching for a practical reason: their hardware often loses deals not because the silicon is worse, but because the software around it is thinner. A shared, high-performance inference layer reduces that disadvantage. The same logic applies to European chip startups including Axelera, Fractile, Kalray, SiPearl and VSORA, which have engineering talent but limited resources to build a mature software stack from scratch.
ZML is also going further than supporting existing chips. Founder Steeve Morin says the company is co-designing silicon with chipmakers before tapeout, mapping kernels to instruction sets before the chip is finalised. According to reports, Morin says this can compress a two-year hardware bring-up to around six months, a claim worth watching as partner chips reach production. This kind of software-hardware co-design is where inference infrastructure competition is increasingly being decided.
Europe’s sovereignty play
ZML’s launch connects to something larger than a single product release. The company sits at the centre of a partnership announced June 18 at VivaTech in Paris, where the Île-de-France region, cloud operator Scaleway, French chipmaker VSORA and ZML announced plans for a sovereign European AI stack running from silicon to cloud. In that arrangement, Scaleway committed to deploying VSORA’s Jotunn8 inference chip in its data centres, with ZML providing the software layer that makes the hardware accessible without proprietary lock-in.
ZML has raised $20 million to date, with backing from Xavier Niel’s Kima Ventures and support from AI researcher Yann LeCun. The European Commission’s Chips Act 2.0 and a proposed Cloud and AI Development Act provide the policy backdrop, though neither directly funds ZML. The geopolitical logic is straightforward: if European chipmakers produce competitive silicon but cannot offer the software stack to accompany it, customers default to American vendors regardless. ZML is trying to close that gap.
A crowded field with deep pockets
The inference optimisation market is not short of competition. Baseten is reportedly valued at $13 billion. Gimlet Labs closed an $80 million Series A, bringing total funding to $92 million, and claims 3x to 10x faster inference at equivalent cost by slicing workloads across mixed hardware, according to the company’s own figures. Modal Labs was reportedly in talks in February 2026 to raise at a $2.5 billion valuation. vLLM, commercialised by Inferact, and SGLang, backed by RadixArk, are also active in the space.
ZML’s differentiation is the free release of LLMD; unlike ZML’s earlier ML framework, which is open-source under Apache 2.0, LLMD itself is closed-source. The business model comes later, likely through managed services, enterprise support and custom silicon co-design contracts. This is the same sequencing that built Linux, Redis and other foundational open-source infrastructure: maximise adoption first, monetise the services layer once the runtime is ubiquitous. Whether a 20-person team can execute that playbook against well-capitalised rivals is the open question.
What to watch next
ZML’s performance claims need third-party validation across all five target architectures, particularly against TensorRT on Nvidia and ROCm on AMD. GitHub activity, stars, forks and pull requests, will show whether developers are actually building on it or watching from a distance. Morin has indicated commercial tiers will follow once adoption patterns are clearer. How ZML prices enterprise support without alienating the open-source community it needs to grow will be the harder problem to solve. Stay up to date with the latest AI developments at Auton AI News.
