ZML Launches Free LLM Server Across 5 Chip Platforms as Inference Costs Rise
Updated
Updated · TechCrunch · Jul 8
ZML Launches Free LLM Server Across 5 Chip Platforms as Inference Costs Rise
1 articles · Updated · TechCrunch · Jul 8
Summary
French startup ZML released ZML/LLMD, a free inference server designed to run open-source large language models at high speed across Nvidia, AMD, Google TPU, Apple Metal and Intel Arc hardware.
The product targets vendor lock-in in AI inference, which founder Steeve Morin said has become more critical than model training as enterprises seek cheaper, lower-energy ways to deploy LLMs.
ZML says the software could let clouds and enterprises mix chips more freely, potentially helping newer AI chipmakers compete while still working alongside Nvidia rather than against it.
Competition is already intense in the inference boom, with rivals including Baseten—valued at $13 billion—as well as projects tied to vLLM and SGLang.
Backed by $20 million and a 20-person Paris team, ZML is using the free launch to study adoption before deciding whether ZML/LLMD should become a paid product.
Is ZML a true tech disruptor, or is it Europe’s strategic bet to achieve digital sovereignty in the global AI race?
Can a software-only startup break Nvidia's hold when rivals are raising billions for their own hardware and cloud platforms?
Is hardware flexibility worth sacrificing the peak performance promised by custom AI chips like OpenAI's Jalapeño?
ZML Version 2: Cutting LLM Inference Costs and Breaking Vendor Lock-In with a 2.4GB Open-Source Stack
Overview
ZML Version 2 addresses the rising costs and hardware lock-in challenges of large language model (LLM) inference by launching an open-source, hardware-agnostic stack. Built under the Apache 2.0 license, ZML empowers organizations to optimize costs and gain flexibility in their LLM deployments. Its core philosophy focuses on explicit control, composability, and predictability, enabling high-performance, production-ready inference across diverse hardware. By providing a robust, open-source solution, ZML helps organizations move away from proprietary systems and better manage the growing demands and expenses of deploying advanced LLMs.