NVIDIA Blackwell Tops First AgentPerf Benchmark, Running 20x More Agents per Megawatt
Updated
Updated · NVIDIA Blog · Jun 12
NVIDIA Blackwell Tops First AgentPerf Benchmark, Running 20x More Agents per Megawatt
3 articles · Updated · NVIDIA Blog · Jun 12
Summary
AgentPerf’s first published results put NVIDIA’s GB300 NVL72 at the top of the new agentic AI benchmark, with up to 20 times more agents per megawatt than the HGX H200.
The benchmark was built because agentic AI chains dozens to hundreds of model and tool calls, making single-request inference tests a poor measure of responsiveness, concurrency and infrastructure efficiency.
AgentPerf uses real coding-agent trajectories across more than 12 programming languages and simulates tool-call CPU time, isolating accelerated computing performance while measuring how many tasks a platform can run within latency and token-rate thresholds.
NVIDIA said the edge comes from rack-scale design linking 72 GPUs, plus CUDA and TensorRT-LLM optimizations that improve mixture-of-experts execution and sustain efficiency as concurrent agent sessions rise.
Baseten, DeepInfra and Together AI are already serving agentic workloads on Blackwell, underscoring growing enterprise demand as NVIDIA says Vera Rubin systems are now in full production.
Is AgentPerf a neutral industry standard or a test designed to make NVIDIA's new hardware shine?
If CPUs cause 88% of agent AI latency, how much does a 20x more power-efficient GPU truly help?
With AI agents' 'hidden token tax,' will their immense power become too costly for mainstream business use?
NVIDIA Blackwell Achieves 20x Agents per Megawatt, Transforming Agentic AI Performance and Market Dynamics
Overview
NVIDIA's Blackwell GB300 NVL72 system, announced in June 2026, marks a major leap in agentic AI performance by achieving a 20x efficiency gain over previous generations. This breakthrough sets a new benchmark for AI workloads, especially in the crucial 'agents per megawatt' metric, which is vital for data center operators aiming to optimize power and costs. The system's capabilities were rigorously validated using the AgentPerf benchmark, demonstrating how Blackwell fundamentally redefines the deployment and scaling of AI agents. These advancements position NVIDIA at the forefront of next-generation AI infrastructure, enabling more efficient and scalable AI solutions.