As enterprise artificial intelligence transitions from exploratory pilots to massive, production-grade deployments, the fundamental economics of AI inference have taken center stage. Organizations are no longer asking merely what models can achieve, but rather how much revenue those models can generate per watt, per dollar, and per rack.

Addressing this imperative directly, NVIDIA has released its preview and official performance metrics for the MLPerf Inference v6.1 suite, showcasing staggering leaps in performance driven by its next-generation Vera Rubin architecture and the continued optimization of the GB300 NVL72 platform.

The newly released MLPerf Inference v6.1 results underscore how hardware-software codesign, platform fungibility, and aggressive scaling efficiency are redefining the boundaries of what enterprise AI infrastructure can accomplish. From hyperscale data centers processing complex reasoning models to edge devices executing autonomous agentic workflows, NVIDIA’s latest hardware and software ecosystem offer a masterclass in modern compute economics.

Main Facts

The latest MLPerf Inference v6.1 benchmarks, released by MLCommons, establish several critical milestones for enterprise AI infrastructure:

• Vera Rubin NVL72 Dominance: In preview submissions on two of the industry’s most demanding workloads—DeepSeek-R1 and Qwen3-VL—the NVIDIA Vera Rubin NVL72 platform demonstrated unprecedented capability. It delivered up to 3.7x higher throughput than the GB300 NVL72 on Qwen3-VL across offline, server, and interactive scenarios, utilizing vLLM integrated with the NVIDIA Dynamo open-source inference framework. On DeepSeek-R1, powered by the NVIDIA TensorRT-LLM library, throughput surged up to 2.5x higher than its predecessor.
• The Rise of Agentic AI Performance: Reflecting the industry-wide shift toward AI agents that reason, plan, and execute multi-step workflows, early testing on benchmarks like SemiAnalysis AgentX revealed that the Vera Rubin NVL72 delivered 30x better performance than the GB300 NVL72.
• Exceptional Rack-Scale Efficiency: NVIDIA’s GB300 NVL72 demonstrated 99% scaling efficiency when scaling a DeepSeek-R1 workload from a single rack (72 GPUs) to four racks (288 GPUs) in the offline scenario, proving that throughput scales in near-perfect proportion to added hardware investments.
• A Sprawling Partner Ecosystem: A total of 19 global technology leaders and cloud service providers—including major players like Microsoft Azure, Oracle Cloud Infrastructure, CoreWeave, Nebius, Dell Technologies, HPE, and Supermicro—participated in the v6.1 round, with eight deploying multi-node Blackwell NVL72 systems.

Chronology of Innovation: The Path to v6.1

The journey toward the MLPerf Inference v6.1 results represents an accelerated cadence of hardware architecture rollouts and continuous software optimization that has come to define the modern AI landscape.

The Foundation: Platform Fungibility and Codesign

For years, the economics of data centers have relied on utilization rates. In the AI era, this concept translates to platform fungibility—the architectural capability of a single infrastructure stack to run any model, any workload, from training to inference, recommender systems to reasoning, and language to video processing.

Underlying NVIDIA’s v6.1 achievements is a relentless focus on full-stack codesign. The architectural timeline leading up to these benchmarks has seen hardware elements such as enhanced Tensor Cores and the Transformer Engine evolve specifically to accelerate both the prefill and decode stages of inference. Furthermore, the adoption of NVFP4 precision has drastically reduced memory footprints across model weights, attention mechanisms, and Key-Value (KV) caches, enabling higher throughput without sacrificing output fidelity.

Software Velocity and Version-Over-Version Gains

Hardware alone no longer dictates compute performance; software velocity is the primary multiplier of infrastructure value. Between the MLPerf Inference v6.0 and v6.1 rounds, NVIDIA engineers extracted massive performance gains simply through software refinement.

For instance, GB300 NVL72 performance on the Qwen3-VL benchmark improved by up to 1.6x over v6.0 results alone. These gains were unlocked by lowering KV cache precision, implementing advanced kernel fusion, developing superior custom kernels, and applying disaggregated serving via vLLM and NVIDIA Dynamo. Notably, software optimization did not pause at the v6.1 submission freeze date; post-submission evaluations on models like GPT-OSS-120B and DLRMv3 continue to yield compounding performance improvements.

Supporting Data: Deep Dive into the Benchmarks

To understand why these benchmark results matter to enterprise Chief Information Officers (CIOs) and Chief Technology Officers (CTOs), one must examine the specific data points across scale-up domains, specialized workloads, and edge computing environments.

1. Vera Rubin NVL72 vs. GB300 NVL72

The Vera Rubin NVL72 preview submissions highlight a generational leap in inference capability. By leveraging disaggregated serving—which physically separates the prefill and decode phases—combined with large-scale expert parallelism, Vera Rubin maximizes efficiency across the Mixture-of-Experts (MoE) layers that power advanced models like DeepSeek-R1 and Qwen3-VL.

• Qwen3-VL Performance: Up to 3.7x higher throughput across offline, server, and interactive scenarios using vLLM and NVIDIA Dynamo.
• DeepSeek-R1 Performance: Up to 2.5x higher throughput using the NVIDIA TensorRT-LLM library.
• Interconnect Foundation: The scale-up domain of the NVL72 is anchored by sixth-generation NVIDIA NVLink and NVLink Switches, delivering 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet solutions, allowing rack-scale distributed computing to operate as a single monolithic processor.

2. GB300 NVL72 Scaling Efficiency

Adding hardware to a cluster historically suffers from diminishing returns due to communication overhead and synchronization bottlenecks. NVIDIA’s GB300 NVL72 shatters this limitation.

• DeepSeek-R1 (DSR1) Scaling: Scaling a deployment from 72 GPUs (one rack) to 288 GPUs (four racks) achieved an astonishing 99% scaling efficiency in offline scenarios. This proves that enterprises can expand their clusters linearly without seeing their return on investment (ROI) diluted by scaling penalties.
• Text-to-Video Processing: On the WAN 2.2 text-to-video benchmark, the GB300 NVL72 achieved 0.65 720p videos per second at 5.7 seconds per video, representing a 9x increase in throughput and a 7.5x reduction in latency compared to single-node configurations.

3. Edge-Agentic Computing and Jetson AGX Thor

AI inference is no longer confined to hyperscale data centers. Recognizing the explosive growth of physical AI, robotics, and smart infrastructure, NVIDIA also submitted results for the Jetson AGX Thor platform using NVIDIA TensorRT Edge-LLM on the newly introduced Edge-Agentic benchmark running Qwen3.6-27B, demonstrating that high-performance reasoning is now viable at the extreme edge.

Official Responses and Ecosystem Reactions

The breadth of participation in the MLPerf Inference v6.1 round illustrates an unprecedented level of alignment across the global tech ecosystem. Nineteen major hardware vendors, cloud providers, and system integrators validated the platform’s readiness for enterprise deployment.

"System performance, efficient infrastructure scaling, and continuous software optimization are the foundational levers that determine AI inference economics," industry analysts note, echoing the core thesis of NVIDIA’s platform strategy. Higher system performance directly equates to a greater volume of generated tokens, which in turn drives higher enterprise revenue streams.

Key partner contributions highlighted during the release include:

• Nebius: Successfully submitted Vera Rubin NVL72 preview results alongside NVIDIA, proving the commercial readiness and exportability of the architecture for specialized cloud providers.
• Hyperscalers and OEM Leaders: Major global infrastructure providers—including Microsoft Azure, Oracle Cloud Infrastructure, Dell Technologies, HPE, Supermicro, Cisco, and ASUS—showcased multi-node Blackwell NVL72 deployments, ensuring that enterprises can procure and deploy these optimized systems across private, hybrid, and public cloud environments seamlessly.

Implications for Enterprise AI Economics

For business leaders and technology architects, the MLPerf Inference v6.1 results carry profound strategic implications.

1. The Compression of Cost Per Token

In generative AI, the cost to serve a single token dictates whether an enterprise application is economically viable. By dramatically increasing throughput per rack—while simultaneously driving down latency and power footprints—platforms like Vera Rubin NVL72 and GB300 NVL72 fundamentally compress the cost per token. Organizations can serve exponentially more users, process larger multimodal inputs, and maintain high-fidelity interactive sessions without incurring linear infrastructure cost expansions.

2. Preparing for the Agentic AI Era

Traditional benchmarking focused heavily on static throughput for single-turn text generation. However, as businesses transition to autonomous AI agents that must reason, browse, invoke APIs, and plan multi-step workflows, infrastructure bottlenecks have shifted. The preview results on agentic workloads—such as the 30x performance advantage demonstrated by Vera Rubin on agentic frameworks—signal that future-proofing enterprise IT requires hardware capable of handling complex control loops and dynamic reasoning states. Upcoming standardized tests, such as the forthcoming MLPerf Endpoints benchmark, will make this capability even more transparent.

3. Mitigating Obsolescence Through Software Velocity

The rapid pace at which AI models evolve creates a constant threat of hardware obsolescence. NVIDIA’s ability to extract an additional 1.6x performance boost on existing hardware (like the GB300 NVL72 running Qwen3-VL) purely through software optimization demonstrates that enterprise investments appreciate in value over time. Continuous updates to TensorRT-LLM, vLLM integration, NVIDIA Dynamo, and low-precision quantization techniques ensure that deployed infrastructure continuously extracts higher value from capital investments.

Conclusion

As AI factories scale from thousands to hundreds of thousands of accelerators, the margin for inefficiency shrinks to zero. Through the combination of the Vera Rubin architecture, the relentless scaling efficiency of the GB300 NVL72, and a robust global partner ecosystem, NVIDIA’s MLPerf Inference v6.1 results prove that the economics of AI are entering a new, highly optimized era. For enterprises looking to deploy profitable, scalable, and future-proof AI systems, the path forward relies on full-stack codesign, platform fungibility, and uncompromising software velocity.