Vera Rubin's 10x Tokens per Megawatt — Will It Bring Down AI Prices?

Wed, 22 Jul 2026 21:13:54 +0800

According to the vendor benchmarks published by NVIDIA for DeepSeek-R1 inference, the Vera Rubin NVL72 delivers ten times the tokens-per-megawatt metric compared to its GB200 NVL72 counterpart. This is a rack-level system comparison, not a single-GPU conclusion.

This number is large enough to lead one to the next conclusion: since the same megawatt can do ten times the work, AI should become cheaper soon.

I think the general direction is right, but the conclusion was reached too quickly.

First, put this “10x” back into its testing context. It is a vendor benchmark by NVIDIA measuring DeepSeek-R1 inference workloads, rack-scale NVL72 versus rack-scale NVL72. It states how many tokens can be delivered per megawatt in power-constrained scenarios; it does not claim all models are 10x faster, nor that the total cost per million tokens has dropped by 90%. The publicly available materials are insufficient to convert this metric into real-world serving costs: training, long context, low concurrency, small batches, different precisions, and different service-quality targets could each produce a different picture.

However, this metric still matters. In projects constrained by power access, liquid cooling, or data center delivery, if the same power quota can run more inference, expansion does not have to wait for substations and data centers every time.

The problem is that electricity is just one line item on a cost sheet.

A Vera Rubin NVL72 rack isn’t just a matter of swapping out cards in older racks. New GPUs, interconnects, switching fabrics, liquid cooling, and deployment engineering all enter the capital expenditure together. Service providers also have to face depreciation cycles, financing costs, software migration, fault tolerance, and utilization. When machines sit idle, no matter how impressive the per-megawatt throughput looks, it doesn’t automatically translate into cheaper Tokens.

Simplify the per-unit service cost of the service provider as:

$$ \text{Unit Token Cost} \approx \frac{\text{Depreciation + Electricity + Data Center Network + Operations \& Maintenance}}{\text{Actually Sold Tokens}} $$

This is a simplified framework for explaining variable relationships, not a TCO calculated from public materials; the denominator here is the actual Tokens sold, not the device’s theoretical throughput. The per-megawatt Tokens baseline directly improves the inference capacity that can be supported per unit of electricity, and may reduce the electricity input per Token; the unit service cost improves through higher actual Tokens sold only when demand and utilization keep pace. The new system’s capital expenditure, depreciation, and deployment costs may also rise. Networking, storage, queuing, and scheduling can likewise become limiting factors — the system doesn’t run on a single GPU.

What happens first is not necessarily a price reduction. Service providers may first use the new capacity for longer context, faster response, higher concurrency, or to handle enterprise demands that were previously queued up; this is valuable for developers, yet the price list does not have to change immediately.

The real price drops will only come once new racks evolve from pilot projects into repeatable configurations in actual data center facilities, and once utilization is meaningfully driven up. Until then, whether cloud providers, model companies, and inference service providers are willing to pass efficiency gains on to customers will depend on competition. The more intractable problem lies on the demand side: if cheap inference immediately triggers a tenfold increase in call volume, the industry might end up with even greater aggregate spending and more data centers, rather than everyone feeling things are cheaper.

AI infrastructure produces both deflation and expansion at the same time. A single generation may use less power, and users may get better model capabilities, yet providers may end up building more data centers as a result, because new capabilities pull forth new use cases.

Therefore, Vera Rubin is ten times more like a ticket toward price reductions than a notice of price reductions. It improves inference capacity in power-constrained scenarios and leaves room for service providers to cut prices. Who ultimately captures this room depends on hardware pricing, depreciation, data center operations, utilization rates, and competition.

End users do not ultimately need to fixate on whether “APIs will immediately become ten times cheaper.” Two more down-to-earth signals are worth watching instead: whether the same budget can stably buy faster, longer, and more reliable inference, and once supply ramps up, how much of the efficiency providers will retain on their profit-and-loss statement versus how much they will pass through into the pricing sheet. The former may first manifest itself in the form of product capabilities; whether the latter occurs, and when, still depends on supply, utilization, and competition.

References

写作附记

Original Prompt

NVIDIA Vera Rubin has entered full-scale mass production, with Spectrum-6 networking and NVL72 energy efficiency improving by 10x. DeepSeek R1 inference testing shows that Vera Rubin NVL72 delivers a 10x increase in tokens throughput per megawatt compared to GB200 NVL72. Once widely deployed, will AI costs decrease? The energy efficiency improvement is evident, but hardware investment has also increased significantly.

Reasoning on Uncle Xiang's Notebook

Vera Rubin's 10x Tokens per Megawatt — Will It Bring Down AI Prices?

References

Original Prompt