For decades, the semiconductor race has been defined by who controls the most powerful chips and the fabs that make them. In AI, that has meant Nvidia: an American company whose GPUs are the gold standard for training the world’s largest models, most of them manufactured in Taiwan.
But China’s strategic challenge is that training incumbency does not guarantee inference leadership. The decisive battleground is shifting from FLOPs and GPU allocations to something more prosaic but more binding: tokens-per-watt and capacity-per-rack. That shift creates a wedge—a narrow but real opening—for China to compete on efficiency rather than on access to restricted GPUs.
From “biggest model” to “cheapest token”
Training is episodic; inference is continuous. Every chatbot query, recommendation, and code suggestion is an inference event. At scale, the limiting factor is not theoretical compute but the cost of electricity and space in data centers. The International Energy Agency projects that global data-center electricity demand will more than double by 2030 to 945 terawatt-hours—roughly Japan’s entire annual consumption. In such an environment, metrics like tokens-per-watt and kilowatt-hours per billion tokens become central to both corporate strategy and national planning. Energy efficiency is not a peripheral consideration but a precondition for sustainable AI growth.
The Nvidia–Taiwan dependency
China’s cloud giants—Alibaba, Tencent, and Baidu—have relied heavily on Nvidia GPUs to build their generative AI platforms. Yet U.S. export bans restrict access to Nvidia’s most advanced processors, and the irony is stark: even when chips are acquired, they are largely fabricated in Taiwan, the “enemy next door” in Beijing’s calculus. This dual dependency leaves Chinese operators exposed on two fronts: to geopolitics that can throttle supply overnight, and to infrastructure economics that make GPU-based inference unsustainable at national scale.
Domestic progress, persistent limits
Domestic champions such as Huawei (Ascend series) and Cambricon have poured resources into developing AI accelerators. Yet progress has been incremental. Analysts estimate Huawei may ship only 200,000 AI chips in 2025—a fraction of global demand—and many of those units still rely on overseas packaging and advanced memory. The result is that China’s so-called “neo-clouds” remain a hybrid ecosystem: a patchwork of imported GPUs, gray-market workarounds, and still-maturing local alternatives. The unanswered question is whether domestic innovation can close the gap before reliance on foreign supply hardens into a structural liability.
Category proof: inference-first hardware
Globally, the market is beginning to validate the idea that inference requires different silicon. Positron, a U.S. company, recently raised $51.6 million from Valor Equity Partners, Atreides Management, and DFJ Growth to scale its Atlas accelerator. Atlas is not a GPU clone. It is built for inference efficiency, engineered around memory bandwidth rather than raw compute. The design achieves 93 percent memory bandwidth utilization—compared with roughly 30 percent typical in GPU deployments—and supports multi-model concurrency on a single card.
The payoff is visible in benchmarks. In vendor-reported tests, Atlas sustained about 280 tokens per second in Llama 3.1 8B within a 2,000-watt power envelope. By comparison, an Nvidia DGX H200 system consumed nearly 6,000 watts to generate around 180 tokens per second. That translates to three times the performance-per-watt, a difference that directly determines how many racks can be deployed within a fixed grid allocation. Atlas is already in production at hyperscalers such as Cloudflare and Parasail, proof that operators are beginning to procure on the basis of tokens-per-watt rather than brand loyalty.
Why this matters for China
For Beijing, the emergence of inference-first designs like Positron’s highlights both risk and opportunity. The risk is that continuing to scale AI on GPU clusters—restricted by U.S. export controls and reliant on Taiwan fabs—locks China into a cost structure and supply chain it cannot fully control. The opportunity lies in the fact that inference is less path-dependent than training. Software stacks are increasingly portable, and workloads can be benchmarked directly on tokens-per-watt and tokens-per-rack. That opens a lane where China can compete on efficiency economics rather than on access to scarce GPUs.
A practical path forward
If China’s AI ambitions are to scale sustainably under energy and supply constraints, its procurement strategies will need to evolve. Training will continue to depend on GPUs, where their versatility and throughput excel. But inference, which constitutes the vast majority of real-world usage, should increasingly be served on accelerators optimized for efficiency. Chinese operators can begin standardizing on new KPIs—tokens-per-watt, tokens-per-rack-unit, kilowatt-hours per billion tokens—that better reflect the real economics of serving. They should also favor systems designed for the grid they already have, rather than exotic cooling requirements that demand greenfield construction. This is how efficiency translates from a design philosophy into a deployment reality.
The open lane
China’s AI crossroads is not just about catching up to Nvidia or insulating from Taiwan. It is about recognizing that the ground has shifted. The decisive contest is inference efficiency—who can serve the most tokens within the strict limits of power, cooling, and space. In that contest, training incumbency does not guarantee victory. The wedge is real, and it is open. For China, exploiting it will mean the difference between scaling AI on its own terms—or remaining bound by foreign hardware and unsustainable costs.
