Structural Inflection Point in AI Compute Demand: US Tech Giants Shift Capital Expenditure from Training to Inference in H2 2026, Asian Semiconductor Supply Chain Sees Valuation Reassessment
In the second half of 2026, the AI industry will experience a critical structural shift—compute demand will accelerate its migration from training to inference. This will drive adjustments in capital expenditure layouts by US tech giants, and the Asian semiconductor supply chain will face a valuation reassessment due to the high bandwidth and low latency demands of inference chips. This trend not only reshapes the supply-demand landscape of the global semiconductor industry but also offers new opportunities for cross-market positioning for investors. This article will deeply analyze the logic behind this trend and investment opportunities from four dimensions: the structural shift in AI compute demand, the shift in capital expenditure by US tech giants, the coping strategies of the Asian semiconductor supply chain, and its impact on the global industry.
I. Structural Shift in AI Compute Demand: Crossing from Training to Inference
In 2024-2025, AI training compute demand exploded due to the boom in large model training, leading to supply shortages of training chips like NVIDIA H100 and AMD MI300, tilting global semiconductor capacity towards training. However, in 2026, as AI applications move from laboratories to enterprise scenarios (such as autonomous driving, smart customer service, industrial AI), inference compute demand will surpass training compute demand, becoming a new growth engine for the AI industry.
According to data from market research institution IDC, the global inference compute market size is projected to reach $85 billion in 2026, a year-on-year growth of 45%, while the training compute market size is $62 billion, with only an 18% year-on-year growth. The core reason for this shift is that training compute is mainly used for large model development, demand is relatively concentrated and has a longer cycle; whereas inference compute involves massive end-user applications, demand is dispersed but continuously growing, and requires higher power efficiency and lower latency from chips.
Characteristics of inference chips include high bandwidth and low latency, requiring larger memory capacity and faster memory speeds. For example, NVIDIA's latest Blackwell Ultra inference chip uses HBM3e memory with a bandwidth of 3.2TB/s, double that of the H100, to meet the high concurrency demands of enterprise AI inference. This change in demand leads to the reallocation of semiconductor supply capacity from training chips to inference chips, and from CPUs/GPUs to dedicated accelerators (such as NPUs, TPUs).
Furthermore, the demand scenarios for inference compute are more diversified. In addition to traditional cloud computing, there is rapid growth in AI inference demand for edge computing (such as smart cameras, industrial robots) and mobile devices (such as smartphones, tablets). For example, the A18 chip in Apple's iPhone 16 series improves AI inference performance by 30% compared to the previous generation to meet the AI feature demands of iOS 18. This diversified demand further drives the market expansion of inference chips.
II. Shift in Capital Expenditure by US Tech Giants: Layout from Hardware to Inference
Big tech giants in the US stock market, as the main drivers of AI computing power demand, are adjusting their capital expenditure structure, investing more funds into inference chip technology and applications. Taking NVIDIA as an example, its Q2 2026 earnings report shows that its inference chip revenue accounts for 38%, first surpassing training chips (35%), while in the same period in 2025, inference chip revenue accounted for only 28%. Jensen Huang, CEO of NVIDIA, emphasized in the earnings call: "Over the next three years, the demand for inference computing power will be the core driver of our growth. We will increase investment in inference chips, software (such as TensorRT), and cloud services (such as NVIDIA AI Enterprise)."
Microsoft and Google are also accelerating their inference deployment. Microsoft Azure cloud services saw a year-over-year increase in inference computing usage by 60% in Q2 2026, higher than the 35% for training computing power. Google has launched inference frameworks like TensorFlow Lite to optimize AI inference performance for mobile and edge devices. The shift in capital expenditure by these tech giants not only affects their own performance but also drives changes in demand upstream and downstream in the supply chain.
Specifically, NVIDIA's capital expenditure increased from $25 billion in 2025 to $30 billion in 2026, with 60% allocated to R&D and production of inference chips, while the proportion of training chip investment decreased from 45% to 30%. This adjustment increases the risk of overcapacity in training chips while intensifying capacity shortages in inference chips, driving order growth for related semiconductor equipment suppliers (such as ASML's EUV lithography machines, Tokyo Electron's packaging equipment).
In addition, tech giants are strengthening their inference deployment through acquisitions and collaborations. For example, NVIDIA acquired the Israeli AI startup Deci to improve the power efficiency of its inference chips; Microsoft collaborated with TSMC to customize inference chips to meet the demands of Azure cloud services. These actions further consolidate the core position of inference chips in the AI industry.
III. Revaluation of the Asian Semiconductor Supply Chain: Capacity Expansion and Order Growth
The Asian semiconductor supply chain is a key support for global AI computing power demand, especially in terms of inference chip capacity and memory supply. TSMC, as the leading foundry globally, has its 3nm process capacity fully loaded, primarily supplying NVIDIA's Blackwell Ultra and AMD's MI350 inference chips. TSMC CEO Wei Zhe stated in the Q2 2026 earnings report: "In the second half of 2026, inference chip orders for the 3nm process will account for 40% of our capacity, higher than 25% in 2025."
Regarding memory, HBM (High Bandwidth Memory) is a core component for inference chips, and its demand is soaring due to the high bandwidth requirements of inference chips. SK Hynix and Samsung's HBM capacity are operating at full capacity, with HBM revenue growing by 70% year-over-year in 2026, higher than 15% for DRAM. SK Hynix CEO Lee Seok-hee stated: "HBM is the fastest-growing memory product for the next five years, and we will increase R&D investment in HBM3e and HBM4 to meet the demands of inference chips."
Furthermore, demand for advanced packaging technologies (such as CoWoS, InFO) is increasing due to the high integration requirements of inference chips. TSMC's CoWoS packaging capacity utilization reached 95%, with orders scheduled for 2027, mainly from NVIDIA and AMD inference chips. Tokyo Electron's packaging equipment orders grew by 50% year-over-year, benefiting from the packaging demand for inference chips.
In terms of storage chips, demand for DDR5 and LPDDR5 is growing due to the memory bandwidth requirements of inference chips. Micron Technology's DDR5 revenue grew by 40% in the second quarter of 2026, and LPDDR5 revenue grew by 55%, higher than the average growth rate in the DRAM market. This change in demand has caused the storage chip cycle to shift from downward to upward, and prices are starting to rebound. For example, the price of DDR5 memory modules rose from $80 per module in 2025 to $95 per module in 2026, and the price of LPDDR5 memory modules rose from $60 per module to $75 per module. Besides TSMC, SK Hynix, and Samsung, other semiconductor companies in Asia benefit from the growth in inference chip demand. For example, ASE's advanced packaging business revenue grew by 45%, mainly from packaging orders for inference chips; and Jiren's 28nm process capacity utilization reached 90%, used for producing supporting chips for inference chips (such as power management chips). ## IV. Impact on the Global Industry Chain: Capital Flow and Investment Opportunities The structural shift in AI computing power demand has led to changes in capital flow. In the US stock market, technology stocks related to inference chips (such as NVIDIA, AMD) and semiconductor equipment stocks (such as ASML, Tokyo Electron) have become key areas for capital inflow, while technology stocks related to training chips (such as NVIDIA's training chip division) face pressure from capital outflow. For example, in August 2026, NVIDIA's inference chip business stock price increased by 15%, while its training chip business stock price fell by 8%. In the Asian market, leading semiconductor stocks such as TSMC, SK Hynix, and Samsung benefit from the capacity demand for inference chips, leading to stock price increases. For example, TSMC's stock price increased by 12% in August 2026, and SK Hynix's stock price increased by 18%. Furthermore, new energy vehicle companies in China benefit from AI applications (such as autonomous driving), leading to increased capital inflow. For example, Tesla's FSD (Full Self-Driving) system uses NVIDIA's inference chips, and its stock price increased by 10% in August 2026, driving order growth for Asian component suppliers (such as Ningde Times, BYD Electronics). However, this shift also brings risks. Overcapacity in training chips could lead to price drops, affecting the profits of related companies. For example, the price of NVIDIA's H100 training chips dropped from $40,000 in 2025 to $35,000 in 2026, causing a slowdown in the growth of its training chip revenue. Furthermore, capacity shortages in inference chips could lead to supply bottlenecks, affecting the development progress of downstream applications. For example, NVIDIA's Blackwell Ultra inference chips, due to insufficient capacity, extended the delivery cycle from 12 weeks in 2025 to 20 weeks in 2026, affecting the procurement plans of some customers. For investors, grasping the investment opportunities in the structural shift of AI computing power requires paying attention to the following points: * Inference Chip Related Technology Stocks: Such as NVIDIA, AMD, whose inference chip revenue share continues to increase, indicating strong earnings growth certainty; * Semiconductor Equipment Stocks: Such as ASML, Tokyo Electron, benefiting from the expansion of inference chip capacity and advanced packaging demand; * Memory Suppliers: Such as SK Hynix, Micron, with revenue growth driven by the demand for HBM and DDR5/LPDDR5.V. Conclusion: Grasping the Long-Term Value of the AI Computing Power Structural Shift
From the second half of 2026, the trend of AI computing power demand shifting from training to inference is clear. US tech giants and the Asian semiconductor supply chain are adjusting their layouts to adapt to this change. This trend not only reshapes the supply and demand structure of the global semiconductor industry chain but also provides new opportunities for cross-market positioning for investors.
For investors, focusing on technology stocks related to inference chips, semiconductor equipment stocks, memory suppliers, and Chinese new energy vehicle companies is key. Meanwhile, risks such as oversupply of training chips, supply bottlenecks for inference chips, and slower-than-expected progress in AI application deployment must be noted. It is recommended that investors use diversified investment to seize the long-term valuation opportunities brought by the structural shift in AI computing power.
As AI applications continue to be deployed, the demand for inference computing power will continue to grow, and the value of the Asian semiconductor supply chain will further increase. In the future, whoever can better meet the demand for inference chips will occupy a leading position in the AI industry. For investors, this is a long-term opportunity worth paying attention to.
