One of Huawei’s mission is to help China become more chip independent. To really understand the significance of this, and how this is crucial to AI, let’s think of AI as a 5 layer cake.
Figure: A hierarchy representation of the AI stack. Applications are the user interface like Cursor, model layer is the actual AI model, such as GPT models. Infrastructure includes data centers, compute clusters, AI models are trained or run using them. Chips are orchestrated together in numbers to make infrastructures useful, like H100 from Nvidia. Last, chips are powered by electric energy.
- Layer 1 Energy; China has invested in sustainable energy since the 2000s; it is a global leader.
- Layer 2 Chips; This is the layer China lags behind (for now).
- Layer 3 Infrastructure; China is naturally fast at building; in Jensen Huang’s words: “They can build a hospital in a weekend.”
- Layer 4 Models like DeepSeek and Qwen are narrowing the gap quickly in many areas. China is "nanoseconds behind".
- Layer 5 Application. China is often exceptionally strong in application.
The only layer lacking is chips. Semiconductor is the most hardcore, capital intensive industry in the world. China is rapidly catching up. Within 10 years, for the first time in history, we might witness a country that has vertically integrated much of the semiconductor industry, allowing rapid iteration and strong adaptability.
After winning 1st in Huawei’s competition, I spent 6 months in Huawei’s 2012 Laboratories as a CPU Architecture intern, the highest level R&D of Huawei that aims to develop technologies 5–10 years into the future. Without giving away too much, I can say that I worked on memory architecture and branch predictor related work. It feels awesome to be part of a very big project. These experiences allowed me to gain hands-on experience understanding how computer microarchitecture works as a whole, which directly connects to the choke point of AI.
Why Chips?
As you know, the bottleneck of AI lies within chips. Data movement within memory accounts for a large share of energy cost in AI inference, prior work on DNN accelerators reports ~30% to 80% of system energy can be consumed by DRAM. More broadly, the energy cost of a DRAM access can be orders of magnitude higher than an on-chip operation. This is not sustainable.
However, some readers may not be aware that a long-term goal of AI is to build “Continuous learning”, where “inference is learning”, similar to how biological neural networks operate.
To achieve this we need to massively reduce AI cost so that we can iterate faster as “in analysing complexity, speed of iteration beats quality of iteration.” To achieve this there are a few points to take in mind.
We’re in a post Moore’s law era.
Let’s say you’re a software developer in the 2010s, you could create software that is slightly ahead of the computer processors capability and expect processors to get better and better each generation to catch up. In 2025, this approach is very dangerous: conventional scaling has slowed, and the underlying assumption has changed.
Figure: Growth in single threaded processor performance over 40 years, the slope representing performance increase acceleration, is flattening.
Now, you have to take compute capability into the equation. Coincidentally the most compute intensive application, AI, has emerged at this time. This points to one thing: software hardware co-design is a must in the future.
Recently Google offered a plan that’s significantly cheaper (and in some cases better) than other model providers. One underlying reason is vertical integration: Google runs models on its own TPU stack (custom accelerators + system/software stack), which can improve cost/performance and iteration speed. The power of hardware software co-design is beginning to show.
More friendly AI training loop
To understand this, let’s dive into the underlying engine of modern AIs. Deep Neural Network, comprises three parts.
Figure: Showcasing a classic DNN training loop.
- The first part is feedforward, where the model generates an output given input.
- Second, loss function calculation.
- Third, weights update, which uses back propagation to compute slope of gradient and update weights.
Of all three parts, back propagation takes the most energy and time. Performing chain rule layer by layer requires caching activations from feedforward part, training is memory intensive because back-propagation requires storing activations of intermediate layers. These burdens can’t be off loaded into larger storages like SSDs due to latency. We need to address this issue. There are methods like quantization and mixed precision trying to address this (often by reducing memory footprint/bandwidth pressure), which requires fundamental knowledge of how microarchitecture and memory hierarchy works.
The goal is not to build “AI” but to build systems that are useful. As the genius of Edison lies not in inventing lightbulb alone, but in building electric grids, laying out cable, and at the end massively lower cost of the entire system to the threshold of mass adoption, in other words Edison made light as a service, usable.