AMD takes a deep dive into architecture for the AI PC chips

Advanced Micro Devices executives revealed the details of the chipmaker’s latest AI PC architecture, which includes a newneural processing unit (NPU) in the company’s latest AMD Ryzen AI chips.

The company announced the latest AMD Ryzen AI processor (code-named Strix Point) and other next-generation technologies at the Computex trade show in Taiwan last month, and then it took a deep dive into the design at a recent press event in Los Angeles.

AMD said that NPUs are must-have components for AI PCs, and the exponential growth and specialization of AI workloads requires a new compute architecture. The size and diversity of models is growing and becoming increasingly integral to the operating system.

Vamsi Boppana, SVP of the AI group at AMD, said in a presentation that neural programming units (NPUs), which speed up neural network calculations used for AI applications, can operate on AI models at 35 times the performance per watt of a standard central processing unit (CPU). And NPUs can also achieve eight times the performance of a integrated graphics processing unit (iGPU), which combines a CPU and a GPU on the same chip.

Jack Huynh, SVP of computing at graphics at AMD, said it was an epic journey for AMD as it moves into the AI era with innovations in processors, starting with AMD’s Zen architecture in 2017, a 7nm CPU in 2019, 3D SoCs in 2020, and its first NPU for PCs in 2023. Now it’s introducing an NPU with best overall performance for AI PCs, Huynh said.

“AMD has a long and extensive history of firsts,” Huynh said.

He said AI PCs are important because they can contain the data to your own local network, ensuring privacy in your AI work and play. AMD has had more than 300 design wins for its three generations of AI processors.

XDNA 2

AMD said the Ryzen AI series introduced the world’s first x86 processor integrated with an NPU. And the company said there are more than 100 AI powered experiences across a broad set of computer makers including Acer, Asus, Dell, HP and Lenovo. Asus said its AMD-based AI PC laptops are debuting at Best Buy later this month.

The company is now on its third generation of AMD Ryzen AI chips, and this one will have 16 RDNA 3.5 GPU compute units, 12 Zen CPU cores that can compute 24 threads at a time, and an NPU that can do 50 TOPS thanks to the AMD XDNA 2 architecture.

AMD said it has more than 150 AI powered software vendors in 2024 launching applications, including ones for immersive collaboration, revolutionary creating and editing, games and entertainment, personal AI assistance, and enterprise productivity.

AMD refers to its NPU architecture as built on its XDNA foundation. It consists of a series of AI Engine tiles spread across a single chip, with connections to memory titles. That’s much different than a traditional multi-core processor with cores that are connected via different levels of memory.

The designs also have programmable interconnections and flexible partitioning. The XDNA 2 architecture has resulted in chips that have leadership performance, AMD said.

The XDNA 2 chips will have 32 AI engines and 50 NPU TOPS when it comes to performance. That compares to 20 AI engines and 10 NPU TOPs for the first XDNA.

AMD said the XDNA 2 chips have five times the compute capacity of the AMD Ryzen 7040 Series chips and two times the power efficiency. The NPU floating point focuses on 16-bit for greater accuracy.

AMD says that the result is the most powerful NPU for next-gen AI PCs, with performance coming out at 50 TOPs, equivalent to peak float16 TFLOPS. That is more performance than the Apple M4 ANE, the expected Intel “Lunar Lake” NPU and the Qualcomm Snapdragon Elite X NPU (estimated at 45 TOPS). HP also announced today that it can hit 55 TOPS on its AI PCs thanks to its close work with AMD.

AMD said all of Microsoft’s AI models are up and running now, including those for perceptive shell, generative AI and collaboration/communication. AMD claims its LLM performance on Meta’s Llama2 model is over five times faster than the Intel Core Ultra 7 155H NPU.

AMD has a unified AI software stack with open source platforms. AMD said that it will have leadership across multiple AI PC generations. Asus said that you can save four hours in AI processing using its AMD-based desktop.

Zen 5 architecture

Meanwhile, AMD CTO Mark Papermaster said described the Zen 5 architecture, which is part of an architectural Zen series that has kept AMD ahead of rival Intel in the x86 microprocessor market in recent years. The Zen 5 CPU core is making its appearance in the upcoming Granite Ridge desktop processors for gamers, developers and content creators.

Papermaster said the designs have more instructions per clock cycle, dispatch and execution expanded width, doubled cache data bandwidth and AI acceleration. It has features such as wider dispatch and execute abilities, increased data bandwidth for load/store work and a 512-bit AI datapath.

“There will be no letup. Zen 5 will not disappoint you. It represents a huge leap forward and it is a pedestal on which we will build the next generations of Zen,” Papermaster said. “We redesigned key elements on the front end. It yields more instructions to the backend. We feed the beast and we avoid execution stalls.”

Overall, Zen5 delivers 16% average instruction per clock cycle improvements over Zen5 on a wide variety of games and applications. On math, the chip also has up to 32% single-core machine learning and up to 35% single core AES-XTS processing.

AMD said the Zen 5 architecture will be used across four-nanometer and 3nm process technology. It will have faster, smaller and lower power transistors. And there are multiple products in development.

The first one is 3rd Gen AMD Ryzen AI, code-named Strix Point. AMD said its 5th Gen AMD Epyc processors are coming out in the second half of 2024. These will have Zen 5 architecture, up to 192 cores and 384 threads, confidential AI-based on new Trusted IO features and 4nm and 3nm process technologies.

Papermaster also said AMD foresees AMD leading the way on CPUs with its roadmap for Zen 6 designs in the future.

As for the RDNA 3.5 graphics features, Papermaster said AMD expects Strix Point to have 32% better performance on 3DMark Timespy benchmark compared to the prior Hawk Point, and it also expects 19% better performance on 3DMark Night Raid.

The future of AI PCs

Sebastien Nussbaum, corporate vice president for computing and graphics at AMD, said AI processing will mark a new era in computing in the 2020s, with processing for generative intelligence, agentic AI and artificial general intelligence.

He noted there are more than 740,000 AI models available (https://hugginface.co/models). And there were 15.5 billion AI-generated images created in 2023, based on stats from Techreport.com. And the model compute size has expanded 1,000 times in the last decade, and today there are 314 million AI tool users in the world in 2024, according to Statista.

And he noted the age of the AI PC is here, with Microsoft’s Copilot showing off AI-assisted operating system functions. In the future, we’ll see more natural language-based human and computer interaction, agentic AI and seamless, always on, AI data-driven user experiences.

AMD predicts transformational experiences when it comes to collaboration, creation, assistance and gaming/entertainment, Nussbaum said. He noted that local AI PCs have inherent advantages over cloud AI with enhanced privacy and security, reduced latency and response time, and reduced cloud costs.

Nussbaum expects people to take a hybrid approach between the edge and the cloud. They could design cars using AI in the home, and then they could turn the final output of the design to a data center.

David McAfee, AMD corporate vice president for graphics and client channel, dove into the detials at the briefing on the AMD Ryzen 9000 series processor, the consumer processor unveiled at Computex 2024. There are four processors in the series, ranging from six Zen cores (12 threads) to 16 cores (32 threads) and 65 watts to 170 watts of power.

McAfee said during the briefing, “Our industry is being reshaped by AI, and AI PCs are reshaping the way that we work, communicate, play — it’s an incredible moment in history and AMD is leading this transition.”

McAfee said the AMD Ryzen 9 9900X will have gaming leadership versus Intel’s Core i9 14700K processor, with up to 22% better performance on Horizon Zero Dawn and 41% better performance on Handbrake on the productivity side. The AMD Ryzen 7 9700X will have 31% better performance on Horizon Zero Dawn versus Intel’s Core i7 14700K.

The Ryzen 9700X is 130% faster on Warhammer Dawn of War 3 than the 105-watt Ryzen 7 5800X3D, while using less power. The processor has a 15% thermal resistance improvement and 7% temperature reduction. It also has a lot of overclocking features, and it has a new chipset on the AM5 platform.

On AI, McAfee said that the AMD Ryzen 9 9900X performs 17% faster on tokens per second versus the Intel Core i9 14900K on Llama, and 20% faster on Mistral.

And Jason Banta, corporate vice president for business development on client at AMD, said AMD Ryzen AI 300 processors can outperform both the Intel Core Ultra 9 185H and the Qualcomm Snapdragon X Elite X1E-84-100 on gaming, with 1.65 times better performance on Cyberpunk 2077.