Chip designer Arm has entered the artificial intelligence (AI) hardware arena with its first in-house processor designed to power AI agents. Unlike conventional chatbots, these are much smarter systems that can take proactive actions to achieve their goals without as much human input or supervision.
By focusing specifically on powering AI agents, Arm’s chip could help accelerate the adoption and widespread use of agentic AIs, be that in businesses or in one’s personal life, bringing AI much closer to what people would expect from virtual assistants.
Think of a CPU as the conductor of an orchestra of GPUs and other AI accelerators — hardware that’s specifically designed to run LLMs — in this case.
As such, Arm representatives announced in a statement that its new AGI CPU has a custom design — including 3-nanometer process nodes, up to 136 Neoverse V3 cores that can hit 3.7 GHz clock speeds, and a memory bandwidth of 6 gigabytes per second per core — for use in data centers that are powering active AI agents.
All of these capabilities aim to meet the goal of providing better performance and efficiency than classical CPUs that use the x86 architecture, the dominant computing architecture that was developed by Intel in 1978 and is still used in processors today.
Custom chip future
With the inexorable growth of AI and the deployment of smart agents, there’s a need for more data-center-based hardware to power these systems. However, the general-purpose nature of CPUs means they aren’t intrinsically designed to run the specific orchestration needed for agentic AIs.
Arm’s AGI CPU uses the Armv9.2-A architecture at its core. This architecture has been designed with the specialized needs of running AI in action — known as inference. With this specialty, there’s no need for an AGI CPU to hold legacy support for other processes and applications, as seen in x86 chips — conventional processors used in regular computers.
This should make for faster and more efficient performance targeted at AIs. Arm representatives said that its AGI CPU delivers more than twice the performance per server rack versus x86 CPUs.
The AGI CPU has been designed to pack two chips with dedicated memory and in-out (I/O) functionality into a single server blade with a total of 272 cores per blade. The blades can then be stacked into server racks of 30, delivering a total of 8,160 cores with sustained performance for agentic AI workloads at a “massive scale,” thanks to thousands of cores working in parallel.
Arm’s speciality in chip design centers on offering strong performance for relatively lower power consumption. That’s one of the reasons all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in Snapdragon chips and Apple uses it in its iPhone and MacBook chips.
As AI continues to transition from training LLMs to actively deploying agentic AIs, there will be an increased need for CPU-based processing power in data centers. This is expected to drive a huge increase in AI energy demand.
The AGI CPU has been designed to pack two chips with dedicated memory and in-out (I/O) functionality into a single server blade with a total of 272 cores per blade. The blades can then be stacked into server racks of 30, delivering a total of 8,160 cores with sustained performance for agentic AI workloads at a “massive scale,” thanks to thousands of cores working in parallel.
Arm’s speciality in chip design centers on offering strong performance for relatively lower power consumption. That’s one of the reasons all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in Snapdragon chips and Apple uses it in its iPhone and MacBook chips.
As AI continues to transition from training LLMs to actively deploying agentic AIs, there will be an increased need for CPU-based processing power in data centers. This is expected to drive a huge increase in AI energy demand.
Keumars Afifi-Sabet
Arm has the potential to really shake things up in what’s become something of an arms race in computer chips. If it can offer CPUs that deliver strong AI inference performance while being more efficient than x86-based CPUs, it could dampen the rising energy demand while also disrupting Intel, AMD and hardware giant Nvidia, which has its own Arm-based Vera CPUs.
This architecture is already used in chips for AI data centers, and so the chip designer is in a strong position to make its own foray into providing “off-the-shelf” CPUs.
While Arm has traditionally licensed its designs to other chipmakers, the AGI CPU will be its first attempt to make hardware other companies can buy and deploy in their data centers. It points to a future in which more hardware is custom-designed to power AI, whether it’s to run LLMs more efficiently, as seen with the application-specific integrated circuit (ASIC) architecture found in Google’s TPU and Amazon’s Trainium chip, or for inference, in the case of Microsoft’s Maia 200 chip.
Custom chips that can overcome some of the hardware constraints of operating AI at a large scale could disrupt the traditional makeup of general computing hardware in data centers. This, in turn, could accelerate the path to artificial general intelligence (AGI), a hypothetical AI system that can learn, understand, and apply knowledge across multiple domains at a human-level or beyond.