Google tops Gartner's AI infrastructure magic quadrant

Google has been named a Leader in Gartner's inaugural Magic Quadrant for AI Infrastructure, ranking highest for Ability to Execute and furthest for Completeness of Vision.

The recognition focuses on Google Cloud's AI infrastructure business, where Google is seeking to expand its position among customers building and running large AI models and agent-based systems.

Its infrastructure work builds on systems developed internally for products including Gemini, YouTube, and Search. Google says engineers and researchers have spent more than a decade co-designing hardware and software for those workloads, and that the resulting technology is also available through Google Cloud.

According to Google, its AI stack is used by nine out of 10 frontier AI labs, as well as customers including Citadel Securities and Mercedes-Benz. It also pointed to custom silicon as one of the areas highlighted in Gartner's assessment.

Silicon push

Earlier this year, Google introduced two new generations of tensor processing units, or TPUs, aimed at different parts of the AI workflow. TPU 8t is for training, while TPU 8i is designed for inference workloads tied to agent-based applications.

Google says TPU 8t can link 9,600 chips in a single superpod and delivers nearly three times the compute performance per pod of the previous generation. For inference, TPU 8i includes 288 GB of high-bandwidth memory and 384 MB of on-chip SRAM, three times more than the previous generation, according to Google.

Google is not relying solely on its own chips. It continues to work with Nvidia to offer GPU-based systems through Google Cloud and plans to provide A5X instances based on Nvidia's Vera Rubin platform when that becomes available.

It is also backing open-source software for orchestration and inference, including llm-d and vLLM, and recently announced TorchTPU for PyTorch developers who want to move workloads without extensive code changes.

Integrated stack

Gartner also recognised Google's AI Hypercomputer offering, which combines hardware, networking, storage, and software into a single system for AI training and inference, according to Google. The company argues that customers are increasingly looking for lower costs and better utilisation as spending on AI infrastructure rises.

On storage, Google says Managed Lustre, using C4NX instances and Hyperdisk Exapools, now delivers 10 TB/s of bandwidth. It also says Rapid Buckets can provide up to 20 million operations per second for object storage, helping speed checkpointing and recovery for large training runs.

The network layer is another focus. Google says its Virgo Network can connect more than one million TPUs across multiple data centre sites in a training cluster, or up to 960,000 GPUs across multiple sites without performance degradation.

For model serving, Google says GKE Inference Gateway combines routing, caching, and disaggregated serving through llm-d. According to the company, the service can increase throughput by up to 40% while reducing serving costs by up to 30%.

Scaling workloads

Google is also pitching the platform as a flexible environment for customers running AI workloads across cloud, edge, and on-premises systems. It says Cluster Director and Google Kubernetes Engine can scale training to 130,000 nodes.

For agent-based workloads, Google highlighted GKE Agent Sandbox, which it says can provision up to 300 sandboxes per second per cluster and scale back when demand falls. The goal is to handle bursty workloads while reducing idle compute spending.

Google also says Cross-Cloud Network and Cloud WAN allow customers to run distributed enterprise and AI workloads across multiple environments using Google's private backbone. According to the company, that network spans more than 10 million kilometres of fibre and reaches more than 200 countries and territories.

The ranking comes as large cloud providers and chipmakers compete to supply the infrastructure behind generative AI and newer agentic systems. Demand in that market has shifted from raw compute alone to a broader mix of silicon, networking, storage, orchestration software, and operating costs.

In that context, the Gartner placement gives Google an external endorsement as it competes with rivals for AI infrastructure spending from model developers, financial firms, and large enterprises. Google says the report recognised its commitment to custom silicon and its integrated AI infrastructure approach as core strengths.