TPU vs GPU vs NPU: AI Chips Explained

A GPU is a general-purpose parallel processor, a TPU is Google's matrix-math ASIC built for large neural networks, and an NPU is a small, power-frugal AI block inside a phone or laptop chip. All three run the same basic operation, multiplying big grids of numbers. They differ in how flexible they are, how much memory they sit next to, and how much power they are allowed to burn.

The short version: GPUs are the default for training and serving models in data centers and on desktops, TPUs are rented from Google Cloud, and NPUs handle small models locally on battery power. The sections below explain why, with official specifications as of October 2026.

The three chips in one paragraph each

GPU: many flexible cores, huge memory bandwidth

A graphics processing unit runs thousands of threads at once. Nvidia's data-center parts add dedicated tensor cores for matrix math on top of ordinary programmable cores, so the same chip can train a model, run simulations or render graphics. That flexibility is the main reason GPUs dominate. For the product lineage, see our Nvidia GPU architectures timeline.

TPU: a systolic array built around one job

A tensor processing unit is an application-specific chip designed by Google for machine learning. Its core is the matrix unit, a 128x128 systolic array: data flows through a grid of multiply-accumulate cells without constantly returning to memory. According to Google Cloud's TPU documentation, code must be compiled by the XLA compiler, and the chips suit models dominated by matrix computation rather than programs with frequent branching.

NPU: low-power inference next to the CPU

A neural processing unit is an accelerator inside a system-on-chip, alongside the CPU and GPU cores. It targets sustained, low-power inference such as background blur, transcription and image features. Microsoft's Copilot+ PC guide says NPUs use energy on AI tasks more efficiently than a CPU or GPU, which helps battery life. It also notes that many NPUs support only lower-precision integer math, so models must be quantized (converted to formats such as INT8) before they run.

TPU vs GPU vs NPU at a glance

Where you find it: GPU in servers, workstations and gaming PCs; TPU in Google Cloud (and Google's own data centers); NPU in phones, tablets and laptops.
Flexibility: GPU is highest; TPU is narrower and depends on the compiler; NPU is the most restricted, with a limited set of supported operators.
Typical job: GPU for training, large-scale inference and graphics; TPU for large training and serving runs on Google's stack; NPU for small on-device models.
Power budget: data-center racks for GPU and TPU, a few watts to a few dozen watts for an NPU.
Software: CUDA or ROCm for GPUs; XLA with JAX, PyTorch/XLA or TensorFlow for TPUs; ONNX Runtime, Windows ML or Core ML for NPUs.

Architecture: why the same math looks so different

Neural networks are mostly matrix multiplications. A GPU spreads that work across many cores and relies on fast memory to feed them. A TPU hard-wires the pattern into a systolic array, which removes much of the control logic and memory traffic and leaves more silicon for arithmetic. An NPU does something similar at a much smaller scale, trading peak speed for efficiency.

The cost of specialization is rigidity. Google's documentation says that when a matrix does not fill the matrix unit, the compiler pads it with zeros, which wastes capacity, and that dynamic tensor shapes are a poor fit. A GPU tolerates irregular code far better, which is why researchers often prototype there first.

Memory matters as much as compute

Large models are usually limited by how fast weights can be read, not by raw arithmetic. That is why data-center accelerators sit next to stacks of high-bandwidth memory; our HBM4 vs HBM3E explainer covers the technology, and the CoWoS packaging guide covers how the stacks are attached to the processor.

Two official examples as of October 2026:

Google TPU7x (Ironwood): Google lists 4,614 TFLOPs of peak FP8 compute per chip, 192 GiB of HBM at about 7.37 TB/s, and pods of up to 9,216 chips.
Nvidia DGX B200 (Blackwell): Nvidia lists 1,440 GB of HBM3e across the eight GPUs in the system, 64 TB/s of aggregate memory bandwidth, and 36 PFLOPS of dense FP8 for the whole system. Those are system figures, not single-chip figures.

Do not line these up as a head-to-head benchmark. One is a per-chip number from a cloud documentation page, the other an eight-GPU server, and peak throughput says little about real model speed.

NPUs rarely have dedicated HBM. They share system memory with the CPU and GPU, which caps the model size they can run well. That is one reason phones and laptops run compact, quantized models rather than frontier-scale ones.

Software stacks decide what you can actually use

GPU: Nvidia's CUDA has the broadest ecosystem; AMD offers ROCm. Most open-source AI tooling targets these first.
TPU: models are compiled through XLA. Google's documentation names JAX, PyTorch (through PyTorch/XLA) and TensorFlow.
NPU: on Windows, Microsoft now recommends Windows ML, which uses ONNX Runtime and picks a vendor execution provider such as Qualcomm's QNN or Intel's OpenVINO, falling back to the GPU or CPU if the NPU path fails. Apple devices expose their Neural Engine through its own frameworks.

In practice, the software gap often outweighs the hardware gap. A chip that a framework does not support well is slower for you than a weaker chip that it does.

What TOPS means, and why it misleads

TOPS stands for trillions of operations per second. It is a peak figure for a given number format, usually low-precision integers on NPUs. Microsoft's developer guide says many new Windows AI features need an NPU that runs at 40+ TOPS, but that is an entry threshold, not a speed rating.

Three problems make TOPS a poor comparison tool:

Precision differs. An INT8 or FP4 figure is not comparable with an FP16 or FP32 one, and data-center vendors often quote sparse numbers that are double the dense ones. Nvidia's own page states that its dense FP8 figure is half the sparse one.
Memory is the bottleneck. A large language model generating text is limited by memory bandwidth, so extra TOPS may change nothing.
Software support varies. Peak numbers assume every operator in the model runs on the accelerator.

Apple is a useful reminder: its iPhone 17 Pro announcement describes a 16-core Neural Engine working with the GPU, without leading on a TOPS number.

NPU vs GPU for running a local LLM

Running a chatbot-sized model locally is mainly a memory problem. A discrete GPU with plenty of fast video memory usually generates text faster than a laptop NPU, while the NPU can sip power and leave the GPU free. Which one wins depends on the model size, the quantization, and whether the runtime you use supports your NPU at all. Check your tool's supported-hardware list before buying a laptop for this purpose.

Which one should you care about?

Building or renting AI infrastructure: compare GPUs and TPUs on cost per token or per training run on your own workload, and factor in your code's compiler support.
Buying a laptop or phone: an NPU matters for battery life and built-in AI features; it will not replace a GPU for heavy models.
Following the industry: custom accelerators from cloud providers are a growing share of the story. Our chips coverage and Big Tech section track the announcements.

Frequently asked questions

Is a TPU faster than a GPU?

Not universally. A TPU can be very efficient on large, matrix-heavy workloads that fit Google's software stack, while a GPU handles a wider range of code. Speed depends on the model, the batch size and how well the compiler maps the work.

Can I buy a TPU?

Google offers TPUs as a cloud service through Google Cloud rather than as a card you install in your own PC. Check Google Cloud's documentation for current regions and availability.

Is an NPU better than a GPU for AI?

An NPU is better at running small models for long periods on little power. A GPU is better when you need raw speed, large memory or flexibility, such as training or running big models.

What is a good TOPS number for an NPU?

Microsoft says many Copilot+ PC features on Windows need an NPU rated at 40+ TOPS. Beyond that threshold, TOPS figures from different vendors are hard to compare because precision and memory behavior differ.

What is the difference between a TPU and a CPU?

A CPU runs a few complex threads and handles general code. A TPU is a fixed-purpose accelerator that performs matrix operations in bulk and needs a host to run the rest of the program.