GPUvsTPU

A GPU (graphics processing unit) is a processor with thousands of small cores that run the same calculation on lots of data at once. Built for graphics, it became the main hardware for AI after NVIDIA's CUDA platform, released in 2007, made it programmable for general computing. A TPU (Tensor Processing Unit) is an application-specific chip that Google designed only for the tensor math of neural networks; Google began using TPUs in its data centers in 2015 and announced them in 2016.

The difference is flexibility versus specialization. A GPU runs almost any parallel code, from games and video to simulations and custom AI operations, and nearly every AI library supports it. A TPU gives up that flexibility: its matrix unit is a systolic array that passes numbers from cell to cell, so each value read from memory is reused many times, and it mostly computes in low-precision formats such as bfloat16, which makes it very efficient for large, standard models.

Availability differs as much as design. GPUs are sold by NVIDIA, AMD and others as cards for desktops, laptops and servers, and every major cloud rents them. TPUs are offered mainly as a Google Cloud service, linked by a dedicated network into pods of thousands of chips, and programs reach them through the XLA compiler from JAX, TensorFlow or PyTorch, with JAX the most common choice.

A common misconception is that a TPU is simply a faster GPU. It is faster or cheaper for some workloads, such as large transformer models written in JAX, while models with many custom operations usually run more easily on GPUs, and a TPU can't render graphics at all. Another is that AI always needs one or the other: CPUs still run small models well, and phones and laptops include NPUs, small accelerators that follow the same idea as a TPU.