GPU Glossary
/
Device Hardware·2 min read

CUDA Device Architecture

The unified GPU hardware design behind CUDA, built from many identical, general-purpose Streaming Multiprocessors instead of a fixed graphics pipeline.

CUDA stands for Compute Unified Device Architecture. The name covers several distinct things: a high-level device architecture, a parallel programming model for hardware with that design, and a software platform that extends languages like C with that programming model. This article covers the device architecture.

The vision for CUDA is set out in the Lindholm et al., 2008 white paper. It is well worth reading: it is the original source of many claims, diagrams and even turns of phrase in NVIDIA's documentation.

Key idea The defining feature of a compute unified device architecture is simplicity: one kind of uniform, general-purpose hardware unit instead of many specialized ones.

Before CUDA: fixed pipelines#

Before the GeForce 8800 and the Tesla data center GPUs that followed it, NVIDIA GPUs used a complex pipeline shader architecture. Software shader stages were mapped onto heterogeneous, specialized hardware units.

That design was hard on both sides. Software engineers had to map their programs onto a fixed pipeline, and hardware engineers had to guess the load ratios between pipeline steps.

A fixed-pipeline device architecture (G71), with separate groups of processors for fragment and vertex shading. Adapted from Fabien Sanglard's blog.
A fixed-pipeline device architecture (G71), with separate groups of processors for fragment and vertex shading. Adapted from Fabien Sanglard's blog.

The unified design#

In a unified architecture, the hardware units are entirely uniform, and each can perform a wide range of computations. These units are the Streaming Multiprocessors (SMs). Their main subcomponents are CUDA Cores and, on recent GPUs, Tensor Cores.

A compute unified device architecture (G80). There are no distinct processor types: all meaningful computation happens in the identical Streaming Multiprocessors in the center, fed with instructions for vertex, geometry and pixel threads. Modified from Peter Glaskowsky's 2009 white paper on the Fermi Architecture.
A compute unified device architecture (G80). There are no distinct processor types: all meaningful computation happens in the identical Streaming Multiprocessors in the center, fed with instructions for vertex, geometry and pixel threads. Modified from Peter Glaskowsky's 2009 white paper on the Fermi Architecture.

Further reading#