GPU Glossary
/
Device Hardware·1 min read

GPU Core

The primary compute units inside a Streaming Multiprocessor, such as CUDA Cores and Tensor Cores, best thought of as pipes that transform data.

GPU cores are the primary compute units that make up the Streaming Multiprocessors (SMs). Examples include CUDA Cores and Tensor Cores.

The internal architecture of an H100 GPU's Streaming Multiprocessors. CUDA and Tensor Cores are shown in green. Modified from NVIDIA's H100 white paper.
The internal architecture of an H100 GPU's Streaming Multiprocessors. CUDA and Tensor Cores are shown in green. Modified from NVIDIA's H100 white paper.

Not quite CPU cores#

GPU cores resemble CPU cores in that they are the component that actually performs computation. But the analogy can be quite misleading.

Key idea Think of GPU cores as "pipes": data goes in and transformed data comes out.

This is the viewpoint of the quantitative computer architect. From the hardware's perspective, each pipe is associated with specific instructions. From the programmer's perspective, each offers a particular kind of throughput, e.g. floating point matrix multiplication throughput for the Tensor Cores.

The SM is the closer equivalent of a CPU core. It has register memory to store information, cores to transform it, and an instruction scheduler to specify and command those transformations.