GPU Glossary
/
Performance·3 min read

Occupancy

The ratio of active warps on a device to the maximum number of warps that could be active.

Occupancy is the number of active warps divided by the maximum number of active warps the device supports. It tells you how much of the GPU's capacity for concurrent work is in use, which matters mainly because it affects how well the hardware can hide latency.

Each of four clock cycles has four warp slots, so there are 16 = 4 × 4 slots in total. Active warps fill 15 of them, for an occupancy of about 94%. Diagram inspired by the CUDA Techniques to Maximize Compute and Instruction Throughput talk at GTC 2025.
Each of four clock cycles has four warp slots, so there are 16 = 4 × 4 slots in total. Active warps fill 15 of them, for an occupancy of about 94%. Diagram inspired by the CUDA Techniques to Maximize Compute and Instruction Throughput talk at GTC 2025.

Theoretical vs achieved occupancy#

  • Theoretical occupancy is the upper limit set by the kernel launch configuration and the device's capabilities.
  • Achieved occupancy is the occupancy actually measured while the kernel runs, i.e. on active cycles.

What limits occupancy#

In the CUDA programming model, all the threads of a thread block are scheduled onto the same Streaming Multiprocessor (SM). Each SM has finite resources, such as shared memory space, that must be split among the thread blocks running on it. Those resources cap how many blocks fit on one SM.

Worked example: H100#

An NVIDIA H100 GPU has these per-SM limits:

Maximum warps/SM: 64
Maximum blocks/SM: 32
(32 bit) Registers: 65536
Shared memory (smem): 228 KB

Take a kernel with 32 threads per block, 8 registers per thread, and 12 KB of shared memory per block. Checking each limit:

64 > 1   = warps/block = 32 threads/block ÷ 32 threads/warp
32 < 256 = blocks/register-file = 65,536 registers/register-file ÷ (32 threads/block × 8 registers/thread)
32       = blocks/SM
19       = blocks/smem = 228 KB/smem ÷ 12 KB/block

The register file could hold 256 blocks at once, but shared memory only fits 19. So the SM runs 19 thread blocks, which here means 19 warps.

This is the common case: the intermediates a program keeps in registers are much smaller than the parts of its working set that must stay in shared memory.

Higher isn't always better#

Low occupancy hurts when there aren't enough eligible warps to hide the latency of instructions. That shows up as low issue efficiency and under-utilized pipes.

Once occupancy is high enough to hide latency, though, pushing it further can make things worse. More concurrent work means fewer resources per thread, which can bottleneck the kernel on registers or lower the arithmetic intensity that modern GPUs are built to exploit.

Key idea Occupancy is not usually the thing to optimize. Aim to maximize utilization of compute resources if you're compute-bound, or of memory resources if you're memory-bound.

For example, high-performance GEMM kernels on Hopper and Blackwell architecture GPUs often run at single-digit occupancy percentages. They don't need many warps to saturate the Tensor Cores.