Registers
The lowest level of the CUDA memory hierarchy, holding values manipulated by a single thread, stored in the SM's register file.
Registers are the lowest level of the memory hierarchy. They hold the values a single thread works on.
Where registers live#
Register values are generally stored in the register file of the Streaming Multiprocessor (SM). They can also spill to global memory in GPU RAM, at a substantial performance penalty.
Who manages them#
As on CPUs, high-level languages like CUDA C don't manipulate registers directly. They are visible only to a lower-level language, here Parallel Thread Execution (PTX), and are typically managed by a compiler such as ptxas.
One of the compiler's goals is to limit the registers each thread uses, so more thread blocks can be scheduled onto one SM at the same time, increasing occupancy.
Key idea Fewer registers per thread means more threads fit on an SM, but too few forces costly spills to global memory.
Documentation#
PTX registers are documented here. The registers used in SASS are not, to our knowledge, documented.
Related#
- Register File: the hardware that stores registers
- Register Pressure: what happens when kernels need too many registers
- Occupancy: how register use limits concurrency