GPU RAM
The GPU's large, bottom-level DRAM memory, addressable by all Streaming Multiprocessors and used to implement global memory.
GPU RAM is the bottom level of the GPU's memory: a large store, from many megabytes to gigabytes, that all of the GPU's Streaming Multiprocessors (SMs) can address. It is also called video RAM (VRAM). An H100 can store 80 GiB (687,194,767,360 bits) in its RAM.
DRAM, not SRAM#
GPU RAM uses Dynamic RAM (DRAM) cells. These are slower but smaller than the Static RAM (SRAM) cells used in registers and cache memory. For details on both, see Ulrich Drepper's 2007 article "What Every Programmer Should Know About Memory".
Physical placement#
GPU RAM is generally not on the same die as the SMs. In the latest data center GPUs like the H100, it sits on a shared interposer with them, which lowers latency and increases bandwidth.
These GPUs use High-Bandwidth Memory (HBM), rather than the more familiar Double Data Rate (DDR) memory found in consumer GPUs and CPUs.
What it's used for#
- It implements the global memory of the CUDA programming model.
- It stores register data that spills from the register file.
Related#
- Global Memory: the programming-model view of GPU RAM
- Memory Bandwidth: the rate data moves to and from it
- Memory Hierarchy: where GPU RAM sits relative to other memories