79 terms · 4 layers of the stack

GPUs, explained from the silicon up.

A plain-English reference for the hardware, software and performance ideas behind NVIDIA GPUs. Every page links to the terms it uses, so you can read it front to back or jump around.

Start reading

Good places to start

Streaming MultiprocessorWhat a GPU is actually made of CUDA Programming ModelHow you program thousands of threads Roofline ModelIs your kernel compute- or memory-bound?Tensor CoreWhere the deep-learning FLOPs come from

Browse by layer

Device Hardware

The physical chip. 16 terms.

CUDA Device Architecture SM GPU Core SFU LSU Warp Scheduler CUDA Core Tensor Core TMA All 16 →

Device Software

Code that runs on the GPU. 17 terms.

CUDA Programming Model SASS PTX Compute Capability Thread Warp Warpgroup CTA Kernel All 17 →

Host Software

CPU-side drivers, tools and libraries. 23 terms.

CUDA Software Platform CUDA C++NVIDIA GPU Drivers nvidia.ko CUDA Driver API libcuda.so NVML libnvml.so nvidia-smi All 23 →

Performance

Why kernels are fast or slow. 23 terms.

Performance Bottleneck Roofline Model Compute-bound Memory-bound Arithmetic Intensity Overhead Little's Law Memory Bandwidth Arithmetic Bandwidth All 23 →