Device Software
The programming model, instruction sets and memory spaces of code that runs on the GPU.
This section covers software that runs on the GPU (the "device"). It includes the abstractions you program against, like threads, warps, thread blocks and kernels, the memory spaces those threads can see, and the instruction sets the compiler targets (PTX and SASS).
Start with the CUDA programming model. It ties the other terms together.
- CUDA Programming ModelNVIDIA's model for writing massively parallel programs, built on a hierarchy of thread groups, a hierarchy of memories and barrier synchronization.
- Streaming Assembler SASSThe assembly language for NVIDIA GPUs, the lowest-level human-readable form of GPU code, tied to a specific SM architecture.
- Parallel Thread Execution PTXA forward-compatible intermediate representation and virtual instruction set for NVIDIA GPUs, JIT-compiled by the driver into device-specific SASS.
- Compute CapabilityNVIDIA's versioning system that ties PTX instruction sets to the physical GPUs able to run them.
- ThreadThe smallest unit of GPU programming, the atom of the CUDA thread hierarchy, with its own registers but little else.
- WarpA group of 32 threads that are scheduled together onto one Streaming Multiprocessor and execute the same instruction in parallel.
- WarpgroupA set of four contiguous warps (128 threads), aligned so the first warp's rank is a multiple of 4, used for warpgroup-level instructions.
- Cooperative Thread Array CTAA group of threads scheduled onto the same Streaming Multiprocessor; the PTX/SASS implementation of a CUDA thread block.
- KernelThe unit of CUDA code programmers write, launched once but executed in parallel by many threads organized as a thread block grid.
- Thread BlockThe middle level of the CUDA thread hierarchy, a group of threads that run on one SM and can coordinate with each other.
- Thread Block GridThe collection of thread blocks created by a kernel launch, the top level of the CUDA thread hierarchy, spanning the whole GPU.
- Thread HierarchyThe CUDA programming model's organization of parallel execution into threads, thread blocks and thread block grids.
- Memory HierarchyThe CUDA programming model's levels of memory, each shared by the threads at one level of the thread hierarchy: registers, shared memory and global memory.
- RegistersThe lowest level of the CUDA memory hierarchy, holding values manipulated by a single thread, stored in the SM's register file.
- Shared MemoryThe memory shared by the threads of a thread block, much smaller but much faster than global memory, stored in the SM's L1 data cache.
- Global MemoryThe top level of the CUDA memory hierarchy, accessible to every thread in a grid for the life of the program, stored in GPU RAM.
- CUDA Tile Programming ModelA tile-based programming model for NVIDIA GPUs in which tile-kernels run across a grid of single-threaded tile blocks operating on structured pointers.