GPU Glossary
/
Device Software·2 min read

Thread

The smallest unit of GPU programming, the atom of the CUDA thread hierarchy, with its own registers but little else.

A thread of execution ("thread" for short) is the lowest unit of programming on a GPU: the base and atom of the CUDA programming model's thread hierarchy. A thread has its own registers, but little else.

Threads are the lowest level of the thread group hierarchy (top left) and map onto the cores of a Streaming Multiprocessor. Modified from diagrams in NVIDIA's CUDA Refresher: The CUDA Programming Model and the NVIDIA CUDA C++ Programming Guide.
Threads are the lowest level of the thread group hierarchy (top left) and map onto the cores of a Streaming Multiprocessor. Modified from diagrams in NVIDIA's CUDA Refresher: The CUDA Programming Model and the NVIDIA CUDA C++ Programming Guide.

Compared with CPU threads#

Both SASS and PTX programs target threads. A typical C program in a POSIX environment, by contrast, targets a process, which is a collection of one or more threads. Unlike POSIX threads, CUDA threads are not used to make syscalls.

  • Instruction pointer. Like a CPU thread, a GPU thread can have its own instruction pointer (program counter). For performance, though, GPU programs are usually written so that all threads in a warp share the same instruction pointer and execute in lock-step (see also Warp Scheduler).
  • Stack. GPU threads also have stacks in global memory, used for spilled registers and the function call stack. High-performance kernels generally keep use of both to a minimum.

Key idea A GPU thread looks like a CPU thread, but it is meant to run in lock-step with its warp, not independently.

Hardware mapping#

A single CUDA Core executes instructions from a single thread. Threads map onto the cores of a Streaming Multiprocessor.

  • Warp: the group of 32 threads that executes together
  • Thread Block: the next level up in the hierarchy
  • Registers: the memory private to each thread