Warp Execution State

The warps running a kernel are described by four non-exclusive states: active, eligible, stalled, and selected. Reading profiler output and reasoning about latency hiding both depend on these terms.

State	Meaning
Active	Has started executing and hasn't fully exited the kernel
Eligible	Active and ready to issue its next instruction
Stalled	Active but unable to issue its next instruction
Selected	Eligible and chosen by the scheduler to issue this cycle

Active#

A warp is active from the time its threads begin executing until all of them have exited the kernel. Active warps form the pool from which warp schedulers pick candidates for instruction issue each cycle (i.e. to fill an issue slot).

The maximum number of active warps per Streaming Multiprocessor (SM) varies by architecture and is listed in NVIDIA's documentation for each Compute Capability. An H100 SXM GPU (Compute Capability 9.0), for instance, supports up to 64 active warps per SM (2048 threads).

Active warps aren't necessarily executing instructions. In the diagram, all but one slot+cycle has an active warp: high occupancy.

Eligible#

An eligible warp is an active warp that is ready to issue its next instruction. That requires:

• the next instruction has been fetched,
• the required execution unit is available,
• all instruction dependencies are resolved, and
• no synchronization barrier blocks execution.

Eligible warps are the warp scheduler's immediate candidates for issue. In the diagram they appear on every cycle except n + 2. Having no eligible warps on many cycles can hurt performance, especially if you mainly use lower-latency arithmetic units like CUDA Cores.

Stalled#

A stalled warp is an active warp that can't issue its next instruction because of unresolved dependencies or resource conflicts. Common causes:

• Execution dependencies: waiting on results from earlier arithmetic instructions.
• Memory dependencies: waiting on results from earlier memory operations.
• Pipeline conflicts: the needed execution resources are busy.

A warp waiting on a variable-latency instruction that stays inside the SM is stalled on the "short scoreboard"; one waiting on a variable-latency instruction that leaves the SM is stalled on the "long scoreboard". Both are scoreboard stalls.

In the diagram, stalled warps occupy several slots on every cycle.

Key idea Stalled warps aren't inherently bad. Many concurrently stalled warps may be exactly what's needed to hide the latency of long-running instructions, like memory loads or Tensor Core instructions such as HMMA, which can run for dozens of cycles.

Selected#

A selected warp is an eligible warp that the warp scheduler picks to receive an instruction this cycle. Each cycle, every warp scheduler looks at its pool of eligible warps, selects one if there are any, and issues it an instruction. So any cycle with an eligible warp has a selected warp.

The fraction of active cycles on which a warp is selected and an instruction is issued is the issue efficiency.

Related#

• Latency hiding: why many stalled warps can be fine
• Occupancy: how many warps are active
• Scoreboard stall: a common reason warps stall
• Issue efficiency: how often a warp is selected