GPU Glossary
/
Performance·3 min read

Scoreboard Stall

A stall where an instruction can't issue because it depends on the result of an earlier instruction that hasn't finished.

A scoreboard stall occurs when an instruction can't be issued because it depends on the result of a prior instruction.

A scoreboard is a hardware structure that tracks which registers are waiting to be written by an in-flight instruction. While a warp waits on it, the warp is in the stalled state and cannot progress.

Short vs long scoreboard stalls#

Type Waiting on Examples
Short A variable-latency instruction that stays inside the Streaming Multiprocessor (SM) Shared memory ops like LDS and STS; some Special Function Unit (SFU) ops
Long A memory operation that may leave the SM Global memory loads (LDG) and stores (STG)

Why SFU operations count as short is somewhat obscure, but it's related to their sharing the same Memory Input/Output hardware as shared memory accesses.

Key idea Long scoreboard stalls dominate memory-bound code.

Reading scoreboards in SASS#

Some scoreboard information is visible in Streaming Assembler (SASS). For example, each warp has six distinct barrier identifiers that the compiler uses to track data dependencies between instructions.

Here is what you might see from cuobjdump --dump-sass:

[barrier:  :  :  :  ]  /*line*/  INSTRUCTION Ri, [Rj] ; # Format: scoreboard info, line number, instruction, operands
[B------:R-:W2:-:S04]  /*00f0*/  LDG.E.SYS R0, [R2] ;   # Sets scoreboard 2
[B------:R-:W2:-:S01]  /*0100*/  LDG.E.SYS R5, [R4] ;   # `ptxas` intelligently reuses scoreboard 2
...
[B--2---:R-:W-:Y:S08]  /*0150*/  IMAD R0, R0, c[0x0][0x160], R5 ;  # Waits on scoreboard 2
  • Both LDG instructions increment scoreboard 2 when issued (W2, a write).
  • The IMAD has a barrier on scoreboard 2 (B--2---), so it can't issue until that flag clears. This guarantees R0 and R5 hold the loaded values before it runs.
  • When a data dependency is satisfied, its scoreboard is decremented.
  • A barrier can name several scoreboards: B01--4- means wait until scoreboards 0, 1 and 4 are all cleared.

Because scoreboards get reused, Nsight Compute's stall classification can be wrong: a long and a short scoreboard stall may be conflated if they share a scoreboard.

History#

Scoreboarding for dependency tracking in dynamic instruction scheduling dates back to the "first supercomputer", the Control Data Corporation 6600. One of these machines disproved Euler's sum of powers conjecture in 1966.

Unlike CPUs, GPUs don't use scoreboarding for out-of-order execution within a thread (instruction-level parallelism), only across threads (thread-level parallelism); see this NVIDIA patent.

For more on how GPUs implement scoreboards, see Professor Matthew D. Sinclair's slides.