Pipe Utilization
How close each execution pipe inside an SM gets to its peak rate while it is actively running warps.
Pipe utilization measures how effectively a kernel uses the execution resources inside each Streaming Multiprocessor (SM). It is finer-grained than kernel or SM utilization.
What it measures#
Each SM has several independent execution pipes, each specialized for a type of instruction:
- CUDA Cores for general floating-point arithmetic
- Tensor Cores for tensor contractions
- Load/store units for memory access
- Control flow units for branching
Pipe utilization is the percentage of a pipe's peak rate achieved while that pipe is executing at least one warp, averaged across all active SMs.
Key idea Check the coarser metrics first. Before debugging at the level of pipes, look at GPU kernel utilization and SM utilization.
How to measure it#
Nsight Compute (ncu) reports pipe utilization in the sm__inst_executed_pipe_*.avg.pct_of_peak_sustained_active metrics. The asterisk stands for a specific pipe, such as fma (CUDA Cores), tensor (Tensor Cores), lsu (load/store units) or adu (address).
Related#
- SM utilization — the coarser metric to check first.
- Peak rate — the denominator of pipe utilization.
- Issue efficiency — idle schedulers lead to under-used pipes.