GPU Glossary
/
Device Software·2 min read

Cooperative Thread ArrayCTA

A group of threads scheduled onto the same Streaming Multiprocessor; the PTX/SASS implementation of a CUDA thread block.

A cooperative thread array (CTA) is a collection of threads scheduled onto the same Streaming Multiprocessor (SM). CTAs are how PTX and SASS implement the thread blocks of the CUDA programming model. Each CTA is made of one or more warps.

Cooperative thread arrays correspond to the thread block level of the thread hierarchy in the CUDA programming model. Modified from diagrams in NVIDIA's CUDA Refresher: The CUDA Programming Model and the NVIDIA CUDA C++ Programming Guide.
Cooperative thread arrays correspond to the thread block level of the thread hierarchy in the CUDA programming model. Modified from diagrams in NVIDIA's CUDA Refresher: The CUDA Programming Model and the NVIDIA CUDA C++ Programming Guide.

Coordination within and across CTAs#

Programmers can have threads within a CTA coordinate with each other. Programmer-managed shared memory, which lives in the SM's L1 data cache, makes this coordination fast.

Threads in different CTAs cannot synchronize with barriers. They must coordinate through global memory, for example with atomic update instructions. Because the driver controls CTA scheduling at runtime, CTA execution order is indeterminate, and making one CTA wait on another can easily deadlock.

Key idea Coordinate freely inside a CTA; never assume anything about the order of different CTAs.

CTAs and occupancy#

The number of CTAs that fit on one SM sets the achievable occupancy, and it depends on several factors. The SM has limited resources:

  • lines in the register file
  • "slots" for warps
  • bytes of shared memory in the L1 data cache

Each CTA uses some amount of each resource (calculated at compile time) when it is scheduled onto an SM.