CUDA Tile Programming Model

The CUDA Tile programming model is a tile-based programming model for NVIDIA GPUs. It is an alternative to the traditional thread-based "CUDA SIMT" model, aimed at GPUs whose compute is dominated by Tensor Cores.

The traditional SIMT model#

The traditional CUDA programming model exposes a hierarchy of threads and a hierarchy of memories. Programs receive pointers and execute concurrently, mutating memory relative to those pointers. The same instructions are issued to many threads in parallel, so it is a "single-instruction, multiple thread" (SIMT) model. It is the model used by, for example, CUDA C/C++ and the PTX IR used by pre-CUDA-Tile programs targeting NVIDIA GPUs.

That model was defined for a "unified" hardware substrate, the "U" in "CUDA": homogeneous Streaming Multiprocessors (SMs) with homogeneous CUDA Cores implement most operations. Before CUDA, graphics programming generally used devices made of specialized cores, programmed heterogeneously.

Why a new model#

SIMT is a poor fit for GPUs of the latest SM architectures, where the vast majority of arithmetic bandwidth is in the Tensor Cores. Tensor Cores can only perform matrix multiplications, and they must be programmed with thread-level instructions and asynchrony, rather than the warp-level asynchrony used for the rest of the hardware.

How CUDA Tile works#

In CUDA Tile, programs are written as tile-kernels: instances of the program that run concurrently across a grid of tile blocks, each of which is a single thread of execution.

In the happy path, tile-kernels operate on structured pointers. A structured pointer combines a pointer with information about an array: its total extent (shape) and its access pattern (stride). This is similar to the CuTe type system for Layouts and Tensors.

Key idea Instead of programming many threads that each touch a few elements, you program single-threaded tile blocks that operate on whole shaped arrays.

As with CUDA SIMT in CUDA C/C++ and PTX, the model is shared between high-level languages and an intermediate representation, here Tile IR.

Availability#

As of mid-2026, the CUDA Tile programming model is new, and how far it will replace the existing CUDA SIMT model is still unclear. It is available through:

• cuTile Python
• cuTile BASIC (experimental)
• cuTile Rust (experimental)

Related#

• CUDA Programming Model: the SIMT model CUDA Tile contrasts with
• cuTile BASIC: one language implementing the model
• CuTe: a similar type system for layouts and tensors
• Tensor Core: the hardware motivating the model