Host Software
The drivers, libraries, compilers and tools that run on the CPU to drive a GPU.
This section covers software that runs on the CPU (the "host") when you use a GPU. It includes the kernel driver, the CUDA Driver and Runtime APIs, the compiler toolchain, the monitoring and profiling tools, and the libraries most people call instead of writing kernels themselves (cuBLAS, cuDNN, CUTLASS).
Start with the CUDA software platform for an overview of how the pieces fit together.
- CUDA Software PlatformThe collection of software, including compilers, APIs and libraries, used to build and run CUDA programs.
- CUDA C++An extension of C++ that implements the CUDA programming model, adding syntax for defining and launching GPU kernels.
- NVIDIA GPU DriversThe software that mediates all interaction between host programs or the host operating system and an NVIDIA GPU.
- nvidia.koThe binary Linux kernel module at the core of the NVIDIA GPU drivers, which talks directly to the GPU.
- CUDA Driver APIThe low-level, closed-source userspace API of the NVIDIA CUDA drivers, used to manage GPU memory, kernels and devices.
- libcuda.soThe usual name of the shared library that implements the CUDA Driver API on Linux.
- NVIDIA Management Library NVMLA library for monitoring and managing the state of NVIDIA GPUs, such as power draw, temperature and memory use.
- libnvml.soThe usual name of the shared library that implements NVML on Linux.
- nvidia-smiNVIDIA's command line tool for querying and managing GPU state, built on the NVML library.
- CUDA Runtime APIA higher-level, closed-source API that wraps the CUDA Driver API and is the usual way applications use CUDA.
- libcudart.soThe usual name of the shared library that implements the CUDA Runtime API on Linux.
- CUDA GraphA graph of kernel launches and other GPU work that the host submits to the device in a single step.
- NVIDIA CUDA Compiler Driver nvccThe toolchain that compiles CUDA C/C++ programs into "fat binaries" containing host code plus PTX and/or SASS for the GPU.
- NVIDIA Runtime Compiler nvrtcA library that compiles CUDA C++ to PTX at runtime, inside your process, without launching nvcc.
- NVIDIA CUDA Profiling Tools Interface CUPTIA set of NVIDIA APIs for profiling CUDA C++, PTX and SASS code running on GPUs, with CPU–GPU timestamp synchronization.
- NVIDIA Nsight SystemsNVIDIA's GUI tool for performance debugging of CUDA C++ programs, combining profiling, tracing and expert-systems analysis.
- CUDA Binary UtilitiesA collection of NVIDIA tools, including cuobjdump and nvdisasm, for examining binaries produced by nvcc.
- cuBLASNVIDIA's proprietary, highly optimized GPU implementation of the BLAS standard for linear algebra operations like matrix multiplication.
- cuDNNNVIDIA's library of highly optimized GPU primitives for deep neural networks, such as convolution, attention and normalization.
- CUTLASSNVIDIA's open-source CUDA C++ template library for building high-performance linear algebra kernels, especially matrix multiplications.
- CuTeA header-only CUDA C++ template library inside CUTLASS for describing and manipulating tensors of data and threads through composable Layouts.
- CuTe DSLA Python domain-specific language in CUTLASS for writing high-performance GPU kernels that are JIT-compiled to PTX and SASS.
- cuTile BASICA real but toy implementation of the CUDA Tile programming model in BASIC, released by NVIDIA as an April Fools' joke.