nvidia-smi
NVIDIA's command line tool for querying and managing GPU state, built on the NVML library.
nvidia-smi is a command line tool for querying and managing GPU state, as exposed by the NVML management library. Its output (a sample is below) is so familiar to NVIDIA GPU users that it has become a meme.
What it reports#
- Identity: the card's model name, a UUID and the PCI ID.
- Live utilization: kernel execution time and memory allocation.
- Live power and thermals.
For how to interpret these metrics, especially power and thermal readings, see this page in the Modal docs.
What it manages#
nvidia-smi can list processes currently using the GPU (-q, --query, pmon). Common management tasks:
| Task | Flag |
|---|---|
| Persistence mode | -pm |
| Compute mode | -c |
| Power limit | -pl |
| Application / locked clocks | -ac, -lgc, -lmc |
| GPU reset | -r |
Output formats and stability#
Output can be human-readable text or XML (-x). The text format is not guaranteed to stay stable, so tools should use the underlying NVML C library, which offers a stable API.
Key idea Don't parse
nvidia-smi's text output in tools; call NVML instead.
The documentation is here, and the official Python bindings are here.
Sample output#
Eight B200 GPUs, idle:
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA B200 On | 00000000:51:00.0 Off | 0 |
| N/A 27C P0 136W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA B200 On | 00000000:52:00.0 Off | 0 |
| N/A 25C P0 140W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA B200 On | 00000000:62:00.0 Off | 0 |
| N/A 27C P0 138W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA B200 On | 00000000:63:00.0 Off | 0 |
| N/A 26C P0 138W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 4 NVIDIA B200 On | 00000000:75:00.0 Off | 0 |
| N/A 27C P0 139W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 5 NVIDIA B200 On | 00000000:76:00.0 Off | 0 |
| N/A 25C P0 140W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 6 NVIDIA B200 On | 00000000:86:00.0 Off | 0 |
| N/A 27C P0 142W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 7 NVIDIA B200 On | 00000000:87:00.0 Off | 0 |
| N/A 26C P0 138W / 1000W | 0MiB / 183359MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
Related#
- NVML — the library behind every
nvidia-smireading - libnvml.so — NVML's shared library on Linux
- NVIDIA GPU Drivers — the driver version shown in the header