What CUDA Is: The Programming Model Running Every AI Workload and Its Security Surface
For executives
CUDA is NVIDIA's parallel computing platform and programming model. Every PyTorch training run, every TensorFlow inference, every large language model generation runs on CUDA. CUDA is not just a library — it is a software layer that sits between AI applications and GPU hardware, managing execution, memory, and device access. Understanding CUDA is essential for understanding the security surface of AI infrastructure, because many of the vulnerabilities in this library exist at the CUDA boundary.
What CUDA does
CUDA allows developers to write code that runs on the GPU in parallel. A CUDA program consists of host code (running on the CPU) and device code (running on the GPU). Device code is organised into kernels — functions that the CPU launches onto the GPU and that execute across thousands of parallel threads simultaneously.
When a PyTorch model runs a matrix multiplication, it is calling a CUDA kernel: a compiled GPU function that performs the multiplication across GPU threads in parallel. The operation is invisible to the CPU while it executes.
The CUDA software stack
CUDA application libraries (cuBLAS, cuDNN, cuFFT): pre-built GPU-accelerated implementations of common operations. AI frameworks call these rather than writing raw CUDA kernels.
CUDA runtime: the C/C++ API that manages GPU memory, launches kernels, and handles synchronisation between CPU and GPU.
CUDA driver API: a lower-level API that provides finer-grained control over contexts, modules, and kernel loading.
NVIDIA GPU driver (nvidia.ko): the kernel module that the CUDA driver communicates with via ioctl calls.
The CUDA context as a security boundary
A CUDA context is the GPU-side analogue of a process. It is created when a CUDA application first calls any CUDA function. The context has its own GPU virtual address space, its own loaded modules, and its own resource allocation.
Multiple processes can have CUDA contexts on the same GPU. The GPU driver is supposed to isolate contexts from each other. This isolation is enforced by the driver, not by hardware in non-MIG, non-Confidential Computing configurations.
The CUDA execution model and why it matters for security
CUDA execution is asynchronous. The CPU launches a kernel on the GPU and continues executing CPU code while the GPU runs the kernel in parallel. This asynchrony means that GPU-side execution is not visible in the CPU-side process's system call trace. An EDR tool tracing system calls sees the kernel launch (a single ioctl call), but not the GPU execution that follows.
Why CUDA is an attractive attack surface
CUDA's position between applications and hardware makes it a high-value target. The CUDA runtime and driver are installed on every GPU server. CUDA's asynchronous execution model creates a gap in CPU-side monitoring. CUDA's memory management is the interface through which model weights, inference data, and user data are handled.