What the NVIDIA AI Stack Is: Container Toolkit, GPU Operator, Triton, and How They Fit Together
For executives
NVIDIA's AI software stack is a collection of interdependent components that together enable AI workloads to run at scale in containerised, Kubernetes-managed environments. Each component has its own CVE history. Each component's vulnerabilities are exploitable through the component's specific role in the stack. Understanding how the components fit together reveals why the attack surface spans so many different layers and why patching requires tracking multiple component versions simultaneously.
The stack from bottom to top
GPU hardware: the physical H100, A100, or other NVIDIA data centre GPU. The hardware itself is not directly configurable by software security tools, but its firmware (BIOS/SBIOS as covered in the DGX A100 article) can be a vulnerability surface.
GPU driver (nvidia.ko): the kernel module that connects the hardware to the software stack. Every operation above this layer routes through it. Driver CVEs provide privilege escalation paths. The driver is patched via display driver updates.
NVIDIA Container Toolkit: the software that makes GPU hardware accessible inside containers. Manages OCI hooks, device file bind-mounts, and CUDA library injection. The most active vulnerability surface in the stack — five security bulletins since September 2024. Patched independently from the GPU driver.
NVIDIA Kubernetes Device Plugin (k8s-device-plugin): manages GPU resource allocation within Kubernetes. Exposed by CVE-2024-0135 (Device Plugin API bypass). Carries its own OS package CVE history. Deployed as a DaemonSet, updated independently.
NVIDIA GPU Operator: Kubernetes operator that manages deployment of all of the above components. Controls which versions of the Container Toolkit, device plugin, and driver are installed on GPU nodes. Patching the Operator is required for Container Toolkit patches to propagate correctly.
NVIDIA Triton Inference Server: the inference serving layer. Manages deployment of AI models and handles inference requests. Has its own CVE series — four bulletins in 2024-2025 covering the full range from file write vulnerabilities to unauthenticated RCE chains. Deployed separately from the Kubernetes GPU infrastructure stack.
CUDA Toolkit: the developer toolchain. Includes CUDA runtime libraries, nvdisasm, cuobjdump, and other tools. Carries its own CVE series targeting developer environments that process compiled CUDA binaries.
AI frameworks: PyTorch, TensorFlow, JAX, and similar. Run on top of CUDA. Their CUDA kernels are what actually executes on the GPU.
Why the stack creates a multi-component patching challenge
Each component has its own:
Version number tracked separately from all others. CVE feed published on its own bulletin schedule. Patching mechanism (driver updates vs container image updates vs Helm chart updates). Deployment dependency on other components (the GPU Operator must be updated to propagate Container Toolkit patches).
A security team that patches the GPU driver and the Container Toolkit but misses a GPU Operator update may find that the Operator reconciles the nodes back to the unpatched Container Toolkit version.
A security team that patches all Kubernetes infrastructure components but runs an outdated Triton version is running a CVSS 9.8 unauthenticated RCE vulnerability in their inference serving layer.
Effective security coverage of the NVIDIA AI stack requires tracking the vulnerability and patch status of: GPU driver, Container Toolkit, GPU Operator, k8s-device-plugin, DCGM, Triton (if deployed), CUDA Toolkit (in development environments), and any AI framework components with CUDA extensions.