Container Runtime Process Injection: What Syscall Monitoring Misses Below the GPU Boundary
For executives
Most container runtime security tools work by monitoring system calls made by container processes. GPU containers have a problem: the most interesting events in a GPU container's lifecycle do not happen in the container's process. They happen in the NVIDIA Container Toolkit hooks that run before the container process exists, in the kernel-mode driver that handles GPU operations below the syscall layer, and in the CUDA runtime that manages GPU execution. This article covers what falls below the visibility horizon of conventional runtime security in GPU environments.
Where conventional runtime security operates
Container runtime security tools observe behaviour at the syscall boundary. A process makes a system call: open a file, connect to a network, create a process. The monitoring tool intercepts that syscall, evaluates it against policy, and allows or denies it.
Three distinct execution contexts in GPU containers fall outside this model.
Gap one: OCI hook execution before container startup
OCI hooks execute before the container's process namespace is established. The nvidia-ctk binary runs as a host process, not a container process. Its syscalls appear as host-level activity, not as container activity. Container runtime security policies that apply to container processes do not apply to hooks.
This is why the CVE-2025-23266 exploit is invisible to container-level monitoring: the malicious library loaded via LD_PRELOAD executes in the hook context. The monitoring tool has no container to monitor yet.
Gap two: kernel module operations below the syscall layer
CUDA operations translate to ioctl calls against /dev/nvidia* device files. The ioctl() syscall is observable at the syscall boundary. However, the semantics of what that ioctl does are not visible at the syscall layer. An ioctl to /dev/nvidia0 looks identical whether it is a legitimate matrix multiplication or a malicious memory-scanning kernel.
Standard syscall monitoring tools do not decode CUDA ioctl semantics.
Gap three: GPU kernel execution on the device
Once a CUDA kernel is dispatched to the GPU hardware, it executes entirely on the GPU device. The host CPU and the host OS have no visibility into GPU kernel execution. GPU utilisation monitoring reports aggregate utilisation percentages — it does not report what code is executing.
What fills the gaps
Gap 1 (OCI hooks): monitoring the nvidia-ctk process itself at the host level — what libraries it loads, what files it opens. Gap 2 (ioctl semantics): monitoring at the NVIDIA driver's internal boundaries — parsing and interpreting CUDA ioctl payloads. Gap 3 (GPU execution): monitoring via the NVIDIA driver's event interfaces — UVM events, CUDA context events, and GPU module load events.