Kernel bypass eliminates OS network stack overhead by moving packet processing directly to user space. When paired with GPUDirect RDMA, network packets travel directly from the Network Interface Card (NIC) to GPU memory over PCIe.

This zero-copy path bypasses the host CPU and system RAM entirely, yielding the deterministic, ultra-low latency required for distributed AI training and high-frequency data ingestion.

This guide provides the exact deployment path for Ubuntu 22.04 LTS, NVIDIA datacenter GPUs (A100/H100), and ConnectX NICs on GPUYard bare-metal infrastructure.

1. Hardware & Topology Prerequisites

Do not assume optimal hardware routing. Peer-to-peer PCIe transfers require precise alignment.

GPU & NIC: NVIDIA A100/H100 GPU and a ConnectX-5 (or newer) NIC.
NUMA Affinity: The NIC and GPU must reside on the same NUMA node.
PCIe Topology: Ideally, both devices sit behind the same PCIe switch (PIX or PXB status).
Out-of-Band Access: You need a secondary management interface. Misconfiguring VFIO or DPDK bindings on your primary interface will sever your SSH session.

Verify your topology before configuring software:

bash

lspci | grep -Ei 'mellanox|nvidia'
nvidia-smi topo -m

2. Host Operating System Preparation

Configure the Linux kernel to support isolated CPU cores, IOMMU passthrough, and hugepages.

Edit /etc/default/grub to reserve 1GB hugepages at boot and isolate cores for DPDK polling:

bash

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G hugepages=16 isolcpus=2-15 nohz_full=2-15 rcu_nocbs=2-15"

Apply the changes and reboot:

bash

sudo update-grub && sudo reboot

Install the NVIDIA datacenter driver, CUDA toolkit, and MLNX_OFED (or DOCA-OFED). Ensure the nvidia-peermem module loads successfully. This module is strictly required for GPUDirect RDMA.

bash

sudo modprobe nvidia-peermem
lsmod | grep nvidia_peermem

3. Building DPDK for GPUDirect

Distribution package managers strip the CUDA dependencies from DPDK. You must build DPDK from source to enable the gpudev library.

Install build dependencies:

bash

sudo apt install -y build-essential meson ninja-build libnuma-dev python3-pyelftools

Compile DPDK, explicitly enabling the MLX5 network driver and the CUDA GPU driver:

bash

export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

tar xf dpdk-<lts_version>.tar.xz && cd dpdk-<lts_version>
meson setup build -Denable_drivers=net/mlx5,common/mlx5,gpu/cuda -Denable_apps=testpmd,test-gpudev
ninja -C build
sudo ninja -C build install
sudo ldconfig

The ConnectX Driver Exception

Unlike standard Intel NICs, NVIDIA ConnectX adapters use a bifurcated driver model. Do not bind a ConnectX NIC to vfio-pci or uio_pci_generic. The DPDK mlx5 Poll Mode Driver (PMD) runs on top of the in-kernel mlx5_core and RDMA verbs. Leave the NIC bound to its default kernel driver.

4. Bridging NIC and GPU via DPDK API

To land packets directly into device memory, you must configure a DPDK memory pool (mempool) backed by the GPU.

The standard C API sequence requires five steps:

Allocate GPU Memory: Use rte_gpu_mem_alloc() to allocate GPUDirect-capable device memory.
Register External Memory: Map the allocation to DPDK using rte_extmem_register().
DMA Map the Region: Map the GPU memory for the NIC via rte_dev_dma_map().
Create the Mempool: Call rte_pktmbuf_pool_create_extbuf() pointing to the mapped GPU region.
Attach to RX Queue: Pass this specialized mempool into rte_eth_rx_queue_setup().

By allocating via rte_gpu_mem_alloc, the DPDK CUDA driver automatically sets the CU_POINTER_ATTRIBUTE_SYNC_MEMOPS flag required for stable peer-to-peer PCIe writes.

5. Validation and Benchmarking

Isolate the hardware capability before testing your software stack. Use the CUDA-enabled perftest suite to confirm raw NIC-to-GPU DMA works:

bash

ib_write_bw -d mlx5_0 --use_cuda=0 -F --report_gbits

Once raw DMA is validated, use DPDK's testpmd application to verify kernel bypass and GPU memory allocation. The exact flags vary by DPDK release, but require allowing both the NIC and GPU PCI addresses:

bash

sudo dpdk-testpmd -l 2-5 -n 4 -a 0000:3b:00.0 -a 0000:3d:00.0 -- \
  -i --nb-cores=2 --forward-mode=rxonly --mp-alloc=gpu --gpu-id=0

At the testpmd> prompt, type start. Watch show port stats all. The RX counters will increment here, while the host OS ip -s link counters remain static for steered flows.

Latency Measurement Rules

Rely on hardware timestamping (e.g., TRex or pktgen-dpdk) from an external traffic generator. Software timestamps introduce unacceptable jitter at this scale. Report the 99th and 99.9th percentiles (p99/p99.9). Poll-mode architectures primarily optimize the tail latency distribution, making median averages an incomplete metric.

6. Infrastructure Requirements

Low-latency architectures fail in virtualized environments. Hypervisor interference, shared PCIe lanes, and restricted IOMMU controls introduce microsecond-level jitter. Achieving these metrics requires dedicated bare-metal access to the PCIe root complex and NUMA topology. GPUYard provides the unabstracted hardware control necessary to deploy and tune kernel bypass workloads effectively.

Frequently Asked Questions

What is kernel bypass?

Kernel bypass is a networking technique where applications process packets directly from the Network Interface Card (NIC) in user space, completely skipping the Operating System's kernel network stack to eliminate CPU interrupts, context switches, and latency.

Why shouldn't I bind my ConnectX NIC to vfio-pci?

NVIDIA ConnectX NICs use a bifurcated driver model. Unlike standard Intel NICs, the DPDK mlx5 Poll Mode Driver requires the in-kernel mlx5_core driver and RDMA verbs to function. Binding it to vfio-pci will break the driver functionality.

Why are bare-metal servers required instead of VPS or cloud instances?

Virtual environments use hypervisors that abstract hardware, leading to shared PCIe lanes, restricted IOMMU controls, and CPU scheduling interrupts. To achieve deterministic, microsecond-level zero-copy transfers, you need the direct, unabstracted hardware control provided by bare-metal servers.