Optimizing NVMe Over TCP Disaggregated Storage architecture on DPUs

A prominent trend in disaggregated storage is the use of Non-Volatile Memory Express over Fabric (NVME-oF) to connect to storage clusters. DPUs provide local NVMe controller to remote NVMe over Fabric storage offload, where the NVMe devices can appear as local PCIe controllers to the VMs and containers but can reach to namespaces provided over NVMe over Fabric target storage with the help of the DPU without the involvement of the host. This paradigm is getting standardized in AI/ML storage workloads as we see the newer transport protocols like Falcon/UEC adopting this method of storage disaggregation. As on-prem, hybrid and bare metal tenancy become more prevalent, most hyperscalers and datacenter operators are looking at DPUs to provide streamlined and fine-grained control to their infrastructure. DPUs are designed to be a host-independent infrastructure endpoint that not only exposes the device’s storage services to the host but also at the same time securely exposes the device’s capabilities and provisioning control to the broader infrastructure and orchestration mechanisms.
At SDC 2023 we delved into the details and capabilities of DPUs as a solution for disaggregated storage over fabric, specifically NVMe over TCP (https://www.youtube.com/watch?v=d5uqAR0Ny04&list=PLH_ag5Km-YUae2Ba7JNPqI...).
In this presentation we will focus on a specific challenge associated with using DPUs for disaggregated storage when using NVMe over TCP which uses DPU memory as bounce buffers for transmission and reception of NVMe over TCP traffic. Since the NVMe over TCP traffic runs on the DPU, there is an intermediate step of bouncing the data through the DPU DDR memory. This increases latency, causes DDR bandwidth issues on DPU and reduces the achievable performance overall. With recent linux networking stack advancements like dmabuf, TCPdevicemem techniques , these shortcomings can now be overcome.
This talk will explore the details of these techniques and optimizations.