Making Real File Systems Faster with Applied Computational Storage

The exploration of computation near flash storage has been prompted by the advent of network-attached flash-based storage enclosures operating at tens of gigabytes/sec, server memory bandwidths struggling to keep up with network and aggregate I/O bandwidths, and the ever-growing need for massive data storage, management, manipulation and analysis. Multiple tasks from distributed analytical/indexing functions to data management tasks like compression, erasure encoding, and deduplication are all potentially more performant, efficient and economical when performed near storage devices. The emerging NVMe Computational Storage standard requires real-world computational storage offload demonstrations to ensure the standard evolves in a useful direction that enables appropriate task offloads for end-user sites. Demonstrating the enablement of a standard’s-based ecosystem for offloading computation to near data storage is a valued contribution to the computing, networking, and storage communities. The goal of the joint Accelerated Box of Flash (ABOF) project (a collaboration between Eideticom, Nvidia, Aeon, SK hynix, and LANL) was to produce a first version of a network-attached computational storage system that allows a host application to directly leverage distributed and programmable computational elements in the ABOF without hiding any of the computation behind a block storage interface. Applications of interest include user-based computation, such as analytics acceleration for popular data formats, and kernel-based computation including functions common to file systems. Both use cases leverage distributed computational offloads near storage. The team has chosen to initially accelerate a commonly deployed kernel-based file system, ZFS, to appeal to the large ZFS community while also making it easy for vendors to deploy accelerated ZFS appliances and creating interesting business opportunities. The technical details of how this solution works, the useful artifacts produced, and the lessons learned from developing and testing this ABOF will be presented by the partners. Some background knowledge on NVMe, computational storage, and disaggregated storage would be beneficial to the audience.