Software Defined Memory with CXL and Tiered Memory to Enable Hyperscale Use Cases

There is a demand for Hierarchical memory as use cases demand higher amounts of memory capacity and bandwidth as AI/ML, database applications scale up and scale out. Software-Defined Memory (SDM) is an emerging HW-SW co-design architecture paradigm that provides software abstraction between applications and underlying memory resources with dynamic memory provisioning to achieve the desired SLA. With emergence of newer memory technologies and faster interconnects, it is possible to optimize memory resources deployed in cloud infrastructure while achieving best possible TCO. SDM provides a mechanism that also allows to pool disjoint memory domains into a unified memory namespace. This talk will cover SDM Architecture, current industry use cases that drive the need for SDM, academic research and leading use cases (e.g. Memcached, databases etc.) that can benefit from SDM design. This talk will cover how applications (AI/ML, Databases, Cache, Virtualized Servers etc.) can consume different tiers of memory (e.g., DDR, SCM, HBM) and interconnect technologies (e.g. CXL) that are foundational to SDM framework to provide load-store access for large scale application deployments. Ongoing work in software community that enables SDM in Kernel as well as in applications will be covered. SDM value prop will be demonstrated with caching benchmarks and tiering to show how memory can be accessed transparently.