The MadSystems Seminar aims to connect faculties and students interested in computer systems research.
We are interested in all aspects of computer systems, including Distributed and Cloud computing, Operating Systems, File and Storage systems, Formal methods (e.g. modeling, verification), Networked Systems and Machine Learning Systems.
The seminar takes two formats:
Time: Tuesday 4:00-5:00 pm Location: Room 6618, Morgridge Hall (or hybrid for remote invited talks) Calendar: Google Calendar
Invited Talk: EMT: An OS Framework for New Memory Translation Architectures Speaker: Siyuan Chai
Time: 4:00-5:00 pm, Feb 10, 2026 Location: Morgridge Hall 6618, and Online (Join via Zoom)
Abstract:
With terabyte-scale memory capacity and memory-intensive workloads, memory translation has become a major performance bottleneck. Many novel hardware schemes are developed to speed up memory translation, but few are experimented with commodity OSes. A main reason is that memory management in major OSes, like Linux, does not have the extensibility to empower emerging hardware schemes. We develop EMT, a pragmatic framework atop Linux to empower different hardware schemes of memory translation such as radix tree and hash table. EMT provides an architecture neutral interface that 1) supports diverse memory translation architectures, 2) enables hardware-specific optimizations, 3) accommodates modern hardware and OS complexity, and 4) has negligible overhead over hardwired implementations. We port Linux’s memory management onto EMT and show that EMT enables extensibility without sacrificing performance. We use EMT to implement OS support for ECPT and FPT, two recent experimental translation schemes for fast translation; EMT enables us to understand the OS perspective of these architectures and further optimize their designs.
Bio:
Siyuan Chai is a fifth-year PhD student in Computer Science at the University of Illinois Urbana-Champaign, advised by Prof. Tianyin Xu. His research broadly spans computer systems, with a particular focus on operating systems, AI systems, and memory systems. Prior to PhD, Siyuan earned his BS/MS degree from Northwestern University in Spring 2021. More details can be found at https://schai.me/.
We primarily use UW-Madison Systems Group Slack for communication, and in particular the #seminar channel. To join Systems Group Slack, click this invitation link.
#seminar
Meanwhile, we will also use a mailing list to broadcast announcements (e.g., each week's talk topic): sys-seminar@cs.wisc.edu. You can join the email list here.
sys-seminar@cs.wisc.edu
My research explores instance-customizable operating systems—systems that can be safely customized at runtime to hardware, workloads, and failure contexts without breaking compatibility. This talk covers my work on customizing OS I/O subsystem structure (uFS), Ananke, which enables OS-level customization of failure semantics, and KernelX, which supports safe, in-situ tuning of performance-critical constants in deployed kernels, achieving millisecond-scale updates and significant performance gains through a programmable tuning policy plane in user space. Together, these systems illustrate how principled OS abstractions can expose rich optimization opportunities that AI techniques can meaningfully manage.
Jing Liu is a Senior Researcher at Microsoft Research Asia. She received her Ph.D. from the University of Wisconsin–Madison in 2024. Her research includes datacenter-scale operating systems, file and storage systems, and distributed systems, with a focus on redesigning system mechanisms and interfaces so that machine learning–based techniques can meaningfully participate in system problems such as adaptation, control, and optimization. Jing is a recipient of the Erik Riedel Best Paper Award at FAST 2025 and a Meta Ph.D. Research Fellowship in Systems. She has served on the program committees of SOSP, OSDI, NSDI, and FAST, and is a Shadow PC Co-Chair for EuroSys 2026. More details can be found at https://jingliu.xyz.
Fast evolving machine learning workloads have increasing requirements for networking, ranging from simple AllReduce to more challenging AlltoAll. However, host networking hardware like RDMA NICs is evolving slowly, e.g., broken DCQCN congestion control that is challenging to tune, and no multipathing support yet. This mismatch has severely hindered performance scale-up and incurred additional costs.
We present UCCL, an extensible software transport layer for GPU networking. UCCL separates the data path and control path of existing RDMA NICs, and achieves efficient software control for the transport running on host CPUs. The extensibility of UCCL transport allows us to implement a multipath transport with packet spraying to resolve ECMP collisions, a receiver-driven transport to handle network incast, and a selective retransmission scheme to handle packet loss. UCCL provides a drop-in replacement for NCCL (and RCCL), and outperforms NCCL by 2.3-3.3x for GPU collectives over RoCE and AWS EFA RDMA.
Beyond collective communication, UCCL also extends to P2P communication (for PD disaggregation) and GPU-initiated communication (for expert parallelism, e.g., in DeepSeekV3) with superior performance. UCCL is fully open-sourced at https://github.com/uccl-project/uccl.
Yang Zhou is an Assistant Professor at UC Davis. He was a postdoc in UC Berkeley SkyLab, working with Ion Stoica. He got his Ph.D. from Harvard, advised by Minlan Yu and James Mickens. He has equal interests in core systems and ML systems research, including efficient LLMs, GPU communication, and heterogeneous computing. He is currently working on the UCCL project to build efficient GPU communication systems.
For decades, systems researchers have balanced competing objectives: latency versus throughput, performance versus power, and speed versus cost. Traditionally, accuracy was a fixed requirement—non-negotiable and external to the systems equation. But as modern computing increasingly hosts AI-driven workloads, accuracy itself has become a tunable system variable. In this talk, I’ll argue that the next frontier in systems design lies in treating accuracy as a first-class performance knob, to be traded for efficiency in a principled way. Drawing on recent work including DroidSpeak, Murakkab, Slim-SC, and Sherlock, I will show how software–hardware co-design can explicitly expose, quantify, and manage the accuracy-efficiency tradeoff.
Esha Choukse is a Principal Researcher at Microsoft in the Azure Research – Systems group. Her current research focus is on efficient AI in the cloud, spanning the layers of AI platforms, hardware, and datacenter design and provisioning. In the past, Esha has also worked on sustainability, memory systems, and compression. Esha has a PhD from University of Texas at Austin in computer architecture, and has published several papers at ISCA, ASPLOS, MICRO, HPCA, SC, and NSDI.
Spatiotemporal graph neural networks (ST-GNNs) are powerful tools for modeling spatial and temporal data dependencies. However, their applications have been limited primarily to small-scale datasets because of memory constraints. While distributed training offers a solution, current frameworks lack support for spatiotemporal models and overlook the properties of spatiotemporal data. Informed by a scaling study on a large-scale workload, we present PyTorch Geometric Temporal Index (PGT-I), an extension to PyTorch Geometric Temporal that integrates distributed data parallel training and two novel strategies: index-batching and distributed-index-batching. Our index techniques exploit spatiotemporal structure to construct snapshots dynamically at runtime, significantly reducing memory overhead, while distributed-index-batching extends this approach by enabling scalable processing across multiple GPUs. Our techniques enable the first-ever training of an ST-GNN on the entire PeMS dataset without graph partitioning, reducing peak memory usage by up to 89% and achieving up to a 11.78x speedup over standard DDP with 128 GPUs.
Seth Ockerman is a third year Ph.D. student studying Computer Science at the University of Wisconsin-Madison, advised by Professor Shivaram Venkataraman. His research focuses on characterizing and scaling AI workloads on high-performance computing platforms. His graduate studies are supported by the NSF Graduate Research Fellowship Program and an Argonne Student Appointment. Prior to graduate school, Seth received a B.S. in Computer Science from Grand Valley State University.
Memory tiering in datacenters does not achieve its full potential due to hotness fragmentation – the intermingling of hot and cold objects within memory pages. This fragmentation prevents page-based reclamation systems from distinguishing truly hot pages from pages containing mostly cold objects, fundamentally limiting memory efficiency despite highly skewed accesses. We introduce address-space engineering: dynamically reorganizing application virtual address spaces to create uniformly hot and cold regions that any page-level tiering backend can manage effectively. HADES demonstrates this frontend/backend approach through a compiler-runtime system that tracks and migrates objects based on access patterns, requiring minimal developer intervention. Evaluations across ten data structures achieve up to 70% memory reduction with 3% performance overhead, showing that address space engineering enables existing reclamation systems to reclaim memory aggressively without performance degradation.
Vinay Banakar is a PhD candidate in the Computer Sciences Department at the University of Wisconsin-Madison and a Visiting Researcher in the Google Systems Research Group. Advised by Prof. Andrea and Remzi Arpaci-Dusseau, his research focuses on computer systems, including operating systems, distributed storage systems, and systems for ML. He recently cofounded the Research 2 Impact (https://r2i.cs.wisc.edu/) student organization.
CXL is a recent hardware innovation promising server memory capacity and bandwidth expansion. A large body of research has explored automatic data placement across local and CXL-backed memory, with emerging work focusing on adaptable systems that dynamically adjust tiering policies based on workload behavior. In this talk, I will provide: - A high-level overview of CXL and its role in memory expansion. - An introduction to tiered memory systems. Common methods and strategies used. - Examples of recent adaptive tiering approaches. - An overview of my current research on detecting memory access patterns to enable localized tiering strategies.
Hayden Coffey is a fifth-year Ph.D. advised by Michael Swift. His research centers on heterogeneous memory systems, with a particular emphasis on adaptive tiering and resource management. He is especially interested in monitoring workload characteristics to derive actionable metrics for informing dynamic memory management strategies.
The field of AI is experiencing unprecedented scale, which has made power and energy a key concern in building and deploying AI models in the real world. This talk will first discuss current challenges in AI power management and energy consumption, and how increasing scale will exacerbate them. Following this, I will discuss my academic and open-source contributions to systematically measuring, understanding, and optimizing the energy consumption of AI, leading to our vision for a cross-layer optimized, energy-optimal AI stack.
Jae-Won Chung is a fifth year PhD candidate in Computer Science at the University of Michigan. He builds efficient software systems for machine learning, with a recent focus on the efficient management of not only time, but also energy. Jae-Won leads the ML.ENERGY Initiative as part of his research and open-source works, which has received recognition from industry including the PyTorch Foundation, GitHub, and the Mozilla Foundation. Homepage: https://jaewonchung.me/about
The growing context lengths of large language models (LLMs) pose significant challenges for efficient inference, primarily due to GPU memory and bandwidth constraints. We present RetroInfer, a novel system that reconceptualizes the key-value (KV) cache as a vector storage system which exploits the inherent attention sparsity to accelerate long-context LLM inference. At its core is the wave index, an Attention-aWare VEctor index that enables efficient and accurate retrieval of critical tokens through techniques such as tripartite attention approximation, accuracy-bounded attention estimation, and segmented clustering. Complementing this is the wave buffer, which coordinates KV cache placement and overlaps computation and data transfer across GPU and CPU to sustain high throughput. Unlike prior sparsity-based methods that struggle with token selection and hardware coordination, RetroInfer delivers robust performance without compromising model accuracy. Experiments on long-context benchmarks show up to 4.5X speedup over full attention within GPU memory limits and up to 10.5X over sparse attention baselines when KV cache is extended to CPU memory, while preserving full-attention-level accuracy. The paper is available at https://arxiv.org/abs/2505.02922, and the source code has been released at https://github.com/microsoft/RetrievalAttention.
Jing Liu is a Senior Researcher at Microsoft Research Asia. She received her Ph.D. in 2024 from the University of Wisconsin–Madison. Her research focuses on evolving operating systems and file and storage systems to meet modern demands, and she is also exploring ways to better support modern LLM-based applications. She received the Best Paper Award at FAST 2025 and a Meta PhD Research Fellowship in 2022.
This talk discusses how recent advances in GPU hardware as well as a maturing GPU software ecosystem have removed many of the key barriers that have limited the wider adaption of GPU powered data analytics. We will then discuss Sirius, a GPU native SQL execution engine that offers drop-in acceleration for diverse data systems. By leveraging the standard Substrait query representation and libraries like libcudf, Sirius can replace CPU engines without changing the user facing interface.
Devesh is a graduating master’s student working with Shivaram Venkatarman and Xiangyao Yu.
We will discuss Quake, an indexing system for vector search to appear in OSDI’25. Quake is a partitioned index that dynamically adapts the index to the workload. Quake is guided by a cost model that predicts query latency based on index partition sizes and access frequency. Furthermore, it employs a recall estimation model to automatically meet query recall targets without tuning.
Jason is a graduating Ph.D. student here at UW advised by Shivaram Venkatarman and Theodoros Rekatsinas. He primarily works on retrieval systems.
In this talk, we will look at KernMLOps. KernMLOps is an MLOps framework designed for enabling machine learning inside the kernel. We will look at how KernMLOps supports three critical phases of machine learning - data collection, model deployment, and model updates. The talk will also feature a live demo (fingers crossed).
Saurabh is a post-doc at UT-Austin, working with Aditya Akella and Chris Rossbach. He usually works on building infrastructure for ML, ranging from distributed training and inference. Currently, he is interested in how to enable ML for systems and augmented LLM serving systems.
Many cloud applications use both serverless functions, for bursts of stateless parallel computation, and container orchestration, for long-running microservices and tasks that need to interact. Ideally a single platform would offer the union of these systems’ capabilities, but neither is sufficient to act as that single platform: serverless functions are lightweight but cannot act as servers with long-term state, while container orchestration offers general-purpose computation but instance start-up takes too long to support burst parallelism.
σOS is a new multi-tenant cloud operating system that combines the best of container orchestration and serverless in one platform with one API. σOS computations, called procs, can be long-running, stateful, and interact with each other, making them a good match for both serverless and microservice tasks. A key aspect of the σOS design is its cloud-centric API, which provides flexible management of computation, a novel abstraction for communication endpoints, σEPs— which allow procs of a tenant to communicate efficiently but prohibits procs from sending packets to other tenants—and a flexible naming system to name, for example, σEPs. Quick proc start-up is important for serverless uses. A key enabling observation is that both serverless and microservice applications rely on cloud services for much of the work traditionally done by the local OS (e.g., access to durable storage and additional compute resources). σOS exploits this observation by providing only a small and generic local operating system image to each proc, which can be created much more quickly than a container orchestration instance since σOS need not install application-specific filesystem content or (due to σOS’s σEPs) configure an isolated overlay network.
Microbenchmarks show that σOS can cold start a proc in 7.7 msec and can create 36,650 procs per second, distributing them over a 24-machine cluster. An evaluation of σOS with two microservice applications from DeathStarBench, a MapReduce application, and an image processing benchmark, shows that the σOS API supports both microservices and lambda-style computations, and provides better performance than corresponding versions on AWS Lambda and Kubernetes.
Data deduplication is used to conserve storage space and network bandwidth. Content-defined chunking (CDC) algorithms divide data into chunks, dictating the space-saving efficiency of deduplication systems. However, modern CDC algorithms are slow due to their compute-intensive nature and need to scan large amounts of data, becoming one of the main bottlenecks in the deduplication pipeline. In this talk, I will present two solutions to accelerate content-defined chunking. The first solution, VectorCDC, uses AVX-friendly techniques to redesign and accelerate existing chunking algorithms. The second solution, SeqCDC, presents a new vector-friendly algorithm that uses content-defined heuristics to selectively skip scanning data regions, improving throughput without significantly affecting space savings.
Sreeharsha is a 4th year PhD student at the University of Waterloo advised by Prof. Samer Al-Kiswany. His research focuses on incorporating hardware acceleration into large-scale distributed systems.
CXL introduces new opportunities for memory expansion, cross-host memory pooling and sharing. However, its practical adoption is shaped by hardware constraints that directly impact system software and applications. This talk explores key hardware constraints in CXL pod design highlighting scaling limits in existing “fully connected” approaches based on switches and multi-ported devices. We introduce “loosely connected” CXL pods, which open a new design space including a range of new tradeoffs. We present preliminary evaluation results that highlight the benefits of our new pod design for some applications, and the challenges for others.
Autotuning plays a pivotal role in optimizing the performance of systems, particularly in large-scale cloud deployments. One of the main challenges in performing autotuning in the cloud arises from performance variability. We first investigate the extent to which noise slows autotuning and find that as little as 5% noise can lead to a 2.5x slowdown in converging to the best-performing configuration. We measure the magnitude of noise in cloud computing settings and find that while some components (CPU, disk) have almost no performance variability, there are still sources of significant variability (caches, memory). Furthermore, variability leads to autotuning finding unstable configurations. As many as 63.3% of the configurations selected as “best” during tuning can have their performance degrade by 30% or more when deployed. Using this as motivation, we propose a novel approach to improve the efficiency of autotuning systems by (a) detecting and removing outlier configurations and (b) using ML-based approaches to provide a more stable true signal of de-noised experiment results to the optimizer. The resulting system, TUNA (Tuning Unstable and Noisy Cloud Applications) enables faster convergence and robust configurations. Tuning postgres running mssales, an enterprise production workload, we find that TUNA can lead to 1.88x lower running time on average with 2.58x lower standard deviation compared to traditional sampling methodologies.
DeepSeek recently open-sourced its distributed filesystem, which fully utilizes the bandwidth of modern SSDs and RDMA networks. This session will discuss 3FS design choices, including interface, metadata management, replication, data placement, and failure handling.
Reading:
Resource as a Service (RaaS) is a cloud computing model that enables fine-grained, pay-as-you-go access to computational resources, with provider-controlled economic mechanisms shaping resource availability and pricing. In this talk we will discuss RaaS, the challenges and opportunities presented by the model, and example price-aware systems.
Blockchain is the technology behind cryptocurrencies like Bitcoin and Ethereum. It enables building a distributed storage system in a trustless setting, which can then be used to create various web3 applications. In this talk, we will provide an overview of how blockchains work and what are some ongoing research directions, with emphasis on system-related topics.
This talk presents the Aurora single level store, an OS design that uses continuous checkpointing for application persistence and deployment. Aurora provides submillisecond application checkpoint and restore operations to efficiently turn applications into on-disk images and back. Fast checkpointing/restore as an OS service also serves as a foundation for further research into open problems like efficient persistence APIs for memory-mapped data and serverless computing.
Aurora’s single level store-based persistence has recently become practical because of advances in hardware and file system technology. Modern SSD storage devices have low latency at 10μs, allowing us to persist application checkpoints to the disk with minimal latency overhead. Modern CPUs also have IO throughput that rivals that of their memory bandwidth, making it possible to continuously checkpoint and forward in-memory application state to the disk.
This talk describes three systems that demonstrate the efficiency and flexibility of the single level store paradigm. I will first be presenting Aurora (SOSP 2021), an OS design capable of continuous application checkpointing at a fast enough granularity to provide transparent persistence. I will be following up with MemSnap (ASPLOS 2024), an OS single level store API and associated virtual memory mechanism. MemSnap persists application data, e.g., database data, more efficiently than the file API. Finally, I will be presenting Metropolis, a serverless invoker that uses the single level store paradigm to create serverless function instances at sub-millisecond latency.
Many users today have tens to hundreds of accounts with web services that store sensitive data, from social media to tax preparation and e-commerce sites. While users have the right to delete their data (via e.g., the GDPR or CCPA), users want and deserve more nuanced controls over their data that don’t exist today. For example, a user might wish to hide and protect data of an e-commerce or dating app profile when inactive, but also want their data to be present should they return to use the application. Today, however, services often provide only coarse-grained, blunt tools that result in all-or-nothing exposure of users’ private information.
This thesis introduces the notion of disguised data, a reversible state of data in which sensitive data is selectively hidden. To demonstrate the feasibility of disguised data, this thesis also presents Edna—the first system for disguised data—which helps database-backed web applications allow users to remove their data without permanently losing their accounts, anonymize their old data, and selectively dissociate personal data from public profiles. Edna helps developers support these features while maintaining application functionality and referential integrity in the database via disguising and revealing transformations. Disguising selectively renders user data inaccessible via encryption, and revealing enables the user to restore their data to the application. Edna’s techniques allow transformations to compose in any order, e.g., deleting a previously anonymized user’s account, or restoring an account back to an anonymized state.
With Edna, web applications can enable flexible privacy features with reasonable developer effort and moderate performance impact on application operation throughput. In the Lobsters social media application—a 160k LoC web application with >16k users—adding Edna and its features takes less than 1k LoC, and decreases throughput 1-7% in the common case and up to 28% in the worst case (when the user owning 1% of all application data continuously disguises and reveals their account).
Lily Tsai currently works as part of SystemsResearch@Google (SRG), researching systematic ways to achieve better data privacy and security in data warehouses, ML frameworks, and more. She just graduated with her PhD at MIT in the PDOS group in 2024, where her research with Malte Schwarzkopf and Frans Kaashoek aimed to design systems for better data protections and security in web applications. Besides research, Lily loves to play violin, read sci-fi, hike, climb, and explore the world around her.
Sujay Yadalam will lead the discussion on cloud platforms and workloads. The discussion will focus on this recent paper from Microsoft: Workload Intelligence: Punching Holes Through the Cloud Abstraction
He will also briefly talk about resource harvesting:
Cluster managers, such as Kubernetes, are among the most critical systems in modern clouds. Modern cluster managers are architected as a fleet of controllers that manage all the applications, services, and resources in the cloud, thus making controller correctness paramount. We have developed several testing techniques (including Acto for functional testing and Sieve for fault-tolerance testing) for cluster management controllers in the past few years. Besides many serious controller bugs we discovered through testing, we learned valuable lessons that controller bugs have diverse causes such as concurrency, asynchrony, and failures, and often lead to liveness violations. Formal verification is promising for avoiding bugs in system software, but most work so far focused on safety, instead of liveness.
In this talk, I will present Anvil, our effort for building provably correct cluster managers. Anvil is a framework for developing practical controller implementations and verifying that the controllers correctly implement liveness and safety properties. One key challenge is to develop a formal specification that precludes a broad range of controller bugs and is generally applicable to diverse controllers. I will present how we address this challenge with Eventually Stable Reconciliation, a general and powerful specification written as a concise temporal logic liveness property. I will also present how to use Anvil to verify three Kubernetes controllers for managing ZooKeeper, RabbitMQ, and FluentBit, which can readily be deployed in Kubernetes platforms and achieve feature parity and the same performance compared to widely used unverified controllers. The proof effort is reasonable compared to previous work due to Anvil’s rich features for assisting temporal reasoning. Lastly, I will talk about our ongoing work on extending Anvil to verify cross-controller interactions compositionally and reducing the trusted computing base by verifying Kubernetes core controllers using Anvil.
Xudong Sun is a final year CS PhD student at the University of Illinois Urbana-Champaign, advised by Prof. Tianyin Xu. Xudong’s research focuses on improving system reliability using formal verification, systematic testing, fault injection, and model checking. Xudong’s research was recognized with the Jay Lepreau Best Paper Award at OSDI’2024, Mavis Future Faculty Fellowship, and Yunni and Maxine Pao Memorial Fellowship. Xudong is on the job market looking for research positions in academia and industry in the 2024 - 2025 cycle.
Prof. Tej Chajed will give a discussion/tutorial on systems verification.
Reading list:
Serverless computing enables scalable, cost-efficient, and simple to program applications by allowing users to develop their workloads as compositions of lightweight functions, while the cloud provider manages the underlying infrastructure. However, serverless workloads differ significantly from traditional cloud applications—they are short-lived, have smaller data and instruction footprints, experience bursty request arrival patterns, involve high I/O activity, and undergo frequent context switches. These characteristics result in performance, energy, and resource inefficiencies when run on conventional server-class processors.
To address these challenges, I introduce μManycore (ISCA ‘23), a processor architecture optimized for serverless environments. Unlike traditional processors, μManycore focuses on minimizing tail latency, the key performance metric in cloud environments. It achieves this goal by removing the main contention points in the system. First, instead of having a chip-wide cache coherence, μManycore organizes its cores into small cache-coherent villages. Then, it connects the villages via a leaf-spine network topology that provides low-latency, redundant paths between clusters of villages. Finally, μManycore is augmented with a hardware support for request scheduling and context switching.
Building on μManycore, I propose Mosaic (MICRO ‘24), a core micro-architecture optimized for serverless workloads. Mosaic slices micro-architectural structures, such as caches and branch predictors, into small chunks and assigns tiles of such chunks to functions. The processor retains the state of functions in their tiles across context switches, thereby improving performance. Furthermore, currently inactive tiles are set to a low power mode, thereby reducing energy consumption. Together, μManycore and Mosaic deliver significant improvements in tail latency, throughput, and power efficiency for serverless environments.
Jovan Stojkovic is a final year PhD student at the University of Illinois at Urbana-Champaign advised by Professor Josep Torrellas. His research interests are in hardware and software abstractions for cloud platforms and emerging computing paradigms, such as microservices and serverless computing. His work appeared in major computer architecture conferences, such as ISCA, MICRO, ASPLOS, and HPCA. He was awarded multiple distinctions, such as Mavis Future Faculty Fellowship, Young Researcher at the Heidelberg Laureate Forum, Kenichi Miura Award for excellence in high performance computing, and IEEE Top Picks Honorable Mention. He is currently at the academic job market.
Anjali will lead the topic discussion on serverless computing.
Cloud providers seek to deploy CXL-based memory to increase aggregate memory capacity, reduce costs, and lower carbon emissions. However, CXL accesses incur higher latency than local DRAM. Existing systems use software to manage data placement across memory tiers at page granularity. Cloud providers are reluctant to deploy software-based tiering due to high overheads in virtualized environments. Hardware-based memory tiering could place data at cacheline granularity, mitigating these drawbacks. However, hardware is oblivious to application-level performance.
We propose combining hardware-managed tiering with software-managed performance isolation to overcome the pitfalls of either approach. We introduce Intel Flat Memory Mode, the first hardware-managed tiering system for CXL. Our evaluation on a full-system prototype demonstrates that it provides performance close to regular DRAM, with no more than 5% degradation for more than 82% of workloads. Despite such small slowdowns, we identify two challenges that can still degrade performance by up to 34% for “outlier” workloads: (1) memory contention across tenants, and (2) intra-tenant contention due to conflicting access patterns.
To address these challenges, we introduce Memstrata, a lightweight multi-tenant memory allocator. Memstrata employs page coloring to eliminate inter-VM contention. It improves performance for VMs with access patterns that are sensitive to hardware tiering by allocating them more local DRAM using an online slowdown estimator. In multi-VM experiments on prototype hardware, Memstrata is able to identify performance outliers and reduce their degradation from above 30% to below 6%, providing consistent performance across a wide range of workloads.
Yuhong Zhong is a third-year PhD student in Computer Science at Columbia University, advised by Asaf Cidon. He is broadly interested in computer systems, especially CXL, memory tiering, storage systems, and eBPF. His research was recognized with the Best Paper Award at OSDI ‘22. Before starting his PhD, Yuhong was a software engineer at VMware in the vSAN group.
