This episode examines why a stock CXL-attached SSD, meaning flash capacity exposed as load/store memory behind a small device-side DRAM cache, serves reused prompt prefixes about three times slower than local DRAM. It explains how prefix caching cuts time-to-first-token by skipping prefill, and how LMCache stores KV in fixed-size token chunks across GPU, host DRAM, and storage tiers. A ramdisk experiment on Qwen3-4B with a 34k-token prefix shows the block I/O interface alone costs 1.8x in TTFT, through staging copies and last-level cache contention, even with no flash involved. A generic next-n prefetcher in the device also fails, because the device sees only page addresses while the serving engine knows which chunks a request needs and in what order. That semantic gap motivates the paper's redesign around the chunk as the shared unit between engine and device, and the episode compares it briefly with the concurrent HyMCache and ITME work.
Sources:
1. Bridging LLM Serving and CXL-SSDs with Chunk-Aware KV Cache Management — Hyunsun Chung, Taewan Noh, Minji Kim, Joo-Young Hwang, Hong-Yeon Kim, Youngjae Kim, 2026
http://arxiv.org/abs/2609.26828v1
2. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms — Huaicheng Li, Daniel S. Berger, Lisa Hsu, Daniel Ernst, Pantea Zardoshti, Stanko Novakovic, Monish Shah, Samir Rajadnya, Scott Lee, Ishwar Agarwal, Mark D. Hill, Marcus Fontoura, Ricardo Bianchini, 2023
https://scholar.google.com/scholar?q=Pond%3A+CXL-Based+Memory+Pooling+Systems+for+Cloud+Platforms
3. TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory — Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Johannes Weiner, Niket Agarwal, Pallab Bhattacharya, Chris Petersen, Mosharaf Chowdhury, Shobhit Kanaujia, Prakash Chauhan, 2023
https://scholar.google.com/scholar?q=TPP%3A+Transparent+Page+Placement+for+CXL-Enabled+Tiered-Memory
4. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, Ion Stoica, 2023
https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention
5. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion, and the LMCache system line (LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference) — Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang (CacheBlend); LMCache team from the University of Chicago and Tensormesh (LMCache), 2024-2025
https://scholar.google.com/scholar?q=CacheBlend%3A+Fast+Large+Language+Model+Serving+for+RAG+with+Cached+Knowledge+Fusion%2C+and+the+LMCache+system+line+%28LMCache%3A+An+Efficient+KV+Cache+Layer+for+Enterprise-Scale+LLM+Inference%29
6. HyMCache: A KV cache framework for multi-turn LLM serving with CXL-Hybrid Memory — Hakbeom Jang, Inho Song, Hoshik Kim, Sam H. Noh, Jongryool Kim, 2026
https://scholar.google.com/scholar?q=HyMCache%3A+A+KV+cache+framework+for+multi-turn+LLM+serving+with+CXL-Hybrid+Memory
7. ITME: Inference tiered memory expansion with disaggregated CXL-hybrid memories — Hakbeom Jang et al., 2026
https://scholar.google.com/scholar?q=ITME%3A+Inference+tiered+memory+expansion+with+disaggregated+CXL-hybrid+memories
8. Cylon: Fast and accurate full-system emulation of CXL-SSDs — Dongha Yoon, Hansen Idden, Jinshu Liu, Berkay Inceisci, Sam H. Noh, Huaicheng Li, 2026
https://scholar.google.com/scholar?q=Cylon%3A+Fast+and+accurate+full-system+emulation+of+CXL-SSDs
9. SkyByte: Architecting an efficient memory-semantic CXL-based SSD with OS and hardware co-design — Haoyang Zhang, Yuqi Xue, Yirui Eric Zhou, Shaobo Li, Jian Huang, 2025
https://scholar.google.com/scholar?q=SkyByte%3A+Architecting+an+efficient+memory-semantic+CXL-based+SSD+with+OS+and+hardware+co-design
10. Overcoming the memory wall with CXL-Enabled SSDs — Shao-Peng Yang, Minjae Kim, Sanghyun Nam, Juhyung Park, Jin-Yong Choi, Eyee Hyun Nam, Eunji Lee, Sungjin Lee, Bryan S. Kim, 2023
https://scholar.google.com/scholar?q=Overcoming+the+memory+wall+with+CXL-Enabled+SSDs
11. Performance characterizations and usage guidelines of Samsung CXL Memory Module Hybrid prototype — Jianping Zeng et al., 2025
https://scholar.google.com/scholar?q=Performance+characterizations+and+usage+guidelines+of+Samsung+CXL+Memory+Module+Hybrid+prototype
12. CacheGen: KV cache compression and streaming for fast large language model serving — Yuhan Liu, Hanchen Li, Yihua Cheng et al., 2024
https://scholar.google.com/scholar?q=CacheGen%3A+KV+cache+compression+and+streaming+for+fast+large+language+model+serving
13. Mooncake: A KVCache-centric disaggregated architecture for LLM serving — Ruoyu Qin et al., 2025
https://scholar.google.com/scholar?q=Mooncake%3A+A+KVCache-centric+disaggregated+architecture+for+LLM+serving
14. DualPath: Accelerating agentic LLM inference by harvesting disaggregated KV-cache storage I/O — Yongtong Wu et al., 2026
https://scholar.google.com/scholar?q=DualPath%3A+Accelerating+agentic+LLM+inference+by+harvesting+disaggregated+KV-cache+storage+I%2FO
15. CXL topology-aware and expander-driven prefetching: Unlocking solid-state drive performance — Dongsuk Oh et al., 2025
https://scholar.google.com/scholar?q=CXL+topology-aware+and+expander-driven+prefetching%3A+Unlocking+solid-state+drive+performance
