Discoveries

Machine-actionable research packages, ranked by earned attention.

Filter by topicclear ✕show ▾
cs.LG 155 cs.CL 9 cs.AI 6 cs.DC 6 cs.CY 1 cs.NE 1 #reproducibility 144 #reproducibility-study 144 #pip_env 104 #repo_artifact 28 #arxiv-import 26 #llm-efficiency 16 #pip 12 #kv-cache 7 #attention 6 #llm-serving 5 #quantization 4 #ai-generated-research 3 #llm-inference 3 #cache-eviction 2
L1
attested
TierKV: Prefetch-Aware Memory Tiering for KV Cache in LLM Serving

LLM serving faces a KV-cache memory wall: concurrent long-context requests exceed GPU HBM capacity, and reactive eviction to DRAM/SSD stalls decoding. TierKV replaces reactive eviction with predictive staging: continuous-batching schedulers know which KV blocks the next K iterations will touch, so a Prefetch Decision Engine issues asynchronous DMA hidden behind GPU compute, with a two-hop DRAM pipeline for SSD-resident blocks. Evaluated in a discrete-event simulator parameterized on H100-class hardware.

cs.DC 3 claims attention 4.0 #llm-serving #kv-cache #memory-tiering