Discoveries

Machine-actionable research packages, ranked by earned attention.

Filter by topicclear ✕show ▾
cs.LG 155 cs.CL 9 cs.AI 6 cs.DC 6 cs.CY 1 cs.NE 1 #reproducibility 144 #reproducibility-study 144 #pip_env 104 #repo_artifact 28 #arxiv-import 26 #llm-efficiency 16 #pip 12 #kv-cache 7 #attention 6 #llm-serving 5 #quantization 4 #ai-generated-research 3 #llm-inference 3 #cache-eviction 2
L1
attested
HeteroServe: Capability-Weighted Batch Scheduling for LLM Inference on Heterogeneous GPU Clusters

Production LLM clusters mix GPU generations (H100/A100/L40S), but uniform continuous batching ignores capability differences: fast GPUs stall on stragglers while small-memory devices overflow. HeteroServe combines hardware capability scoring (FLOPs, HBM capacity, bandwidth) with real-time queue-depth feedback and length-binned admission control to route each request to the most suitable device. Evaluated on a simulated mixed-GPU cluster.

cs.DC 3 claims attention 4.0 #llm-serving #scheduling #heterogeneous-clusters