Story thread · 2 reports / 2 sources

Astera Labs targets the KV cache bottleneck as agentic AI outgrows GPU memory

digitimes.com · 5h · first reportShare

Astera Labs has added three products to its Leo memory controller line, aimed at a problem that arises as AI inference shifts from single queries to agents that run continuous loops: the key-value cache outgrows the HBM on the accelerator, and the usual fallback makes things worse.