SK hynix Details Memory-Tiering Technologies for AI Inference

SK hynix is developing two technologies to distribute AI inference KV cache across high-bandwidth memory (HBM), dynamic random-access memory (DRAM) and solid-state drives (SSDs), addressing capacity and cost limits as agentic AI and longer-context workloads expand. The technology update follows SK hynix’s record third-quarter operating profit . KV cache is data AI systems retain and reuse during inference. SK hynix said the amount is growing as agentic AI and long-context applications spread. Keeping the entire cache in HBM or DRAM faces capacity and cost constraints. The company’s tiered approach allocates data among all three memory types based on access patterns.

Source: TokenPost