LLM infrastructure

agentic AI

Agentic AI Scaling Demands New Memory Architecture

As AI agents tackle complex multi-step tasks, traditional memory systems are hitting fundamental scaling limits. New architectural approaches are emerging to handle persistent context across extended workflows.

LLM infrastructure

Joint KV-Cache Encoding: A New Approach to Scalable LLM Serving

New research proposes joint encoding of KV-cache blocks to improve memory efficiency in large language model inference, addressing a key bottleneck in scalable AI deployment.

LLM infrastructure

FlashInfer-Bench: New Framework Optimizes LLM Kernel Performance

Researchers introduce FlashInfer-Bench, a comprehensive benchmarking suite that creates a virtuous cycle for optimizing attention kernels in LLM serving systems, addressing critical infrastructure needs.

LLM infrastructure

FPGA-Based CXL Memory Architecture Tackles LLM KV-Cache Bottlenec

New research proposes CXL-SpecKV, a disaggregated FPGA architecture using CXL memory pooling and speculative prefetching to overcome memory bottlenecks in large language model inference at datacenter scale.