Zyphra
Zyphra's ZHA Delivers True Subquadratic Attention
Zyphra AI unveils Zyphra Hierarchical Attention (ZHA), a sparse attention mechanism that breaks the quadratic cost barrier of transformers, promising efficient scaling for long-context and multimodal models.