Attention is great. Are there other operators that scale?
Excited to share our work on Hyena, an alternative to attn that can learn on sequences *10x longer*, up to *100x faster* than optimized attn, by using implicit long convolutions & gating
📜arxiv.org/abs/2302.10866 1/
