§ feed · storyline

Nous Research Proposes Lighthouse Attention

Nous Research proposes Lighthouse Attention, a training-only hierarchical attention method that improves long-context model pretraining speed by 1.4–1.7x.

May 16 · 02:00:00 · primary fetch1 sourceupdated May 16 · 02:00:00

Nous Research introduced Lighthouse Attention, a training-only hierarchical attention method that improves pretraining speed by 1.4–1.7x for long-context models.

read full article on marktechpost.com ↗

§ sources1 publication · timeline below

marktechpost.comNous Research Proposes Lighthouse Attentionprimary02:00:00