Research papers

I've read

• Attention Is All You Need ↗

  Vaswani et al. · 2017

  Placeholder: the paper that started it all; my notes on reading it.

• The Llama 3 Herd of Models ↗

  Llama Team, Meta AI · 2024

  Placeholder: what the Llama 3 report taught me about training at scale.

I'll read

• Mamba: Linear-Time Sequence Modeling with Selective State Spaces ↗

  Albert Gu, Tri Dao · 2023

• Jamba: A Hybrid Transformer-Mamba Language Model ↗

  Lieber et al., AI21 Labs · 2024