Pinned Loading

1. seq2seq-learning seq2seq-learning Public

   From-scratch PyTorch implementation of Sutskever et al. (2014), sequence to sequence learning with stacked LSTMs, source reversal and beam search, with an analysis of why it plateaus at small scale.

   Python

2. align-and-translate align-and-translate Public

   From-scratch PyTorch implementation of Bahdanau et al. (2014), "Neural Machine Translation by Jointly Learning to Align and Translate": bidirectional GRU encoder, additive attention, maxout decoder…

   Python

3. attention-is-all-you-need attention-is-all-you-need Public

   From-scratch PyTorch implementation of Vaswani et al. (2017), the Transformer with multi-head attention, sinusoidal positional encodings and beam search, verified by built-in checks and trained on …

   Python

4. universal-transformers universal-transformers Public

   From-scratch PyTorch implementation of Dehghani et al. (2019), the Universal Transformer with weight sharing across depth, coordinate embeddings and Adaptive Computation Time, trained on CPU on cop…

   Python

5. gpt-1 gpt-1 Public

   From-scratch PyTorch implementation of Radford et al. (2018), GPT-1, with a decoder-only post-LN Transformer, weight tying, GPT-1’s training recipe, and fine-tuning with the input transformation an…

   Python

6. tiny-serve tiny-serve Public

   A minimal from scratch clone of vLLM's LLMEngine, built to understand LLM inference.

   Python