Pythia Scaling Suite

EleutherAI 's Collections

Quirky Models and Datasets

Pile-T5

Quirky Math (archive)

Pile-T5 T5x Checkpoints

Sparse Autoencoders

updated Nov 21, 2023

Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia

Upvote

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Paper • 2304.01373 • Published Apr 3, 2023 • 8

Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m

Text Generation • Updated Jul 26, 2023 • 335k • 19

Note This model was trained after the paper was released.
EleutherAI/pythia-70m

Updated Nov 21, 2023 • 126k • 55

Note This model was trained after the paper was released.
EleutherAI/pythia-160m

Text Generation • Updated Jul 9, 2023 • 124k • 25
EleutherAI/pythia-410m

Text Generation • Updated Jul 9, 2023 • 34.8k • 21
EleutherAI/pythia-1b

Text Generation • Updated Jul 9, 2023 • 80.7k • 32
EleutherAI/pythia-1.4b

Text Generation • Updated Jul 9, 2023 • 26.3k • 22
EleutherAI/pythia-2.8b

Text Generation • Updated Jun 9, 2023 • 25.4k • 28
EleutherAI/pythia-6.9b

Text Generation • Updated Jun 8, 2023 • 25.9k • 48
EleutherAI/pythia-12b

Text Generation • Updated Jul 9 • 84k • 131
EleutherAI/pythia-70m-deduped

Text Generation • Updated Jul 9, 2023 • 87.9k • 24
EleutherAI/pythia-160m-deduped

Text Generation • Updated Jul 9, 2023 • 13.2k • 3
EleutherAI/pythia-410m-deduped

Text Generation • Updated Jul 9, 2023 • 74.6k • 20
EleutherAI/pythia-1b-deduped

Text Generation • Updated Jul 10, 2023 • 21.7k • 17
EleutherAI/pythia-1.4b-deduped

Text Generation • Updated Jun 8, 2023 • 11.2k • 19
EleutherAI/pythia-2.8b-deduped

Text Generation • Updated Jul 9, 2023 • 8.31k • 14
EleutherAI/pythia-6.9b-deduped

Text Generation • Updated Jun 8, 2023 • 14k • 8
EleutherAI/pythia-12b-deduped

Text Generation • Updated Jun 8, 2023 • 8.66k • 51

Upvote