Pythia Scaling Suite

EleutherAI 's Collections

Quirky Models and Datasets

Pile-T5

Quirky Math (archive)

Pile-T5 T5x Checkpoints

Sparse Autoencoders

updated Nov 21, 2023

Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia

Upvote

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Paper • 2304.01373 • Published Apr 3, 2023 • 8

Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m

Text Generation • Updated Jul 26, 2023 • 312k • 18

Note This model was trained after the paper was released.
EleutherAI/pythia-70m

Updated Nov 21, 2023 • 82.9k • 55

Note This model was trained after the paper was released.
EleutherAI/pythia-160m

Text Generation • Updated Jul 9, 2023 • 94.2k • 25
EleutherAI/pythia-410m

Text Generation • Updated Jul 9, 2023 • 34.6k • 20
EleutherAI/pythia-1b

Text Generation • Updated Jul 9, 2023 • 61.4k • 32
EleutherAI/pythia-1.4b

Text Generation • Updated Jul 9, 2023 • 22.7k • 22
EleutherAI/pythia-2.8b

Text Generation • Updated Jun 9, 2023 • 18.9k • 28
EleutherAI/pythia-6.9b

Text Generation • Updated Jun 8, 2023 • 23.5k • 48
EleutherAI/pythia-12b

Text Generation • Updated Jul 9 • 80.4k • 131
EleutherAI/pythia-70m-deduped

Text Generation • Updated Jul 9, 2023 • 88.7k • 24
EleutherAI/pythia-160m-deduped

Text Generation • Updated Jul 9, 2023 • 11.8k • 3
EleutherAI/pythia-410m-deduped

Text Generation • Updated Jul 9, 2023 • 84.6k • 20
EleutherAI/pythia-1b-deduped

Text Generation • Updated Jul 10, 2023 • 11k • 17
EleutherAI/pythia-1.4b-deduped

Text Generation • Updated Jun 8, 2023 • 11.1k • 19
EleutherAI/pythia-2.8b-deduped

Text Generation • Updated Jul 9, 2023 • 8.27k • 14
EleutherAI/pythia-6.9b-deduped

Text Generation • Updated Jun 8, 2023 • 12.4k • 8
EleutherAI/pythia-12b-deduped

Text Generation • Updated Jun 8, 2023 • 8.62k • 51

Upvote