Post
3276
Small models, BIG impact: SmolLM is here! ๐๐ฌ
We're launching a series of small but mighty language models:
๐๏ธ Super fast - runs on laptops, phones, you name it!
๐ 3 sizes: 130M, 350M, and 1.5B parameters
๐ฅ Outperforms same size models from Meta, Microsoft, and Qwen
๐ Fully open-source: datasets, training code, models
๐๐๐ฒ ๐๐๐๐ญ๐ฎ๐ซ๐๐ฌ
- Trained on FineWeb-Edu and Cosmopedia v2 (largest synthetic pre-training dataset)
- No cloud needed - run locally for privacy and energy efficiency
- Everything is public, from data curation to training steps
๐๐จ๐ญ๐๐ง๐ญ๐ข๐๐ฅ ๐ฎ๐ฌ๐ ๐๐๐ฌ๐๐ฌ
- On-device autocomplete
- Local request parsing
- Custom fine-tuning for specific needs without the need for expensive GPUs
๐๐จ ๐๐๐๐ฉ๐๐ซ
๐ Check it out: https://huggingface.co/collections/HuggingFaceTB/smollm-models-6695016cad7167254ce15966
๐ Run the 360M model in your browser, 100 % private: HuggingFaceTB/SmolLM-360M-Instruct-WebGPU
๐ Read the blog explaining everything in detail: huggingface.co/blog/smollm
Kudos to the stellar team who worked on this project: @loubnabnl @anton-l @eliebak @lvwerra
We're launching a series of small but mighty language models:
๐๏ธ Super fast - runs on laptops, phones, you name it!
๐ 3 sizes: 130M, 350M, and 1.5B parameters
๐ฅ Outperforms same size models from Meta, Microsoft, and Qwen
๐ Fully open-source: datasets, training code, models
๐๐๐ฒ ๐๐๐๐ญ๐ฎ๐ซ๐๐ฌ
- Trained on FineWeb-Edu and Cosmopedia v2 (largest synthetic pre-training dataset)
- No cloud needed - run locally for privacy and energy efficiency
- Everything is public, from data curation to training steps
๐๐จ๐ญ๐๐ง๐ญ๐ข๐๐ฅ ๐ฎ๐ฌ๐ ๐๐๐ฌ๐๐ฌ
- On-device autocomplete
- Local request parsing
- Custom fine-tuning for specific needs without the need for expensive GPUs
๐๐จ ๐๐๐๐ฉ๐๐ซ
๐ Check it out: https://huggingface.co/collections/HuggingFaceTB/smollm-models-6695016cad7167254ce15966
๐ Run the 360M model in your browser, 100 % private: HuggingFaceTB/SmolLM-360M-Instruct-WebGPU
๐ Read the blog explaining everything in detail: huggingface.co/blog/smollm
Kudos to the stellar team who worked on this project: @loubnabnl @anton-l @eliebak @lvwerra