-
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 6 -
Diverse Weight Averaging for Out-of-Distribution Generalization
Paper • 2205.09739 • Published • 1 -
Fusing finetuned models for better pretraining
Paper • 2204.03044 • Published • 5 -
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Paper • 2309.07311 • Published • 2
Niels Horn
nilq
AI & ML interests
Natural language understanding, synthetic emotional speech, mechanistic interpretability.
Recent Activity
updated
a collection
about 1 month ago
Dynamics of Transformer Language Model Features
updated
a model
about 1 month ago
nilq/baby-python-mistral-1L-tiny-TinyStories-ft
updated
a model
about 1 month ago
nilq/baby-python-mistral-1L-tiny-lua-ft
Organizations
Collections
4
Papers
1
models
16
nilq/baby-python-mistral-1L-tiny-TinyStories-ft
Text Generation
•
Updated
•
15
•
1
nilq/baby-python-mistral-1L-tiny-lua-ft
Text Generation
•
Updated
•
19
nilq/baby-python-1L-mistral-lua-stories-slerp
Text Generation
•
Updated
•
290
nilq/baby-python-mistral-1L-tiny-base
Text Generation
•
Updated
•
28
nilq/lua-stories-slerp-mistral-1L-tiny
Text Generation
•
Updated
•
14
nilq/lua-stories-slerp-mistral-2L-tiny
Text Generation
•
Updated
•
14
nilq/mistral-2L-tiny
Text Generation
•
Updated
•
20
nilq/lua-stories-linear-mistral-1L-tiny
Text Generation
•
Updated
•
12
nilq/python-mistral-1L-mini
Text Generation
•
Updated
•
13
nilq/mistral-1L-tiny
Text Generation
•
Updated
•
564
•
5
datasets
9
nilq/baby-python-and-tiny-stories-and-lua
Viewer
•
Updated
•
12.3M
•
51
nilq/baby-python-and-lua
Viewer
•
Updated
•
12.3M
•
55
•
1
nilq/baby-python-and-tiny-stories
Viewer
•
Updated
•
13.9M
•
44
nilq/python-and-tiny-stories
Updated
•
4
nilq/baby-python
Viewer
•
Updated
•
11.7M
•
81
•
1
nilq/small-lua-stack
Viewer
•
Updated
•
559k
•
56
•
2
nilq/small-python-stack
Viewer
•
Updated
•
2.59M
•
77
nilq/babylm-100M
Viewer
•
Updated
•
12.7M
•
52
nilq/babylm-10M
Viewer
•
Updated
•
3.14M
•
64