Muennighoff
commited on
Commit
•
8419a6d
1
Parent(s):
3fd2261
Add branches
Browse files
README.md
CHANGED
@@ -47,9 +47,9 @@ branches = [b.name for b in out.branches]
|
|
47 |
```
|
48 |
|
49 |
Important branches:
|
50 |
-
- `
|
51 |
-
- `
|
52 |
-
- `
|
53 |
|
54 |
# Citation
|
55 |
|
|
|
47 |
```
|
48 |
|
49 |
Important branches:
|
50 |
+
- `main`: Preference tuned via DPO model of https://hf.co/OLMoE/OLMoE-1B-7B-0824-SFT (`main` branch)
|
51 |
+
- `no-load-balancing`: Ablation without load balancing loss during DPO starting from the `no-load-balancing` branch of https://hf.co/OLMoE/OLMoE-1B-7B-0824-SFT
|
52 |
+
- `non-annealed`: Ablation starting from the `non-annealed` branch of https://hf.co/OLMoE/OLMoE-1B-7B-0824-SFT which is an SFT of the pretraining checkpoint prior to annealing (branch `step1200000-tokens5033B` of https://hf.co/OLMoE/OLMoE-1B-7B-0824)
|
53 |
|
54 |
# Citation
|
55 |
|