pdelobelle
commited on
Commit
•
9cc3928
1
Parent(s):
172de46
Update README.md
Browse files
README.md
CHANGED
@@ -35,8 +35,8 @@ We are also continuously working on releasing better-performing models, so watch
|
|
35 |
# The models
|
36 |
| Model | Description | Parameters | Training size | Huggingface id |
|
37 |
|--------------|-------------|------------------|-------------------|------------------------------------------------------------------------------------|
|
38 |
-
| Non-shuffled | Trained on the non-shuffled variant of the oscar corpus, without any operations to preserve this order during training and distillation. | 74 M | 1 GB |
|
39 |
-
| Shuffled | Trained on the publicly available and shuffled OSCAR corpus. | 74 M | 1 GB |
|
40 |
| Merged (p=0.5) | Same as the non-shuffled variant, but sequential sentences of the same document are merged with a probability of 50%. | 74 M | 1 GB | [DTAI-KULeuven/robbertje-1-gb-merged](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-merged) |
|
41 |
| BORT | A smaller version with 8 attention heads instead of 12 and 4 layers instead of 6 (and 12 for RobBERT). | 46 M | 1 GB | [DTAI-KULeuven/robbertje-1-gb-bort](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-bort) |
|
42 |
|
|
|
35 |
# The models
|
36 |
| Model | Description | Parameters | Training size | Huggingface id |
|
37 |
|--------------|-------------|------------------|-------------------|------------------------------------------------------------------------------------|
|
38 |
+
| Non-shuffled | Trained on the non-shuffled variant of the oscar corpus, without any operations to preserve this order during training and distillation. | 74 M | 1 GB | [DTAI-KULeuven/robbertje-1-gb-non-shuffled](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-non-shuffled) |
|
39 |
+
| Shuffled | Trained on the publicly available and shuffled OSCAR corpus. | 74 M | 1 GB | this model |
|
40 |
| Merged (p=0.5) | Same as the non-shuffled variant, but sequential sentences of the same document are merged with a probability of 50%. | 74 M | 1 GB | [DTAI-KULeuven/robbertje-1-gb-merged](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-merged) |
|
41 |
| BORT | A smaller version with 8 attention heads instead of 12 and 4 layers instead of 6 (and 12 for RobBERT). | 46 M | 1 GB | [DTAI-KULeuven/robbertje-1-gb-bort](https://huggingface.co/DTAI-KULeuven/robbertje-1-gb-bort) |
|
42 |
|