Commit
•
4c29a56
1
Parent(s):
4dee57c
Update README.md
Browse files
README.md
CHANGED
|
@@ -60,6 +60,7 @@ This filtration and appropriate selection of texts enable the provision of a con
|
|
| 60 |
|
| 61 |
| **Hyperparameter** | **Value** |
|
| 62 |
|-----------------------------|------------------|
|
|
|
|
| 63 |
| Micro Batch Size | 4 |
|
| 64 |
| Batch Size | 4194304 |
|
| 65 |
| Learning Rate (cosine) | 3e-05 -> 2e-05 |
|
|
|
|
| 60 |
|
| 61 |
| **Hyperparameter** | **Value** |
|
| 62 |
|-----------------------------|------------------|
|
| 63 |
+
| Context length | 4096 |
|
| 64 |
| Micro Batch Size | 4 |
|
| 65 |
| Batch Size | 4194304 |
|
| 66 |
| Learning Rate (cosine) | 3e-05 -> 2e-05 |
|