Update README.md
Browse files
README.md
CHANGED
|
@@ -36,4 +36,6 @@ This repository contains all the intermediate checkpoints of OpenCoder-1.5B-Base
|
|
| 36 |
|
| 37 |
- `anneal_iter_0001000` - `anneal_iter_0023841`: Intermediate checkpoints during the annealing stage.
|
| 38 |
|
|
|
|
|
|
|
| 39 |
We use `pretrain_iter_0373000` as the starting point for the annealing stage, and use `anneal_iter_0023000` as the final base model.
|
|
|
|
| 36 |
|
| 37 |
- `anneal_iter_0001000` - `anneal_iter_0023841`: Intermediate checkpoints during the annealing stage.
|
| 38 |
|
| 39 |
+
The number in each branch name indicates the corresponding current training step, where each step consumes 419,4304 training tokens (1,024 batch size * 4,096 sequence length).
|
| 40 |
+
|
| 41 |
We use `pretrain_iter_0373000` as the starting point for the annealing stage, and use `anneal_iter_0023000` as the final base model.
|