Estimate cost of training the modal may be $30000 (excluding all other expenses) (i may be wrong)
#3
by
nib12345
- opened
Estimate cost for only training the model may be $30k (excluding other expenses).
As per someone on reddit, estimate time was three week to train the model.
Hardware: Each compute node consists of 4 AMD Instinct™ MI250 GPUs. We use 16 nodes for pretraining AMD-OLMo-1B
(I may be totally wrong about estimate.)
