Skip to content
Pretraining at home: 20B tokens from 222 hours to 12
ML OPS AND MODEL ENGINEERING

Pretraining at home: 20B tokens from 222 hours to 12

Optimizing training a Llama 3.2 1B model so we can pretrain in a day without going broke.
Shane Caldwell
2 articles

Coverage

2 articles

Related