Skip to content

Reproduce OLMO-3-Think-SFT #168

@abhash-er

Description

@abhash-er

This issue thread is to report the reproducibility steps and results of OLMO-3-Think-SFT using OLMo-Core and open-instruct. We are trying to target the 7B parameter model.

  • Specifically we start from the hugging face model, convert it into the OLMo-core format using this script.
  • We also need to prepare data before we run the training script. Since OLMO uses packed training instances, we would need to convert the data from HF.
  • Then we can use the script at OLMo-core repository to run the SFT.

One can already find the scripts for running it on SLURM over here.

Metadata

Metadata

Assignees

Type

No type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions