d analysis, the full training data used forthese models is openly available (Dolma; Soldainiet al., 2024), including code that produces the train-ing data, and tools for analyzing pretraining data(Elazar et al., 2024). For evaluation, we build onCatwalk (Groeneveld et al., 2023) for downstreamevaluation and Paloma (Magnusson et al., 2023)for perplexity-based evaluation. For adaptation, weuse Open Instruct (Ivison et al., 2023; Wang et al.,2023) to train with instruction and fee