Skip to content

Latest commit

 

History

History

polypythias

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
data
data
 
 
README.md
README.md
 
 
plots.ipynb
plots.ipynb
 
 

README.md

PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs

To recreate the plots in the paper PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs, download the evaluations available on the Hugging Face Hub at huggingface.co/datasets/EleutherAI/polypythias-evals into the /data subfolder. To reproduce the training map plot, we add the training_maps.tsv file here on GitHub.

Before running the notebook, make sure to download the required packages listed in the first cell of the notebook.

Citation Details

If our work and data is useful to your research, please consider citing our paper via:

@inproceedings{van2025polypythias,
      title={PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs},
      author={van der Wal, Oskar and Lesci, Pietro and M{\"u}ller-Eberstein, Max and Saphra, Naomi and Schoelkopf, Hailey and Zuidema, Willem and Biderman, Stella},
      booktitle={{The Thirteenth International Conference on Learning Representations}},
      year={2025}