Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Config for running data ablations #464

Merged
merged 67 commits into from
Mar 21, 2024
Merged

Config for running data ablations #464

merged 67 commits into from
Mar 21, 2024

Commits on Feb 23, 2024

  1. Makes R2 work, and adds an ablation config

    @dirkgr
    dirkgr committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    50a7704 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into olmo7-ablations

    @dirkgr
    dirkgr authored Feb 23, 2024
    Configuration menu
    Copy the full SHA
    ae538ce View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. Script for running ablations on LUMI

    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    404ea30 View commit details
    Browse the repository at this point in the history

  2. It's no longer just s3.

    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    005c406 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'olmo7-ablations' of https://github.com/allenai/LLM into… 

    … olmo7-ablations
    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    25a0f4f View commit details
    Browse the repository at this point in the history

  4. We now think the 1T checkpoint is better.

    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    399d33c View commit details
    Browse the repository at this point in the history

  5. Try the spawn start method

    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    6d993f3 View commit details
    Browse the repository at this point in the history

  6. Revert "Try the spawn start method" 

    This reverts commit 6d993f3.
    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    a67053f View commit details
    Browse the repository at this point in the history

  7. Set start method right away

    @dirkgr
    dirkgr committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    80fba0c View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. Merge branch 'main' into olmo7-ablations

    @dirkgr
    dirkgr authored Feb 28, 2024
    Configuration menu
    Copy the full SHA
    3834c62 View commit details
    Browse the repository at this point in the history

  2. Different seed, so we don't train on the same data twice

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    0cc7f20 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/mmlu-downstream' into olmo7-abla… 

    …tions
    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    d765e88 View commit details
    Browse the repository at this point in the history

  4. Config for MosaicML

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    cdb6ad9 View commit details
    Browse the repository at this point in the history

  5. Mcli has changed its format

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    305d1a8 View commit details
    Browse the repository at this point in the history

  6. Config tweaks

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    e2d0631 View commit details
    Browse the repository at this point in the history

  7. More mcli changes

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    251e89a View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into olmo7-ablations

    @OyvindTafjord
    OyvindTafjord committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    08c1fcb View commit details
    Browse the repository at this point in the history

  9. Update downstream tasks

    @OyvindTafjord
    OyvindTafjord committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    f979478 View commit details
    Browse the repository at this point in the history

  10. We also changed our formats.

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    0074545 View commit details
    Browse the repository at this point in the history

  11. It's not my day today.

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    c1d664b View commit details
    Browse the repository at this point in the history

  12. Changelog

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    08df810 View commit details
    Browse the repository at this point in the history

  13. isort

    @dirkgr
    dirkgr committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    0b7c26e View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. Huggingface offline datasets

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    1961c25 View commit details
    Browse the repository at this point in the history

  2. Not sure what's going on with openbookqa. 

    Luca says it's redirects.
    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    9e927f1 View commit details
    Browse the repository at this point in the history

  3. Don't use compile

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    c3c1a28 View commit details
    Browse the repository at this point in the history

  4. Back to microbatch of 2

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    7c2b1dd View commit details
    Browse the repository at this point in the history

  5. Same settings as we did for OLMo 7B

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    5083517 View commit details
    Browse the repository at this point in the history

  6. Turn off compile

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    b744d5e View commit details
    Browse the repository at this point in the history

  7. Lots of checkpointing

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    38f8817 View commit details
    Browse the repository at this point in the history

  8. Old version of torch

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    d849d40 View commit details
    Browse the repository at this point in the history

  9. Revert "Old version of torch" 

    This reverts commit d849d40.
    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    d7b2e59 View commit details
    Browse the repository at this point in the history

  10. mbsz 3

    @dirkgr
    dirkgr committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    c773863 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. More GPUs, bigger batch

    @dirkgr
    dirkgr committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    75aacd8 View commit details
    Browse the repository at this point in the history

  2. Set a group name

    @dirkgr
    dirkgr committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    a02ae9c View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2024

  1. Save and eval more often

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    23951d1 View commit details
    Browse the repository at this point in the history

  2. Dolma 1.7 config

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    3d71dd5 View commit details
    Browse the repository at this point in the history

  3. Run less GPUs for longer

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    ead9ac8 View commit details
    Browse the repository at this point in the history

  4. Configure remote save folders

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    20b6514 View commit details
    Browse the repository at this point in the history

  5. Give better names to the configs. Also run the baseline somewhere else.

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    85492da View commit details
    Browse the repository at this point in the history

  6. Warm the cache for starter checkpoints

    @dirkgr
    dirkgr committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    a4c0f09 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. Revert "Warm the cache for starter checkpoints" 

    This reverts commit a4c0f09.
    @dirkgr
    dirkgr committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    5e3bda7 View commit details
    Browse the repository at this point in the history

  2. LLM is now OLMo

    @dirkgr
    dirkgr committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    c077d03 View commit details
    Browse the repository at this point in the history

  3. Adds ability to show all logs

    @dirkgr
    dirkgr committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    17891f1 View commit details
    Browse the repository at this point in the history

  4. Uses ability to show all logs

    @dirkgr
    dirkgr committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    ce70c8b View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Merge branch 'main' into olmo7-ablations 

    # Conflicts:
#	CHANGELOG.md
    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    d5ca6e4 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into olmo7-ablations 

    # Conflicts:
#	CHANGELOG.md
#	olmo/util.py
#	pyproject.toml
    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    2be09c9 View commit details
    Browse the repository at this point in the history

  3. It's called all_ranks now.

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    835dfcf View commit details
    Browse the repository at this point in the history

  4. New MMLU evals

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    c21f6b9 View commit details
    Browse the repository at this point in the history

  5. Config for dedupedocs

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    5411dc9 View commit details
    Browse the repository at this point in the history

  6. Disable the MMLU var evals

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    f4ebb62 View commit details
    Browse the repository at this point in the history

  7. Bring back the vars

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    1a32bec View commit details
    Browse the repository at this point in the history

  8. Fix uninitialized prompts bug

    @OyvindTafjord @dirkgr
    OyvindTafjord authored and dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    aad1e82 View commit details
    Browse the repository at this point in the history

  9. No more checkpointing

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    b8ad5b8 View commit details
    Browse the repository at this point in the history

  10. More speed

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    0ada799 View commit details
    Browse the repository at this point in the history

  11. Compile is still broken

    @dirkgr
    dirkgr committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    608dfe5 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. Let's try SHARD_GRAD_OP

    @dirkgr
    dirkgr committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    d434011 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. Config for dedupeparas

    @dirkgr
    dirkgr committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    86e9a3f View commit details
    Browse the repository at this point in the history

  2. New cluster who dis?

    @dirkgr
    dirkgr committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    6524f87 View commit details
    Browse the repository at this point in the history

  3. Missing budget

    @dirkgr
    dirkgr committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    f640740 View commit details
    Browse the repository at this point in the history

  4. Warm HF Cache in Beaker

    @dirkgr
    dirkgr committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    a0cb855 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. Adds a script to continue the baseline run on Beaker

    @dirkgr
    dirkgr committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    963aa0b View commit details
    Browse the repository at this point in the history

  2. 8 nodes

    @dirkgr
    dirkgr committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    66dc953 View commit details
    Browse the repository at this point in the history

  3. 2xnonweb

    @dirkgr
    dirkgr committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    87fc58d View commit details
    Browse the repository at this point in the history

  4. Refheavy

    @dirkgr
    dirkgr committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    6ff85b9 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. Final2 config

    @dirkgr
    dirkgr committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    cd5c196 View commit details
    Browse the repository at this point in the history

  2. Indentation to make comparisons work

    @dirkgr
    dirkgr committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1fadaf8 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. Merge branch 'main' into olmo7-ablations

    @dirkgr
    dirkgr authored Mar 21, 2024
    Configuration menu
    Copy the full SHA
    51303ea View commit details
    Browse the repository at this point in the history