Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[small bug + generalization] saving config.yaml to output_dir #2137

Open
felipemello1 opened this issue Dec 9, 2024 · 1 comment
Open

[small bug + generalization] saving config.yaml to output_dir #2137

felipemello1 opened this issue Dec 9, 2024 · 1 comment
Labels
better engineering Tasks which help improve eng productivity e.g. building tools, cleaning up code, writing docs bug Something isn't working

Comments

@felipemello1
Copy link
Contributor

felipemello1 commented Dec 9, 2024

Currently our WandBLogger has this code:

try:
    output_config_fname = Path(
        os.path.join(
            config.checkpointer.checkpoint_dir,
            "torchtune_config.yaml",
        )
    )
    OmegaConf.save(config, output_config_fname)

    log.info(f"Logging {output_config_fname} to W&B under Files")
    self._wandb.save(
        output_config_fname, base_path=output_config_fname.parent
    )

except Exception as e:
    log.warning(
        f"Error saving {output_config_fname} to W&B.\nError: \n{e}."
        "Don't worry the config will be logged the W&B workspace"
    )

It is great that is saves the config, however, there are a few problems with it:

  1. This should be done for every config, not only WandBLogger. One option is to add this to every logger, but maybe this should be elsewhere, so it is not related to the logger.
  2. It saves it to the checkpoint_dir, which doesnt make sense. It should save the config to output_dir/...
  3. it does try/except, but if it fails to load output_config_fname, the except will also fail, because it tries to print "output_config_fname"
@felipemello1 felipemello1 added bug Something isn't working better engineering Tasks which help improve eng productivity e.g. building tools, cleaning up code, writing docs labels Dec 9, 2024
@joecummings
Copy link
Contributor

@felipemello1 Can I assign this to you?

@joecummings joecummings added triage review This issue should be discussed in weekly review and removed triage review This issue should be discussed in weekly review labels Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
better engineering Tasks which help improve eng productivity e.g. building tools, cleaning up code, writing docs bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants