Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comet fix #481

Merged
merged 8 commits into from
Nov 12, 2019
Merged

Comet fix #481

merged 8 commits into from
Nov 12, 2019

Conversation

rwesterman
Copy link
Contributor

@rwesterman rwesterman commented Nov 8, 2019

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • Did you read the contributor guideline?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #470

I've fixed the NotImplementedError for the name() and version() class methods. I've also added the following functionality:

  • Added project_name and experiment_name arguments so that experiments can be correctly categorized on Comet.ml
  • Added logic to detach tensors and send them to CPU so they can be logged by Comet.ml
  • Passed step_num in log_metrics() function because comet_ml.Experiment uses this as X-axis in graphs
  • Added optional rest_api_key argument, used to find the number of experiments and determine the version property

@rwesterman
Copy link
Contributor Author

This is my first attempt at a PR so I'm not sure why the CI check is failing. It seems to be a failure when installing dependencies, but I haven't altered the requirements.txt file.

docs/Trainer/Logging.md Outdated Show resolved Hide resolved
super(CometLogger, self).__init__()
self.experiment = CometExperiment(*args, **kwargs)
self.experiment = CometExperiment(api_key=api_key, workspace=workspace, project_name=project_name, *args,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't introduce this problem, but maybe you can fix it while you're working on this class. We need to create the experiment lazily so that we don't get duplicates when running in distributed mode. If you're willing to take a stab at fixing this, take a look at one of the other loggers to see how this is done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Nic, thanks for the tips. I've just pushed a version of the code where I attempt to create the experiment lazily, but it's a bit outside my comfort zone at the moment so please let me know if there's a way I could improve the code.

Additionally, I've noticed that the Travis-ci testing has failed the same way for all PRs in the last 24 hours or so. Is that something that can be addressed easily?

Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still need to add test to prove that it works fine...

pytorch_lightning/logging/comet_logger.py Outdated Show resolved Hide resolved
pytorch_lightning/logging/comet_logger.py Show resolved Hide resolved
pytorch_lightning/logging/comet_logger.py Outdated Show resolved Hide resolved
nb_exps = len(self.comet_api.get_experiments(self.workspace, self.project_name))
return nb_exps + 1
else:
return None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if returning None as a version is a good idea

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can tell, logger.version is only being used in the tqdm progress bar. If version is None, it is ignored, otherwise the version is displayed in the progress bar.

If there are future plans to use this version elsewhere, I'm open to suggestions for a different default return value.

@williamFalcon williamFalcon changed the title Comet fix [WIP] Comet fix Nov 9, 2019
@rwesterman
Copy link
Contributor Author

we still need to add test to prove that it works fine...

@Borda This part I might need some help with. Since Comet.ml does logging remotely, the tests likely won't resemble the other logger tests. Should we set up a dummy account on Comet.ml so we can test a real connection?

tests/test_y_logging.py Outdated Show resolved Hide resolved
@neggert
Copy link
Contributor

neggert commented Nov 11, 2019

Other than that one minor comment about changing the type of exception, looks good to me, assuming the tests pass.

Copy link
Contributor Author

@rwesterman rwesterman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pushed a change to address what I believe was the last request, and all tests are passing. Thank you @Borda and @neggert for the help.

@Borda
Copy link
Member

Borda commented Nov 12, 2019

@rwesterman if you feel it is ready, drop WIP from the PR name :)

@rwesterman rwesterman changed the title [WIP] Comet fix Comet fix Nov 12, 2019
@Borda
Copy link
Member

Borda commented Nov 12, 2019

@williamFalcon I believe that it is ready...

@williamFalcon williamFalcon merged commit d1b6b01 into Lightning-AI:master Nov 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CometLogger does not implement name() and version() class methods
4 participants