Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add one shot profile uploads to tensorboard uploader. #704

Merged
merged 34 commits into from
Oct 11, 2021
Merged

feat: Add one shot profile uploads to tensorboard uploader. #704

merged 34 commits into from
Oct 11, 2021

Conversation

mkovalski
Copy link
Contributor

Add one shot profile uploading to tensorboard uploader.

  • Adds a profile request sender to upload any profile logs.
  • Move common utilities to a uploader_utils.py file
  • Create RunResourceManager to be shared via request senders.
  • Add _FileRequestSender for uploading non-event based items to blob storage backend.

While this currently allows for only one shot uploads, the ProfileRequestSender class will be reused so profiles can be uploaded on demand via tensorboard.gcp

Fixes #519

mkovalski and others added 23 commits August 23, 2021 15:10
@mkovalski
using dispatcher to relay requests for additional event senders
99b3519
@mkovalski
minor change for return value
535a0b3
@mkovalski
Merge branch 'master' into profiler
9c84c23
@mkovalski
reformatted
1640a96
@mkovalski
Merge remote-tracking branch 'upstream/master' into profiler
8c51461
@mkovalski
Merge branch 'profiler' of github.com:mkovalski/python-aiplatform int…
b0594a4 
…o profiler
@mkovalski
Merge branch 'master' into profiler
fadcf9a
@mkovalski
type hints and changing mutable default
83a803e
@mkovalski
Merge branch 'profiler' of github.com:mkovalski/python-aiplatform int…
f71c41f 
…o profiler
@mkovalski
creating separate RunResourceManager for handling runs
49b2727
@mkovalski
Merge branch 'main' of https://github.com/googleapis/python-aiplatform …
17c3f8b 
…into profiler
@mkovalski
moving additional functionality to uploader_utils to be shared by pro…
833bbbb 
…file uploader
@mkovalski
initial profile uploader commit, still needs testing
75048d3
@mkovalski
Merge branch 'profiler' of github.com:mkovalski/python-aiplatform int…
b7ba730 
…o profiler_impl
@mkovalski
abstract methods for call, move items to uploader_utils
400de49
@mkovalski
adding test objects for profiler
4bc3912
@mkovalski
merging from main
097fa34
@mkovalski
have mock client create one platform names for testing
17be71a
@mkovalski
additional tets for profile events
9cd47b6
@mkovalski
more tests
f3c7032
@mkovalski
100% coverage on profiler
9907578
@mkovalski
docstrings and typing
89fcce0
@mkovalski
Merge remote-tracking branch 'upstream/main' into profiler_impl
0107c54
@product-auto-label product-auto-label bot added the api: aiplatform Issues related to the AI Platform API. label Sep 14, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 14, 2021
@mkovalski mkovalski changed the title Add one shot profile uploads to tensorboard uploader. feat: Add one shot profile uploads to tensorboard uploader. Sep 14, 2021
@mkovalski mkovalski requested a review from yfang1 September 20, 2021 16:38
@mkovalski mkovalski self-assigned this Sep 21, 2021
@yfang1 yfang1 requested a review from morgandu September 21, 2021 18:52
yfang1
yfang1 approved these changes Sep 21, 2021
View reviewed changes
google/cloud/aiplatform/tensorboard/uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Outdated Show resolved Hide resolved
@yfang1 yfang1 self-requested a review September 21, 2021 19:14
@mkovalski mkovalski mentioned this pull request Sep 29, 2021
Closed
@mkovalski mkovalski requested review from sasha-gitg and removed request for morgandu and vinnysenthil September 30, 2021 13:31
sasha-gitg
sasha-gitg requested changes Oct 5, 2021
View reviewed changes
google/cloud/aiplatform/tensorboard/plugins/tf_profiler/profile_uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/plugins/tf_profiler/profile_uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/plugins/tf_profiler/profile_uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/plugins/tf_profiler/profile_uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/plugins/tf_profiler/profile_uploader.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Outdated Show resolved Hide resolved
@mkovalski mkovalski requested a review from sasha-gitg October 6, 2021 18:55
sasha-gitg
sasha-gitg reviewed Oct 7, 2021
View reviewed changes
Copy link
Member

@sasha-gitg sasha-gitg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly LGTM. Added one more comment.

google/cloud/aiplatform/tensorboard/__init__.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/tensorboard/uploader_utils.py Outdated Show resolved Hide resolved
@mkovalski mkovalski requested a review from sasha-gitg October 8, 2021 11:48
@sasha-gitg sasha-gitg added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 8, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 8, 2021
@mkovalski mkovalski added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 11, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 11, 2021
@sasha-gitg sasha-gitg merged commit a83f253 into googleapis:main Oct 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yfang1 yfang1 yfang1 approved these changes

@sasha-gitg sasha-gitg sasha-gitg approved these changes

Assignees

@mkovalski mkovalski

Labels
api: aiplatform Issues related to the AI Platform API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add remote tensorflow profiling to training jobs.
4 participants
@mkovalski @sasha-gitg @yfang1 @yoshi-kokoro