Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TF2 environment settings #5230

Merged
merged 33 commits into from
Feb 17, 2020
Merged

TF2 environment settings #5230

merged 33 commits into from
Feb 17, 2020

Conversation

dakshvar22
Copy link
Contributor

@dakshvar22 dakshvar22 commented Feb 12, 2020

Proposed changes:

  • Refactor how tensorflow enviornment is configured for tensorflow 2.0
  • Add environment variables, example shown below:
TF_GPU_MEMORY_ALLOC="0:1024, 1:2048" # This specifies to use 1024 MB of memory from GPU with logical ID 0 and 2048 MB of memory from GPU with logical ID 1"

TF_INTER_OP_PARALLELISM_THREADS="3" # Specifies that atmost 3 CPU threads can be used to parallelize multiple non-blocking operations

TF_INTRA_OP_PARALLELISM_THREADS="2" # Specifies that atmost 2 CPU threads can be used to parallelize a particular operation.

part of https://github.com/RasaHQ/research/issues/62

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

Copy link
Contributor

@Ghostvv Ghostvv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to document it, also there are old tf_config tests, that need to be removed (unless they're already removed)

rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for also writing comprehensive docs for this 🚀

docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Show resolved Hide resolved
tests/utils/test_tf_environment.py Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
@dakshvar22 dakshvar22 requested review from Ghostvv and wochinge and removed request for Ghostvv February 13, 2020 16:56
changelog/5230.feature.rst Outdated Show resolved Hide resolved
changelog/5230.feature.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
docs/api/tensorflow_usage.rst Show resolved Hide resolved
docs/api/tensorflow_usage.rst Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Show resolved Hide resolved
tests/utils/test_tf_environment.py Outdated Show resolved Hide resolved
tests/utils/test_tf_environment.py Outdated Show resolved Hide resolved
Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two more comments, but we are ready to go now. Feel free to merge after checking the comments! 🥇

rasa/utils/tensorflow/environment.py Show resolved Hide resolved
rasa/utils/tensorflow/environment.py Outdated Show resolved Hide resolved
tests/utils/tensorflow/test_tf_environment.py Outdated Show resolved Hide resolved
@dakshvar22 dakshvar22 merged commit 6ca2415 into tf2 Feb 17, 2020
@tmbo tmbo deleted the tf2-tfconfig branch May 1, 2020 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants