TF2 environment settings

docs/api/tensorflow_usage.rst

@@ -0,0 +1,58 @@
+:desc: Find out how to configure your environment for efficient usage of tensorflow inside Rasa
+
+.. _tensorflow_usage:
+
+Setting up Tensorflow Runtime
+=============================
+
+Tensorflow allows setting the runtime environment via
+`TF Config submodule <https://www.tensorflow.org/api_docs/python/tf/config>`_. Rasa supports a smaller subset of these
+configuration options and makes appropriate calls to the `tf.config` submodule.
+This smaller subset comprises of configurations that developers frequently use with Rasa.
+All configuration options are specified using environment variables as shown in subsequent sections.
+
+
+Optimize CPU Performance
+------------------------
+
+Parallelize one operation
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set ``TF_INTRA_OP_PARALLELISM_THREADS`` as an environment variable to specify maximum number of threads that can be used
+to parallelize the execution of one operation. If left unspecified, this value defaults to 0 which means tensorflow should
+pick an appropriate value depending on the system configuration.
+
+
+Parallelize multiple operations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set ``TF_INTER_OP_PARALLELISM_THREADS`` as an environment variable to specify maximum number of threads that can be used
+to parallelize the execution of multiple **non-blocking** operations. If left unspecified, this value defaults to 0
+which means tensorflow should pick an appropriate value depending on the system configuration.
+
+
+Optimize GPU Performance
+------------------------
+
+Limiting GPU memory growth
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Tensorflow by default blocks all the available GPU memory for the running process. This can be limiting if you are running
+multiple tensorflow processes and want to distribute memory across them. To prevent this,
+set an environment variable ``TF_FORCE_GPU_ALLOW_GROWTH`` to ``True``.
+
+
+Restricting absolute GPU memory available
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Often, a developer wants to limit the absolute amount of GPU memory that can be used by a process.
+
+For example, you may have two visible GPUs(GPU:0 and GPU:1) and you want to allocate 1024 MB from first GPU and 2048 MB from second GPU.
+You can do so by setting an environment variable as ``TF_GPU_MEMORY_ALLOC="0:1024, 1:2048"``.
+
+Another scenario can be where you have access to 2 GPUs(GPU:0 and GPU:1) but you would like to use only second GPU for
+Rasa process.
+``TF_GPU_MEMORY_ALLOC="1:2048"`` would make 2048 MB of memory from GPU 1 available for the Rasa process
+
+
+

docs/index.rst

@@ -93,6 +93,7 @@ Understand messages, hold conversations, and connect to messaging channels and A
    api/lock-stores
    api/training-data-importers
    api/core-featurization
+   api/tensorflow_usage
    migration-guide
    changelog