Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs on the use of heterogeneous test / val manifests #6352

Merged
merged 2 commits into from
Apr 3, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/source/asr/configs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,25 @@ An example ASR train and validation configuration should look similar to the fol
num_workers: 8
pin_memory: true

There are two ways to test/validate on more than one manifest:

- Specify a list in the `manifest_filepath` field. Results will be reported for each, the first one being used for overall loss / WER (specify `val_dl_idx` if you wish to change that). In this case, all manifests will share configuration parameters.
- Use the ds_item key and pass a list of config objects to it. This allows you to use differently configured datasets for validation, e.g.

.. code-block:: yaml

model:
validation_ds:
ds_item:
- name: dataset1
manifest_filepath: ???
# Config parameters for dataset1
...
- name: dataset2
manifest_filepath: ???
# Config parameters for dataset2
...

By default, dataloaders are set up when the model is instantiated. However, dataloader setup can be deferred to
model's `setup()` method by setting ``defer_setup`` in the configuration.

Expand Down