Skip to content

Conversation

romesco
Copy link
Contributor

@romesco romesco commented Dec 19, 2020

Adds configs for Dataset variants and Sampler variants.

Adds tests for all configs except:
TensorDatasetConf and DistributedSamplerConf.

The former has a funny issue with instantiation due to the asterisk in it's signature which I'm investigating. The latter requires setting up a distributed env to test instantiation. Not sure about the cleanest way to automate this test yet.

Addresses: #1, #45

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 19, 2020
@romesco romesco changed the title torch1.6 data configs [hydra-configs-torch] torch1.6 data configs Dec 19, 2020
@romesco
Copy link
Contributor Author

romesco commented Dec 22, 2020

One other change in this PR:

I've moved the configen source confs to include separate folders per project.

configen/conf/configen.yaml

->

configen/conf/torch/configen.yaml
configen/conf/torchvision/configen.yaml

It might be better if these were just torch.yaml and torchvision.yaml, but that's not currently supported in configen. Lower priority, but I will change it if I have time.

@romesco romesco marked this pull request as ready for review December 22, 2020 00:06
@romesco romesco requested a review from omry December 22, 2020 00:55
Copy link
Contributor

@omry omry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.
There was also the problem with the missing type annotation you encountered, right?
can you also file an issue for that one?

data.IterableDataset(),
id="IterableDatasetConf",
),
# TODO: investigate asterisk in signature instantiation limitation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you file an issue on the repo so we don't lost track?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You saw the rest of the issues, but here:
facebookresearch/hydra#1243

Will get the minimal repro for this one. The others, once we nail them down, we can get minimal repros for those as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a chance that this will one will be difficult to solve. Let's assume we need a workaround.

@romesco romesco merged commit eeabcad into master Dec 25, 2020
@romesco romesco deleted the data_1.6 branch December 25, 2020 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[hydra-configs-torch][tests] Dataset and Sampler [hydra-configs-torch] Configs for Datasets and Samplers

3 participants