Add support for dbt selector arg for DAG parsing

[jbandoro]

Description

This PR adds support for a new selector arg in RenderConfig so users can reference dbt yaml selectors when rendering a project using dbt ls by adding the --selector arg to the cli command. This was pretty straightforward for dbt ls, though in order to support custom and manifest filtering we would need to add logic for translating the yaml selector definitions in follow up PR(s).

Added a coverage test and integration test here, and a small update in tests/dbt/test_graph.py to use a fixture for a ProfileConfig that is repeated 9 times.

An open question I have is whether we want to log a warning in RenderConfig.__post_init__ if a user uses both select/exclude with selector. dbt docs here state that when using --selector with --exclude and/or --select the select/exclude flags are ignored (no error or warnings are logged by dbt if both are used).

Related Issue(s)

Closes #718

Breaking Change?

None

Checklist

• I have made corresponding changes to the documentation (if required)
• I have added tests that prove my fix is effective or that my feature works

[netlify]

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name	Link
🔨 Latest commit	a0bd087
🔍 Latest deploy log	https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/6579d2ba5e4196000886c341

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (4369987) 93.18% compared to head (a0bd087) 93.20%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #755      +/-   ##
==========================================
+ Coverage   93.18%   93.20%   +0.01%     
==========================================
  Files          55       55              
  Lines        2451     2458       +7     
==========================================
+ Hits         2284     2291       +7     
  Misses        167      167              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tatiana]

Hi @jbandoro, as usual, great work!

An open question is whether we want to log a warning in RenderConfig.post_init if a user uses both select/exclude with selector. dbt docs here state that when using --selector with --exclude and --select, the select/exclude flags are ignored (no error or warnings are logged by dbt if both are used).

I like the way you developed the feature. While checking for this in Cosmos would make users aware of the dbt limitation, if dbt decides to support both types of selectors simultaneously - we would have to change Cosmos to keep this.

The one check I think we need to add is to explicitly raise an exception if users try to use selector with a load mode that does not support it.

Do you think we should log a follow-up ticket to support this in the other load modes?

[jbandoro]

The one check I think we need to add is to explicitly raise an exception if users try to use selector with a load mode that does not support it.

Good point, I think that should be done in DbtGraph incase RenderConfig.load_method = LoadMode.AUTOMATIC. Will update.

Do you think we should log a follow-up ticket to support this in the other load modes?

Yes definitely, I can follow-up and create one after #718 is closed.

[tatiana]

Looks great, thanks, @jbandoro for adding this feature. Once the checks pass, we can merge this PR.

[jbandoro]

@tatiana added a follow-up issue for support manifest and custom load modes with the selector arg: #767