Add --exclude for source snapshot-freshness

[tayloramurphy]

Describe the feature

Add --exclude to source snapshot-freshness

Describe alternatives you've considered

Not having this breaks the syntax model that's expected for basically every other dbt selection command.

Additional context

I'm on Snowflake, but should work for all.

Who will this benefit?

Evidently very few people are doing this, but I can share how we're wanting to use it. Basically, we're isolating our source extractions into separate dags with the following dbt plan:

• source snaphot freshness
• source test
• snapshot source data
• run source-based models
• test source-based models

As we're migrating to this pattern I want to be able to exclude from our main freshness dag the models I've already broken out, but I can't.

Are you interested in contributing this feature?

I can if you point me in the right direction!

[beckjake]

Historical note: this goes back to the implementation of snapshot-freshness, it was never in the original spec. I can't remember if that was by design, a "we'll do it later", or just accidental.

I think fixing this will require edits in two places:

• in main.py, add a --exclude argument to the source freshness argument parser. Probably just copy+paste it?
• in task/freshness.py, update the get_selection_spec method to calculate exclude (I assume the same way as include, but with a default of [] instead of ['*']). Then pass the exclude along to parse_difference instead of the current None.

It's possible this is more complicated than I'm thinking, but I don't see why that would be.

@jtcohen6 we should decide if source freshness deserves to get selectors, too - right now that's also not wired up (I think because --exclude was missing, I decided not to mess with it)

[jtcohen6]

Thanks for opening this, @tayloramurphy. Even if it isn't a frequent need, I'm definitely in favor of bringing dbt source snapshot-freshness up to parity with dbt seed and dbt snapshot. It already supports --select; it should support --exclude with the same syntax.

@beckjake I think that feature parity with seed and snapshot means adding the rest of node selection support, too. In terms of new functionality that's actually usable, I guess that means supporting the tag:, package:, state: methods and YAML--selector. Intersections, parent/child depth operators and path:, test_type:, test_name:, config: methods aren't really relevant for sources today. That doesn't have to come in this issue/PR.

Side note: I know source configs today only contain enabled. Someday, it might be cool to do stuff like:

dbt source snapshot-freshness --select config.loader:fivetran

[joellabes]

This just bit us - after posting this message about excluding a test provided by a Fivetran package which checks a table we aren't syncing, I thought I could maybe do dbt source snapshot-freshness --exclude email_event as a workaround but no joy.

[stujo]

👍 Would find this helpful

[jtcohen6]

Closing in favor of #2987, to be resolved by #3554