Remove --dry-run flag from dbt deps

[jtcohen6]

resolves #9100

Problem

In theory, dbt deps --add-package --dry-run only updates packages.yml with the package entry being added, without actually resolving packages, writing package-lock.yml, or installing packages.

It took me a while to understand this workflow:

# add a package to *just* packages.yml / dependencies.yml
$ dbt deps --add-package dbt-labs/[email protected] --dry-run
# now do 'package resolution' and update lock file
$ dbt deps --lock
# now actually install packages, using lock file
$ dbt deps

It doesn't seem to work in practice. Now that I've understood the workflow, I understand why we added this flag — but I don't think it's essential that we support it. I'd rather remove the flag entirely than have it hanging out as extra weight. It's already possible to achieve what is (IMO) the important part of this workflow with dbt deps --lock --add-package ....

Solution

Remove the dbt deps --dry-run flag.

Checklist

• I have read the contributing guide and understand what's expected of me
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
• This PR includes type annotations for new and modified functions

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1740df5) 86.71% compared to head (fc6538d) 85.39%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9169      +/-   ##
==========================================
- Coverage   86.71%   85.39%   -1.32%     
==========================================
  Files         179      179              
  Lines       26652    26648       -4     
==========================================
- Hits        23110    22757     -353     
- Misses       3542     3891     +349     

Flag	Coverage Δ
integration	82.08% <100.00%> (-1.58%)	⬇️
unit	65.10% <0.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[seub]

Thanks @jtcohen6 !

Now that I've understood the workflow, I understand why we added this flag

I don't, but maybe I'm missing something. Since --dry-run can only be used with --add-package, whatever effect it has can be the effect of using --lock with --add-package. Right?

Anyway I think that's what your PR is saying, it looks good to me 👍

[jtcohen6]

Since --dry-run can only be used with --add-package, whatever effect it has can be the effect of using --lock with --add-package. Right?

@seub Technically, --dry-run will update only packages.yml/dependencies.yml, whereas --lock will also update package-lock.yml. So they are distinct, but not in a way that I think is important or particularly useful.

[dbeatty10]

Technically, --dry-run will update only packages.yml/dependencies.yml, whereas --lock will also update package-lock.yml.

Did --dry-run work for you @jtcohen6?

With dbt-core 1.7.3, --dry-run still adds/updates the lock file for me:

• Log output includes Updating lock file in file path:
• package-lock.yml is created if it doesn't exist, and it is updated otherwise

Log output

$ dbt deps --add-package dbt-labs/[email protected] --dry-run

14:32:17  Running with dbt=1.7.3
14:32:17  Found duplicate package in packages.yml, removing: {'version': '1.0.0', 'package': 'dbt-labs/dbt_utils'}
14:32:17  Added new package dbt-labs/[email protected] to /dbt-core-9076/packages.yml
14:32:17  Updating lock file in file path: /dbt-core-9076/package-lock.yml

Benefits of this PR

So I think there are two different issues that this PR resolves:

1. dbt deps --dry-run doesn't work as originally intended; it behaves no differently than --lock currently
2. dbt deps --lock is sufficient to add a new dependency without actually installing the dependency

It also avoids multiple types of confusion:

• Removing --dry-run avoids potential confusion with other "dry run" type functionality in dbt (e.g. #8971 or #7839 or the --dry flag that was removed in 0.7.0).
• My personal expectation is that dry runs don't do any modifications, so removing this flag will also avoid the non-intuitive behavior that --dry-run is actually modifying the packages.yml /dependencies.yml file.

Trade-offs

By removing dbt deps --dry-run ..., users will get an error message that says "dbt.cli.exceptions.DbtUsageException: No such option: --dry-run".

There is no command that adds to packages.yml without also locking. The workaround will be to add the dependency to the file manually. If this turns out to be a desirable feature, then we can always add a new flag in the future.

[dbeatty10]

Looks good! Full rationale for supporting the details in this PR are here.

Not blocking suggestion:

• so that we can completely excise "dry_run", my only suggestion would be to change the name of the test_deps_add_dry_run test to be test_deps_add_without_install instead.

[jtcohen6]

Thank you @dbeatty10 !

[FishtownBuildBot]

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#4622