[Bug] query-comments aren't appended when a Manifest is passed to dbtRunner

[racheldaniel]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The IDE uses the dbtRunner as an entry point for dbt invocations. One common flow is to get the manifest obj returned after calling a parse as a part of the dbtRunnerResult, then pass it back to the runner obj for subsequent commands until the user makes a change to their project. We have observed that passing a Manifest object to the dbtRunner results in the removal of expected query-comments. We confirmed the comments are in the sql as expected as long as dbtRunner is instantiated without a cached manifest.

Expected Behavior

If a query-comment is defined, sql returned from dbt invocations could contain those comments regardless of how the dbtRunner is invoked

Steps To Reproduce

Here are reliable repro steps from a python repl (this is using my locally installed dbt 1.7.7).

1. Add the following to a local dbt project's dbt_project.yml:

query-comment:
  comment: "comment {{ dbt_version }}"
  append: true

2. Start a python repl on a dbt project in a venv with dbt installed and import the dbtRunner

(dbt-cloud-env) racheldaniel:jaffle_shop/ (master✗) $ python3
[12:40:18] Python 3.8.10 (default, Nov 29 2023, 10:42:27)
>>> from dbt.cli.main import dbtRunner

3. Create an instance without a manifest and run a build

>>> dbt = dbtRunner()
>>> dbt.invoke(["--log-format=json", "--log-level=debug", "build", "--select", "stg_customers"])

4. Search for your comment in the logs, it should be present (log truncated):

{"data": {"conn_name": "model.jaffle_shop.stg_customers", "node_info": {"materialized": "view", ... "sql": "create or replace  ... nselect * from renamed\n  )\n/* comment 1.7.7 */;", "name": "SQLQuery", "pid": 34081, "thread": "Thread-1", "ts": "2024-02-08T18:43:37.207812Z"}}

5. Run a parse and store the result:

>>> result = dbt.invoke(["--log-format=json", "--log-level=debug", "parse"])

6. Reinstantiate the dbtRunner using the manifest in that result, then call the same build:

>>> dbt = dbtRunner(manifest=result.result)
>>> dbt.invoke(["--log-format=json", "--log-level=debug", "build", "--select", "stg_customers"])

7. Observe the query comment is now missing (log truncated):

{"data": {"conn_name": "model.jaffle_shop.stg_customers", "node_info": {"materialized": "view", ... "sql": "create or replace   view DEVELOPMENT.dbt_dev_rdaniel_tests.stg_customers\n  \n   as (\n    with source as (\n    select * from DEVELOPMENT.dbt_dev_rdaniel_tests.raw_customers\n\n),\n\nrenamed as (\n\n    select\n        id as customer_id,\n        first_name,\n        last_name\n\n    from source\n\n)\n\nselect * from renamed\n  );"}...}

Relevant log output

No response

Environment

- OS: Sonoma 14.1
- Python: 3.8.10
- dbt: 1.7.7 (observed behavior in 1.6.0+, did not test with dbtRunner in 1.5 as IDE uses rpc for <=1.5)

Which database adapter are you using with dbt?

snowflake

Additional Context

Experience team ticket with historical context -- https://dbtlabs.atlassian.net/browse/XP-2671

[dbeatty10]

Thanks for such a nice write-up @racheldaniel -- it allowed me to reproduce the same thing that you are seeing.

[davidharting]

Amazing, thank you @gshank!