[Bug] `xcom_push` failing with Airflow 3.1.0

[jmcbroom]

Describe the bug
Using the operators returns an error in Airflow 3.1.0:

AttributeError: 'DbtBuildOperator' object has no attribute 'xcom_push'
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/sdk/execution_time/task_runner.py", line 920 in run
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/sdk/execution_time/task_runner.py", line 1302 in _execute_task
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/sdk/bases/operator.py", line 416 in wrapper
File "/home/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/operators/dbt.py", line 304 in execute
File "/home/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/operators/dbt.py", line 347 in xcom_push_dbt_results

this comment looks possibly relevant but i'm not sure.

i also found this issue

• '_DockerDecoratedOperator' object has no attribute 'xcom_push' apache/airflow#53077

we don't use the xcoms, so still easy enough to use with passing do_xcom_push=False.

Airflow

• Version: 3.1.0
• Provider: locally via Docker Compose

Dbt

• Version: dbt-core 1.10.3, dbt-snowflake 1.10.2

[tomasfarias]

Thanks for reporting this! I'll take a look at the links you have shared and try to reproduce.

[tomasfarias]

Okay, I've managed to reproduce the issue. Indeed a bunch of imports and whatnot have changed with airflow 3.1 and we are not compatible.

It seems like the comment you linked is very relevant. I'll work through it to add support for Airflow 3.1.

[tomasfarias]

Just a quick update from me:

• Basically, a bunch of import paths are now under the airflow.sdk module. This required usage of a compatibility module to continue supporting latest airflow 2.x. I will have to drop Python 3.9 support (which at this point is considered legacy) to help here.
• Moreover, the xcom_push method no longer exists in operators, like mentioned in the OP, so I had to update that to use the task instance directly.
• Finally, tests completely broke. We now have to use a bunch of new fixtures to get things to work like they used to.

All in all, I think I've got the necessary changes applied to the actual code to make things work based on the first two points. But I am struggling with test failures in CI pipeline, as mentioned in the third point.

Will continue trying to make CI pass and hopefully can get this shipped during this week.

[tomasfarias]

After much struggling, all our tests are passing on Airflow 2.10.4, 3.0.6, and 3.1.1. I'll release a new version later today or early tomorrow with the included patches.