[CT-997] [Feature] Support dbt.var + dbt.env_var in Python models

[jtcohen6]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Copying from thread in #beta-feedback-python-models:

A quick thought, why not expose a dbt.var function (and similar for env var) rather than overloading config? If you apply the same limitations as for config function (ie. Only literal args) should be pretty straightforward to statically analyse which vars need to be injected?

It's true! The mechanism for dbt.var and dbt.env_var could be just the same as dbt.config.get. We detect the literal arguments passed in:

dbt-core/core/dbt/parser/models.py

Lines 113 to 117 in eb72dbf

	if func_name in ["dbt.ref", "dbt.source", "dbt.config", "dbt.config.get"]:
	# drop the dot-dbt prefix
	func_name = func_name.split(".")[-1]
	args, kwargs = self._get_call_literals(node)
	self.dbt_function_calls.append((func_name, args, kwargs))

dbt-core/core/dbt/parser/models.py

Lines 182 to 184 in eb72dbf

	if func == "get":
	context["config"](utilized=args)
	continue

And then provide those values at "compile" time (which we're currently doing via Jinja... gross), as new methods (var + env_var) on the dbtObj class:

dbt-core/core/dbt/include/global_project/macros/python_model/python.sql

Lines 33 to 40 in eb72dbf

	{%- for key in model.config.utilized -%}
	{# TODO: weird type testing with enum, would be much easier to write this logic in Python! #}
	{%- if key == 'language' -%}
	{%- set value = 'python' -%}
	{%- endif -%}
	{%- set value = model.config[key] -%}
	{%- do config_dict.update({key: value}) -%}
	{%- endfor -%}

dbt-core/core/dbt/include/global_project/macros/python_model/python.sql

Lines 71 to 77 in eb72dbf

	class dbtObj:
	def __init__(self, load_df_function) -> None:
	self.source = lambda x: source(x, dbt_load_df_function=load_df_function)
	self.ref = lambda x: ref(x, dbt_load_df_function=load_df_function)
	self.config = config
	self.this = this()
	self.is_incremental = {{ is_incremental() }}

Describe alternatives you've considered

Not doing this, and continuing to ask users (as we are currently) to pass those vars + env_vars in as configs in .yml files:

version: 2

models:
  - name: my_python_model
    config:
      materialized: table
      target_name: "{{ target.name }}"
      specific_var: "{{ var('SPECIFIC_VAR') }}"
      specific_env_var: "{{ env_var('SPECIFIC_ENV_VAR') }}"

Who will this benefit?

Usability and ergonomics for people developing and configuring Python models, especially ones with more advanced conditional logic or env-specific behavior

Are you interested in contributing this feature?

No response

Anything else?

To be clear, the environment variable values would need to be the ones evaluated during dbt parsing. (They are NOT the environment variables set in the runtime environment, e.g. Snowpark.)

That's slightly different from how environment variables work in SQL models today, since the {{ env_var() }} Jinja is re-rendered at compile/execute runtime.

But I think it probably should work this way across the board, where all inputs (including env vars) are resolved during parsing and pre-execution.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[devmessias]

I believe this relates to #10914