Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unit Testing] Support primative types + objects and arrays in dbt-bigquery #1090

Closed
3 tasks done
MichelleArk opened this issue Feb 5, 2024 · 2 comments · Fixed by #1031
Closed
3 tasks done

[Unit Testing] Support primative types + objects and arrays in dbt-bigquery #1090

MichelleArk opened this issue Feb 5, 2024 · 2 comments · Fixed by #1031
Assignees
Labels
enhancement New feature or request

Comments

@MichelleArk
Copy link
Contributor

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Currently, the unit testing framework in core uses the safe_cast utils macro to convert a user-provided value for an input mock to the appropriate type. In some instances, this safe_cast may fail and need special handling depending on the type.

The acceptance criteria for this issue is to introduce a test that provides coverage across any non-STRUCT input types for unit testing input data, potentially updating the safe_cast macro as necessary.

Describe alternatives you've considered

No response

Who will this benefit?

No response

Are you interested in contributing this feature?

yes

Anything else?

STRUCT support is out of scope for this initial leg of work.

@MichelleArk MichelleArk added enhancement New feature or request triage labels Feb 5, 2024
@MichelleArk MichelleArk changed the title [Unit Testing] Support primative types + objects and arrays in dbt-snowflake [Unit Testing] Support primative types + objects and arrays in dbt-bigquery Feb 5, 2024
@MichelleArk MichelleArk self-assigned this Feb 5, 2024
@dbeatty10 dbeatty10 removed the triage label Feb 7, 2024
@github-christophe-oudar
Copy link
Contributor

Hey @MichelleArk, is there a plan to support UDFs as well? I tried it and it doesn't work. It fails like that:

 Traceback (most recent call last):
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 376, in safe_run
    result = self.compile_and_execute(manifest, ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 322, in compile_and_execute
    result = self.run(ctx.node, manifest)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 423, in run
    return self.execute(compiled_node, manifest)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 251, in execute
    unit_test_result = self.execute_unit_test(test, manifest)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 182, in execute_unit_test
    unit_test_manifest = self.build_unit_test_manifest_from_test(unit_test_def, manifest)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 176, in build_unit_test_manifest_from_test
    return loader.load()
           ^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/parser/unit_tests.py", line 54, in load
    self.parse_unit_test_case(unit_test_case)
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/parser/unit_tests.py", line 98, in parse_unit_test_case
    get_rendered(unit_test_node.raw_code, ctx, unit_test_node, capture_macros=True)
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/clients/jinja.py", line 146, in get_rendered
    rendered = render_template(template, ctx, node)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 540, in render_template
    return template.render(ctx)
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 145, in render
    return self.environment.handle_exception()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
TypeError: sequence item 0: expected str instance, Undefined found

I couldn't find an opened issue for that need. Do you want me to make one?

@github-christophe-oudar
Copy link
Contributor

github-christophe-oudar commented Apr 19, 2024

☝️ cc @colin-rogers-dbt (to confirm your team has on the radar 👀 )
Edit: dbt-labs/dbt-core#9775

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants