Patch for `dbt show` for a model with a `struct` column containing a `json` datatype by matt-winkler · Pull Request #974 · dbt-labs/dbt-bigquery

matt-winkler · 2023-10-21T21:48:36Z

resolves #972

Problem

dbt show --select currently breaks when attempted on a json field because of this bigquery python client issue

Solution

Patch as suggested by @dbeatty10

# Override broken json deserializer for dbt show --inline
def _json_from_json(value, _):
    """NOOP string -> string coercion"""
    return json.loads(value)

bigquery._helpers._CELLDATA_FROM_JSON["JSON"] = _json_from_json

Checklist

I have read the contributing guide and understand what's expected of me
I have run this code in development and it appears to resolve the stated issue
This PR includes tests, or tests are not required/relevant for this PR
This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc)

dbeatty10 · 2023-10-23T14:16:00Z

@cla-bot check

cla-bot · 2023-10-23T14:16:06Z

The cla-bot has been summoned, and re-checked this pull request!

colin-rogers-dbt · 2023-10-23T16:22:53Z

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Patch for json inline --show


Looks like you generated a second changelog instead of updating this one, you can just delete this file

Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>

mikealfare

I left one minor comment that doesn't affect functionality. I'm providing an approval so that you can merge following that update.

…winkler/json-show-patch

dbeatty10 · 2023-10-23T17:35:53Z

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Patch for json inline --show


Suggested change

body: Patch for json inline --show

body: Patch for `dbt show` for `json` within a `struct`

…`json` datatype (#974) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458)

…`json` datatype (#974) (#976) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458) Co-authored-by: matt-winkler <75497565+matt-winkler@users.noreply.github.com>

…`json` datatype (#974) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458)

…`json` datatype (#974) (#978) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458) Co-authored-by: matt-winkler <75497565+matt-winkler@users.noreply.github.com>

…xes (#1719) While the initial x.x.0 versions of dbt adapters may have compatibility issues with certain dependencies, I observed that these incompatibilities are progressively addressed in later patch releases. The issue reported in #1709 appears to stem from dbt-bigquery accessing a protected member `google.cloud.bigquery._helpers._CELLDATA_FROM_JSON`, which was removed in `google-cloud-bigquery 3.31.0`. This access I believe was introduced in [dbt-bigquery#974](dbt-labs/dbt-bigquery#974) and later removed in [dbt-bigquery#1061](dbt-labs/dbt-bigquery#1061), with the fix subsequently back-ported to the 1.7 series via [dbt-bigquery#1074](dbt-labs/dbt-bigquery#1074). Therefore, I believe that relying on the latest patch versions is a better approach to avoid such issues, rather than individually resolving discrepancies. This PR aligns with that strategy. closes: #1709 --------- Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>

patch for --show with json

9b93327

matt-winkler requested a review from a team as a code owner October 21, 2023 21:48

matt-winkler requested a review from nathaniel-may October 21, 2023 21:48

cla-bot Bot added the cla:yes label Oct 21, 2023

dbeatty10 added the triage:ready-for-review Externally contributed PR has functional approval, ready for code review from Core engineering label Oct 23, 2023

dbeatty10 reviewed Oct 23, 2023

View reviewed changes

Comment thread .changes/unreleased/Fixes-20231021-154237.yaml Outdated

update matts handle in changelog

a5c9b8c

colin-rogers-dbt reviewed Oct 23, 2023

View reviewed changes

Comment thread dbt/adapters/bigquery/connections.py

colin-rogers-dbt reviewed Oct 23, 2023

View reviewed changes

matt-winkler and others added 3 commits October 23, 2023 10:27

Update dbt/adapters/bigquery/connections.py

e49cfc0

Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>

add json struct functional test

a32baf1

remove old change log

30f817e

mikealfare reviewed Oct 23, 2023

View reviewed changes

Comment thread tests/functional/adapter/dbt_show/test_dbt_show.py

mikealfare approved these changes Oct 23, 2023

View reviewed changes

dbeatty10 changed the title ~~patch for --show with json~~ Patch for dbt show for a model with a json column Oct 23, 2023

colin-rogers-dbt added 2 commits October 23, 2023 10:33

add comment to test

b01397e

Merge remote-tracking branch 'origin/mwinkler/json-show-patch' into m…

8b6bd15

…winkler/json-show-patch

dbeatty10 changed the title ~~Patch for dbt show for a model with a json column~~ Patch for dbt show for a model with a struct column containing a json datatype Oct 23, 2023

colin-rogers-dbt enabled auto-merge (squash) October 23, 2023 17:34

colin-rogers-dbt merged commit 6a3f458 into main Oct 23, 2023

colin-rogers-dbt deleted the mwinkler/json-show-patch branch October 23, 2023 17:35

dbeatty10 reviewed Oct 23, 2023

View reviewed changes

colin-rogers-dbt added the backport 1.6.latest label Oct 23, 2023

github-actions Bot mentioned this pull request Oct 23, 2023

[Backport 1.6.latest] Patch for dbt show for a model with a struct column containing a json datatype #976

Merged

colin-rogers-dbt added the backport 1.7.latest label Oct 24, 2023

github-actions Bot mentioned this pull request Oct 24, 2023

[Backport 1.7.latest] Patch for dbt show for a model with a struct column containing a json datatype #978

Merged

dbeatty10 mentioned this pull request Feb 16, 2024

[Bug] Error when loading NULL result from JSON column #1101

Closed

2 tasks

colin-rogers-dbt mentioned this pull request Apr 24, 2024

backport 1187 to 1.6.latest #1193

Merged

4 tasks

This was referenced Apr 28, 2025

[Bug] dbt-bigquery 1.6.13 incompatible with latest google-cloud-bigquery==3.31.0 dbt-labs/dbt-adapters#1031

Closed

Use latest minor versions for dbt adapters to get in compatibility fixes astronomer/astronomer-cosmos#1719

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype#974

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype#974
colin-rogers-dbt merged 7 commits into
mainfrom
mwinkler/json-show-patch

matt-winkler commented Oct 21, 2023 •

edited by dbeatty10

Loading

Uh oh!

dbeatty10 commented Oct 23, 2023

Uh oh!

cla-bot Bot commented Oct 23, 2023

Uh oh!

Uh oh!

Uh oh!

colin-rogers-dbt Oct 23, 2023

Uh oh!

Uh oh!

mikealfare left a comment

Uh oh!

dbeatty10 Oct 23, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -0,0 +1,6 @@
		kind: Fixes
		body: Patch for json inline --show

	body: Patch for json inline --show
	body: Patch for `dbt show` for `json` within a `struct`

Conversation

matt-winkler commented Oct 21, 2023 • edited by dbeatty10 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Checklist

Uh oh!

dbeatty10 commented Oct 23, 2023

Uh oh!

cla-bot Bot commented Oct 23, 2023

Uh oh!

Uh oh!

Uh oh!

colin-rogers-dbt Oct 23, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mikealfare left a comment

Choose a reason for hiding this comment

Uh oh!

dbeatty10 Oct 23, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

matt-winkler commented Oct 21, 2023 •

edited by dbeatty10

Loading