Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linting union_relation with SQLFluff gives an empty select #831

Open
1 of 5 tasks
martinshjung opened this issue Sep 7, 2023 · 2 comments · May be fixed by #832
Open
1 of 5 tasks

Linting union_relation with SQLFluff gives an empty select #831

martinshjung opened this issue Sep 7, 2023 · 2 comments · May be fixed by #832
Labels
bug Something isn't working triage

Comments

@martinshjung
Copy link

martinshjung commented Sep 7, 2023

Describe the bug

I think this is similar to the resolved issue with dbt_utils.star() here: #605. I have a model using union_relations, that references tables that haven't been built yet. I want to be able to run SQLFluff and lint my model, but I get Found unparsable section: 'select'.

Steps to reproduce

  1. Create two upstream models:
# model1.sql
SELECT 1 AS model1_column
# model2.sql
SELECT 1 AS model2_column
  1. Create the model that uses union_relation. Important you pass source_column_name=None, to trigger the parse error with SQLFluff:
model3.sql
{{
    dbt_utils.union_relations(
        relations=[
            ref('model1'),
            ref('model2'),
        ],
        source_column_name=None
    )
}}
  1. Without running any of these dbt models, run sqlfluff lint against model3.sql.

Expected results

I'd expect the compiled SQL to have * in the SELECT statement, similar to how it's handled with the dbt_utils.star macro.

Actual results

I don't get any text after the SELECT keyword, resulting in a parse error with SQLFluff.

Screenshots and log output

I get:

L:   1 | P:   1 |  PRS | Line 4, Position 13: Found unparsable section: 'select'
L:   1 | P:   1 |  PRS | Line 17, Position 13: Found unparsable section: 'select'

with the compiled SQL looking like this:

        (
            select

                

            from `test-warehouse`.`test`.`model1`

            
        )

        union all
        

        (
            select

                

            from `test-warehouse`.`test`.`model2`

            
        )

System information

The contents of your packages.yml file:

packages:
  - git: https://github.com/dbt-labs/dbt-codegen.git
    revision: e24e5dc5daca47ba350415a59790d6b741be246b
  - package: calogica/dbt_expectations
    version: 0.8.5
  - package: dbt-labs/dbt_project_evaluator
    version: 0.6.2
  - package: dbt-labs/dbt_utils
    version: 1.1.1

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.5.1
  - latest:    1.6.1 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 1.5.1 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Additional context

Using sqlfluff version 2.3.1, with the same version sqlfluff-templater-dbt.

Are you interested in contributing the fix?

A snippet like this in the default macro, maybe?

/* No columns from any of the relations.
   This star is only output during dbt compile, and exists to keep SQLFluff happy. */
{% if dbt_command == 'compile' and ordered_column_names|length == 0 %}
    *
{% endif %}

Not sure what integration tests would be appropriate, since it's for flags.WHICH == compile.

@martinshjung martinshjung added bug Something isn't working triage labels Sep 7, 2023
Copy link

github-actions bot commented Mar 6, 2024

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Mar 6, 2024
Copy link

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 13, 2024
@dbeatty10 dbeatty10 reopened this Jun 27, 2024
@dbeatty10 dbeatty10 removed the Stale label Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants