Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dbt): dbt test description extract #19464

Merged
merged 1 commit into from
Jan 31, 2024

Conversation

CapitanHeMo
Copy link
Contributor

@CapitanHeMo CapitanHeMo commented Jan 29, 2024

Summary & Motivation

According to the problem Adding descriptions to tests is no longer possible from 0.20.0 and the source code SchemaGenericTestParser the general test does not support the description in the schema.

This fix allows users to define a test description in tests/<generic test>/config/meta/description, to display data in Dagster(UI) Asset Check.

models:
  - name: wildberries_incomes
    tests:
      - name: 'Equal_row_count_with_stg__wildberries_incomes'
        config: 
          meta:
            description: "Equal rows before join with mapping table"
        test_name: equal_rowcount
        compare_model: source('production', 'stg__wildberries_incomes')
        compare_model_final: True

Related: dbt-labs/dbt-core#2578

Снимок экрана 2024-01-29 в 14 57 45

How I Tested These Changes

pytest

@CapitanHeMo CapitanHeMo marked this pull request as ready for review January 29, 2024 12:00
Copy link
Contributor

Current dependencies on/for this PR:

This stack of pull requests is managed by Graphite.

@rexledesma
Copy link
Contributor

Thanks for the contribution! I've added tests to this and hoping to merge this soon.

@@ -246,7 +235,7 @@ def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
def test_materialize_no_selection(dbt_commands: List[List[str]]) -> None:
result = _materialize_dbt_assets(dbt_commands, selection=None, raise_on_error=False)
assert not result.success # fail_tests_model fails
assert len(result.get_asset_materialization_events()) == 9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seemed wrong to begin with, since there are 10 expected materializations (4 seeds + 6 models). Updating the manifest resulted in this error, so fixing it.

@rexledesma rexledesma merged commit 5c13175 into dagster-io:master Jan 31, 2024
1 check passed
OwenKephart pushed a commit that referenced this pull request Jan 31, 2024
## Summary & Motivation
According to the problem [Adding descriptions to tests is no longer
possible from 0.20.0](dbt-labs/dbt-core#3599 )
and the source code
[SchemaGenericTestParser](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/parser/schema_generic_tests.py#L39)
the general test does not support the description in the schema.

This fix allows users to define a test description in `tests/<generic
test>/config/meta/description`, to display data in Dagster(UI) Asset
Check.

```yaml
models:
  - name: wildberries_incomes
    tests:
      - name: 'Equal_row_count_with_stg__wildberries_incomes'
        config: 
          meta:
            description: "Equal rows before join with mapping table"
        test_name: equal_rowcount
        compare_model: source('production', 'stg__wildberries_incomes')
        compare_model_final: True
```

Related: dbt-labs/dbt-core#2578


![Снимок экрана 2024-01-29 в 14 57
45](https://github.com/dagster-io/dagster/assets/34104577/2638a04c-935f-4368-b21b-04c5de740614)


## How I Tested These Changes
pytest

Co-authored-by: Rex Ledesma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants