Skip to content

Commit

Permalink
Merge branch 'current' into mirnawong1-patch-26
Browse files Browse the repository at this point in the history
  • Loading branch information
@mirnawong1
mirnawong1 authored Apr 5, 2024
2 parents fa51f69 + b895675 commit 991cad6
Show file tree
Hide file tree
Showing 8 changed files with 77 additions and 28 deletions.
12 changes: 6 additions & 6 deletions website/docs/docs/build/dimensions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,10 @@ You can use multiple time groups in separate metrics. For example, the `users_cr

```bash
# dbt Cloud users
dbt sl query --metrics users_created,users_deleted --dimensions metric_time --order metric_time
dbt sl query --metrics users_created,users_deleted --group-by metric_time__year --order-by metric_time__year
# dbt Core users
mf query --metrics users_created,users_deleted --dimensions metric_time --order metric_time
mf query --metrics users_created,users_deleted --group-by metric_time__year --order-by metric_time__year
```


Expand All @@ -133,10 +133,10 @@ MetricFlow enables metric aggregation during query time. For example, you can ag

```bash
# dbt Cloud users
dbt sl query --metrics messages_per_month --dimensions metric_time --order metric_time --time-granularity year
dbt sl query --metrics messages_per_month --group-by metric_time__year --order-by metric_time__year
# dbt Core users
mf query --metrics messages_per_month --dimensions metric_time --order metric_time --time-granularity year
mf query --metrics messages_per_month --group-by metric_time__year --order metric_time__year
```

```yaml
Expand Down Expand Up @@ -361,10 +361,10 @@ The following command or code represents how to return the count of transactions

```bash
# dbt Cloud users
dbt sl query --metrics transactions --dimensions metric_time__month,sales_person__tier --order metric_time__month --order sales_person__tier
dbt sl query --metrics transactions --group-by metric_time__month,sales_person__tier --order-by metric_time__month,sales_person__tier
# dbt Core users
mf query --metrics transactions --dimensions metric_time__month,sales_person__tier --order metric_time__month --order sales_person__tier
mf query --metrics transactions --group-by metric_time__month,sales_person__tier --order-by metric_time__month,sales_person__tier
```

Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/build/saved-queries.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ To define a saved query, refer to the following parameters:
</VersionBlock>

All metrics in a saved query need to use the same dimensions in the `group_by` or `where` clauses.
When using the `Dimension` object, prepend the semantic model name, for example `Dimension('user__ds')`

## Related docs

Expand Down
23 changes: 23 additions & 0 deletions website/docs/faqs/Docs/documenting-macros.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,26 @@ macros:
```
</File>
## Document a custom materialization
When you create a [custom materialization](/guides/create-new-materializations), dbt creates an associated macro with the following format:
```
materialization_{materialization_name}_{adapter}
```

To document a custom materialization, use the previously mentioned format to determine the associated macro name(s) to document.

<File name='macros/properties.yml'>

```yaml
version: 2

macros:
- name: materialization_my_materialization_name_default
description: A custom materialization to insert records into an append-only table and track when they were added.
- name: materialization_my_materialization_name_xyz
description: A custom materialization to insert records into an append-only table and track when they were added.
```
</File>
4 changes: 2 additions & 2 deletions website/docs/faqs/Troubleshooting/access-gdrive-credential.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ If you're seeing the below error when you try to query a dataset from a Google D
Access denied: BigQuery BigQuery: Permission denied while getting Drive credentials
```

Usually this errors indicates that you haven't granted the BigQuery service account access to the specific Google Drive document. If you're seeing this error, try giving the service account (client email seen [here](https://docs.getdbt.com/docs/dbt-cloud/cloud-configuring-dbt-cloud/connecting-your-database#connecting-to-bigquery)) you are using for your BigQuery connection in dbt Cloud, permission to your Google Drive or Google Sheet. You'll want to do this directly in your Google Document and click the 'share' button and enter the client email there.
Usually, this error indicates that you haven't granted the BigQuery service account access to the specific Google Drive document. If you're seeing this error, try giving the service account (client email seen [here](https://docs.getdbt.com/docs/dbt-cloud/cloud-configuring-dbt-cloud/connecting-your-database#connecting-to-bigquery)) you are using for your BigQuery connection in dbt Cloud, permission to your Google Drive or Google Sheet. You'll want to do this directly in your Google Document and click the 'share' button and enter the client email there.

If you are experiencing this error when using OAuth, and you have verified your access to the Google Sheet, you may need to grant permissions for gcloud to access Google Drive:

```
gcloud auth application-default login --scopes=openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/sqlservice.login,https://www.googleapis.com/auth/drive
gcloud auth application-default login --disable-quota-project
```
For more info see the [gcloud auth application-default documentation](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login)

Expand Down
49 changes: 33 additions & 16 deletions website/docs/reference/resource-configs/databricks-configs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,47 @@ When materializing a model as `table`, you may include several optional configs

<VersionBlock lastVersion="1.5">

| Option | Description | Required? | Example |
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|----------------|
| file_format | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`). | Optional | `delta` |
| location_root | The created table uses the specified directory to store its data. The table alias is appended to it. | Optional | `/mnt/root` |
| partition_by | Partition the created table by the specified columns. A directory is created for each partition. | Optional | `date_day` |
| liquid_clustered_by | Cluster the created table by the specified columns. Clustering method is based on [Delta's Liquid Clustering feature](https://docs.databricks.com/en/delta/clustering.html). Available since dbt-databricks 1.6.2. | Optional | `date_day` |
| clustered_by | Each partition in the created table will be split into a fixed number of buckets by the specified columns. | Optional | `country_code` |
| buckets | The number of buckets to create while clustering | Required if `clustered_by` is specified | `8` |
| Option | Description | Required? | Example |
|---------------------|------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|--------------------------|
| file_format | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`). | Optional | `delta` |
| location_root | The created table uses the specified directory to store its data. The table alias is appended to it. | Optional | `/mnt/root` |
| partition_by | Partition the created table by the specified columns. A directory is created for each partition. | Optional | `date_day` |
| clustered_by | Each partition in the created table will be split into a fixed number of buckets by the specified columns. | Optional | `country_code` |
| buckets | The number of buckets to create while clustering | Required if `clustered_by` is specified | `8` |
| tblproperties | [Tblproperties](https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-tblproperties.html) to be set on the created table | Optional | `{'this.is.my.key': 12}` |

</VersionBlock>

<VersionBlock firstVersion="1.6">


| Option | Description | Required? | Model Support | Example |
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|---------------|----------------|
| file_format | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`). | Optional | SQL, Python | `delta` |
| location_root | The created table uses the specified directory to store its data. The table alias is appended to it. | Optional | SQL, Python | `/mnt/root` |
| partition_by | Partition the created table by the specified columns. A directory is created for each partition. | Optional | SQL, Python | `date_day` |
| liquid_clustered_by | Cluster the created table by the specified columns. Clustering method is based on [Delta's Liquid Clustering feature](https://docs.databricks.com/en/delta/clustering.html). Available since dbt-databricks 1.6.2. | Optional | SQL | `date_day` |
| clustered_by | Each partition in the created table will be split into a fixed number of buckets by the specified columns. | Optional | SQL, Python | `country_code` |
| buckets | The number of buckets to create while clustering | Required if `clustered_by` is specified | SQL, Python | `8` |
| Option | Description | Required? | Model Support | Example |
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|---------------|--------------------------|
| file_format | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`). | Optional | SQL, Python | `delta` |
| location_root | The created table uses the specified directory to store its data. The table alias is appended to it. | Optional | SQL, Python | `/mnt/root` |
| partition_by | Partition the created table by the specified columns. A directory is created for each partition. | Optional | SQL, Python | `date_day` |
| liquid_clustered_by | Cluster the created table by the specified columns. Clustering method is based on [Delta's Liquid Clustering feature](https://docs.databricks.com/en/delta/clustering.html). Available since dbt-databricks 1.6.2. | Optional | SQL | `date_day` |
| clustered_by | Each partition in the created table will be split into a fixed number of buckets by the specified columns. | Optional | SQL, Python | `country_code` |
| buckets | The number of buckets to create while clustering | Required if `clustered_by` is specified | SQL, Python | `8` |
| tblproperties | [Tblproperties](https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-tblproperties.html) to be set on the created table | Optional | SQL | `{'this.is.my.key': 12}` |

</VersionBlock>

<VersionBlock firstVersion="1.7">


| Option | Description | Required? | Model Support | Example |
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|---------------|--------------------------|
| file_format | The file format to use when creating tables (`parquet`, `delta`, `hudi`, `csv`, `json`, `text`, `jdbc`, `orc`, `hive` or `libsvm`). | Optional | SQL, Python | `delta` |
| location_root | The created table uses the specified directory to store its data. The table alias is appended to it. | Optional | SQL, Python | `/mnt/root` |
| partition_by | Partition the created table by the specified columns. A directory is created for each partition. | Optional | SQL, Python | `date_day` |
| liquid_clustered_by | Cluster the created table by the specified columns. Clustering method is based on [Delta's Liquid Clustering feature](https://docs.databricks.com/en/delta/clustering.html). Available since dbt-databricks 1.6.2. | Optional | SQL | `date_day` |
| clustered_by | Each partition in the created table will be split into a fixed number of buckets by the specified columns. | Optional | SQL, Python | `country_code` |
| buckets | The number of buckets to create while clustering | Required if `clustered_by` is specified | SQL, Python | `8` |
| tblproperties | [Tblproperties](https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-tblproperties.html) to be set on the created table | Optional | SQL, Python* | `{'this.is.my.key': 12}` |

\* Beginning in 1.7.12, we have added tblproperties to Python models via an alter statement that runs after table creation.
We do not yet have a PySpark API to set tblproperties at table creation, so this feature is primarily to allow users to anotate their python-derived tables with tblproperties.
</VersionBlock>

## Incremental models
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/target_database.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ snapshots:
Leverage the [`generate_database_name` macro](/docs/build/custom-databases) to build snapshots in databases that follow the same naming behavior as your models.

Notes:
* This macro is not available when configuring from the `dbt_project.yml` file, so must be configured in a snapshot config block.
* This macro is not available when configuring from the `dbt_project.yml` file, so it must be configured in a snapshot config block.
* Consider whether this use-case is right for you, as downstream `refs` will select from the `dev` version of a snapshot, which can make it hard to validate models that depend on snapshots.

<File name='snapshots/orders_snaphot.sql'>
Expand Down
2 changes: 2 additions & 0 deletions website/docs/reference/resource-properties/constraints.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Constraints require the declaration and enforcement of a model [contract](/refer

Constraints may be defined for a single column, or at the model level for one or more columns. As a general rule, we recommend defining single-column constraints directly on those columns.

If you are defining multiple `primary_key` constraints for a single model, those _must_ be defined at the model level. Defining multiple `primary_key` constraints at the column level is not supported.

The structure of a constraint is:
- `type` (required): one of `not_null`, `unique`, `primary_key`, `foreign_key`, `check`, `custom`
- `expression`: Free text input to qualify the constraint. Required for certain constraint types, and optional for others.
Expand Down
Loading

0 comments on commit 991cad6

Please sign in to comment.