Skip to content

Commit

Permalink
Merge branch 'mwong-update-build-docs' into update-doc-faq
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Jun 12, 2024
2 parents ee7f1e7 + 38da545 commit c83f526
Show file tree
Hide file tree
Showing 16 changed files with 126 additions and 57 deletions.
25 changes: 12 additions & 13 deletions website/docs/docs/build/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@ title: "Documentation"
description: "Learn how good documentation for your dbt models helps stakeholders discover and understand your datasets."
id: "documentation"
---
<p style={{ color: '#808080', fontSize: '1.1em' }}>
Good documentation for your dbt models will help downstream consumers discover and understand the datasets which you curate for them.

Good documentation for your dbt models will help downstream consumers discover and understand the datasets you curate for them.
dbt provides a way to generate documentation for your dbt project and render it as a website.
</p>

## Related documentation

Expand Down Expand Up @@ -60,13 +59,13 @@ models:

The default documentation experience in dbt Cloud is [dbt Explorer](/docs/collaborate/explore-projects), available on [Team or Enterprise plans](https://www.getdbt.com/pricing/). Use dbt Explorer to view your project's resources (such as models, tests, and metrics), its [metadata](/docs/collaborate/explore-projects#generate-metadata), and their lineage to gain a better understanding of its latest production state.

dbt Cloud developer and dbt Core users can use [dbt Docs](/docs/collaborate/build-and-view-your-docs#dbt-docs), a legacy feature that generates basic documentation, but it doesn't offer the same speed, metadata, or visibility as dbt Explorer.
dbt Cloud developer and dbt Core users can use [dbt Docs](/docs/collaborate/build-and-view-your-docs#dbt-docs), which generates basic documentation, but it doesn't offer the same speed, metadata, or visibility as dbt Explorer.

Generate documentation for you project by following these steps:

- First, run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files respectively.
- To view the documentation for all columns and not just columns described in your project, ensure that you have created the models with `dbt run` beforehand.
- If you're developing locally, run `dbt docs serve` to use these `.json` files to populate a local website.
1. Run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files, respectively.
2. Ensure that you have created the models with `dbt run` to view the documentation for all columns, not just those described in your project.
3. Run `dbt docs serve` if you're developing locally to use these `.json` files to populate a local website.

To view a resource, its metadata, and what commands are needed in dbt Explorer, refer to [generate metadata](/docs/collaborate/explore-projects#generate-metadata) for more details.

Expand Down Expand Up @@ -131,9 +130,9 @@ In the resulting documentation, `'{{ doc("table_events") }}'` will be expanded t


## Setting a custom overview
*Currently available for the legacy dbt Docs only.*
*Currently available for dbt Docs only.*

The "overview" shown in the legacy dbt Docs website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this:
The "overview" shown in the dbt Docs website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this:

<File name='models/overview.md'>

Expand All @@ -151,7 +150,7 @@ as well as the repo for this project \[here](https://github.com/dbt-labs/mrr-pla
</File>

### Custom project-level overviews
*Currently available for the legacy dbt Docs only.*
*Currently available for dbt Docs only.*

You can set different overviews for each dbt project/package included in your documentation site
by creating a docs block named `__[project_name]__`. For example, in order to define
Expand Down Expand Up @@ -186,13 +185,13 @@ For additional details on how to explore your lineage and navigate your resource
<Lightbox src="/img/docs/collaborate/dbt-explorer/example-model-details.png" width="100%" title="Example of resource details" />

<Expandable alt_header="For dbt Docs">
If you're using the legacy dbt Docs interface, you can navigate to the documentation for a specific model. That might look something like this:
If you're using the dbt Docs interface, you can navigate to the documentation for a specific model. That might look something like this:

<Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/f2221dc-Screen_Shot_2018-08-14_at_6.29.55_PM.png" title="Auto-generated documentation for a dbt model"/>

Here, you can see a representation of the project structure, a markdown description for a model, and a list of all of the columns (with documentation) in the model.

From a legacy dbt Docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring.
From the dbt Docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring.

<Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/ec77c45-Screen_Shot_2018-08-14_at_6.31.56_PM.png" title="Opening the DAG mini-map"/>

Expand All @@ -216,7 +215,7 @@ The `dbt docs serve` command is only intended for local/development hosting of t

dbt's documentation website was built to make it easy to host on the web. The site is "static,” meaning you don't need any "dynamic" servers to serve the docs. You can host your documentation in several ways:

* Use [dbt Cloud](/docs/collaborate/build-and-view-your-docs)'s default documentation experience with [dbt Explorer](/docs/collaborate/explore-projects).
* Use [dbt Cloud's](/docs/collaborate/build-and-view-your-docs) default documentation experience with [dbt Explorer](/docs/collaborate/explore-projects).
* Host on [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html) (optionally [with IP access restrictions](https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-3))
* Publish with [Netlify](https://discourse.getdbt.com/t/publishing-dbt-docs-to-netlify/121)
* Use your own web server like Apache/Nginx
Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/build/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ Env vars works fine with username/password and keypair, including scheduled jobs

However, there are some limitations when using env vars with Snowflake OAuth Connection settings:

- You can't use them in the account/host field, but they can be used for database, warehouse, and role.
- You can't use them in the account/host field, but they can be used for database, warehouse, and role. For these fields, [use extended attributes](/docs/deploy/deploy-environments#deployment-connection).

Something to note, if you supply an environment variable in the account/host field, Snowflake OAuth Connection will **fail** to connect. This happens because the field doesn't pass through Jinja rendering, so dbt Cloud simply passes the literal `env_var` code into a URL string like `{{ env_var("DBT_ACCOUNT_HOST_NAME") }}.snowflakecomputing.com`, which is an invalid hostname.
Something to note, if you supply an environment variable in the account/host field, Snowflake OAuth Connection will **fail** to connect. This happens because the field doesn't pass through Jinja rendering, so dbt Cloud simply passes the literal `env_var` code into a URL string like `{{ env_var("DBT_ACCOUNT_HOST_NAME") }}.snowflakecomputing.com`, which is an invalid hostname. Use [extended attributes](/docs/deploy/deploy-environments#deployment-credentials) instead.
:::

#### Audit your run metadata
Expand Down
18 changes: 16 additions & 2 deletions website/docs/docs/cloud/connect-data-platform/connect-snowflake.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ The following fields are required when creating a Snowflake connection

**Note:** A crucial part of working with dbt atop Snowflake is ensuring that users (in development environments) and/or service accounts (in deployment to production environments) have the correct permissions to take actions on Snowflake! Here is documentation of some [example permissions to configure Snowflake access](/reference/database-permissions/snowflake-permissions).

## Authentication methods

This section describes the different authentication methods available for connecting dbt Cloud to Snowflake.

### Username / Password

**Available in:** Development environments, Deployment environments
Expand All @@ -27,7 +31,7 @@ username (specifically, the `login_name`) and the corresponding user's Snowflake
to authenticate dbt Cloud to run queries against Snowflake on behalf of a Snowflake user.

**Note**: The schema field in the **Developer Credentials** section is a required field.
<Lightbox src="/img/docs/dbt-cloud/snowflake-userpass-auth.png" title="Snowflake username/password authentication"/>
<Lightbox src="/img/docs/dbt-cloud/snowflake-userpass-auth.png" width="70%" title="Snowflake username/password authentication"/>

### Key pair

Expand Down Expand Up @@ -65,12 +69,22 @@ The `Keypair` auth method uses Snowflake's [Key Pair Authentication](https://doc
The OAuth auth method permits dbt Cloud to run development queries on behalf of
a Snowflake user without the configuration of Snowflake password in dbt Cloud. For
more information on configuring a Snowflake OAuth connection in dbt Cloud, please see [the docs on setting up Snowflake OAuth](/docs/cloud/manage-access/set-up-snowflake-oauth).
<Lightbox src="/img/docs/dbt-cloud/dbt-cloud-enterprise/database-connection-snowflake-oauth.png" title="Configuring Snowflake OAuth connection"/>
<Lightbox src="/img/docs/dbt-cloud/dbt-cloud-enterprise/database-connection-snowflake-oauth.png" width="70%" title="Configuring Snowflake OAuth connection"/>

## Configuration

To learn how to optimize performance with data platform-specific configurations in dbt Cloud, refer to [Snowflake-specific configuration](/reference/resource-configs/snowflake-configs).

### Custom domain URL support

To connect to Snowflake through a custom domain (vanity URL) instead of the account locator, use [extended attributes](/docs/dbt-cloud-environments#extended-attributes) to configure the `host` parameter with the custom domain:

```yaml
host: https://custom_domain_to_snowflake.com
```

This configuration may conflict with Snowflake OAuth when used with PrivateLink. IF users can't reach Snowflake authentication servers from a networking standpoint, please [contact dbt Support](mailto:[email protected]) to find a workaround with this architecture.
## Troubleshooting
<!--might need to turn this into details toggle if more troubleshooting items arise -->
Expand Down
38 changes: 38 additions & 0 deletions website/docs/docs/collaborate/access-from-dbt-cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: "Access dbt Explorer from dbt Cloud features"
sidebar_label: "Access from dbt Cloud"
description: "Learn where and how to directly access and interact with dbt Explorer from dbt Cloud features and products."
---

Access dbt Explorer from other features and products inside dbt Cloud, ensuring you have a seamless experience navigating between resources and lineage in your project.

This page explains how to access dbt Explorer from various dbt Cloud features, including the dbt Cloud IDE and jobs. While the primary way to navigate to dbt Explorer is through the **Explore** link in the navigation, you can also access it from other dbt Cloud features.

### dbt Cloud IDE
You can enhance your project navigation and editing experience by directly accessing resources from the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to dbt Explorer for model, seed, or snapshot files. This workflow offers a seamless transition between the IDE and Explorer, allowing you to quickly navigate between viewing project metadata and making updates to your models or other resources without switching contexts.

#### Access dbt Explorer from the IDE
- In your model, seed, or snapshot file, click the **View in Explorer** icon to the right of your file breadcrumb (under the file name tab).
- This opens the model, seed, or snapshot file in a new tab, allowing you to view resources/lineage directly in Explorer.

<Lightbox src="/img/docs/collaborate/dbt-explorer/explorer-from-ide.jpg" title="Access dbt Explorer from the IDE by clicking on the 'View in Explorer' icon next to the file breadcrumbs. " />

### Lineage tab in jobs
The **Lineage tab** in dbt Cloud jobs displays the lineage associated with the [job run](/docs/deploy/jobs). Access dbt Explorer directly from this tab, allowing you understand dependencies/relationships of resources in your project.

#### Access dbt Explorer from the lineage tab
- From a job, select the **Lineage tab**.
- Double-click the node in the lineage to open a new tab and view its metadata directly in dbt Explorer.

<Lightbox src="/img/docs/collaborate/dbt-explorer/explorer-from-lineage.gif" title="Access dbt Explorer from the lineage tab by double-clicking on the lineage node." />

### Model timing tab in jobs <Lifecycle status="enterprise,team"/>
The [model timing tab](/docs/deploy/run-visibility#model-timing) in dbt Cloud jobs displays the composition, order, and time taken by each model in a job run.

Access dbt Explorer directly from the **modeling timing tab**, which helps you investigate resources, diagnose performance bottlenecks, understand dependencies/relationships of slow-running models, and potentially make changes to improve their performance.

#### Access dbt Explorer from the model timing tab
- From a job, select the **model timing tab**.
- Hover over a resource and click on **View on Explorer** to view the resource metadata directly in dbt Explorer.

<Lightbox src="/img/docs/collaborate/dbt-explorer/explorer-from-model-timing.jpg" title="Access dbt Explorer from the model timing tab by hovering over the resource and clicking 'View in Explorer'." />
4 changes: 1 addition & 3 deletions website/docs/docs/collaborate/build-and-view-your-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ description: "Automatically generate project documentation as you run jobs."
pagination_next: null
---

<p style={{ color: '#808080', fontSize: '1.1em' }}>
dbt Cloud enables you to generate documentation for your project and data platform. The documentation is automatically updated with new information after a fully successful job run, ensuring accuracy and relevance.
</p>

The default documentation experience in dbt Cloud is [dbt Explorer](/docs/collaborate/explore-projects), available on [Team or Enterprise plans](https://www.getdbt.com/pricing/). Use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's resources (such as models, tests, and metrics) and their lineage to gain a better understanding of its latest production state.

Expand Down Expand Up @@ -47,7 +45,7 @@ To create and schedule documentation-only jobs at the end of your production job

## dbt Docs

dbt Docs, available on developer plans or dbt Core users, is a legacy feature that generates a website from your dbt project using the `dbt docs generate` command. It provides a central location to view your project's resources, such as models, tests, and metrics, and lineage &mdash; and helps you understand the data in your warehouse.
dbt Docs, available on developer plans or dbt Core users, generates a website from your dbt project using the `dbt docs generate` command. It provides a central location to view your project's resources, such as models, tests, and lineage &mdash; and helps you understand the data in your warehouse.

### Configure project documentation

Expand Down
8 changes: 8 additions & 0 deletions website/docs/docs/collaborate/dbt-explorer-faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,3 +171,11 @@ Yes, users with read-only access can use the dbt Explorer. Specific feature avai
The ability to embed and share views is being evaluated as a potential future capability.

</Expandable>

<Expandable alt_header=" Is dbt Explorer accessible from other areas inside dbt Cloud?" >

Yes, you can [access dbt Explorer from various dbt Cloud features](/docs/collaborate/access-from-dbt-cloud), ensuring you have a seamless experience navigating between resources and lineage in your project.

While the primary way to access dbt Explorer is through the **Explore** link in the navigation, you can also access it from the [dbt Cloud IDE](/docs/collaborate/access-from-dbt-cloud#dbt-cloud-ide), [the lineage tab in jobs](/docs/collaborate/access-from-dbt-cloud#lineage-tab-in-jobs), and the [model timing tab in jobs](/docs/collaborate/access-from-dbt-cloud#model-timing-tab-in-jobs).

</Expandable>
6 changes: 2 additions & 4 deletions website/docs/docs/collaborate/explore-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,7 @@ pagination_next: "docs/collaborate/column-level-lineage"
pagination_prev: null
---

<p style={{ color: '#808080', fontSize: '1.1em' }}>

With dbt Explorer, you can view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their <Term id="data-lineage">lineage</Term> to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.
</p>

## Prerequisites

Expand Down Expand Up @@ -57,7 +54,7 @@ To explore the lineage graphs of tests and macros, view [their resource details
- Hover over any item in the graph to display the resource’s name and type.
- Zoom in and out on the graph by mouse-scrolling.
- Grab and move the graph and the nodes.
- Right click on a node (context menu) to:
- Right-click on a node (context menu) to:
- Refocus on the node, including its upstream and downstream nodes
- Refocus on the node and its downstream nodes only
- Refocus on the node and it upstream nodes only
Expand Down Expand Up @@ -258,6 +255,7 @@ You can explore the metadata from your production or staging environment to info

<Lightbox src="/img/docs/collaborate/dbt-explorer/explore-staging-env.png" width="85%" title="Explore in a staging environment" />


## Related content
- [Enterprise permissions](/docs/cloud/manage-access/enterprise-permissions)
- [About model governance](/docs/collaborate/govern/about-model-governance)
Expand Down
Loading

0 comments on commit c83f526

Please sign in to comment.