Skip to content

Commit

Permalink
[mega branch] main spring launch event PR (#5467)
Browse files Browse the repository at this point in the history
This PR and branch serves as the main (mega) branch the docs team will
use to merge their changes into for the spring launch showcase event.

Some of the changes inlcude:
- lifecycle status updates
- product-related updates
- spring launch joint release notes

---------

Co-authored-by: Ly Nguyen <[email protected]>
Co-authored-by: Matt Shaver <[email protected]>
Co-authored-by: Leona B. Campbell <[email protected]>
Co-authored-by: Ly Nguyen <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
6 people authored May 14, 2024
1 parent 0eac8ee commit 7945f41
Show file tree
Hide file tree
Showing 36 changed files with 323 additions and 144 deletions.
10 changes: 8 additions & 2 deletions website/docs/best-practices/how-we-mesh/mesh-4-faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,9 +245,15 @@ Yes! As long as a user has permissions (at least read-only access) on all projec

By default, cross-project references resolve to the “Production” deployment environment of the upstream project. If your organization has genuinely different data in production versus non-production environments, this poses an issue.

For this reason, we will soon roll out a new canonical type of deployment environment: “Staging.” If a project defines both a “Production” environment and a “Staging” environment, then cross-project references from development and “Staging” environments will resolve to “Staging,” whereas only references coming from “Production” environments will resolve to “Production.” In this way, you are guaranteed separation of data environments, without needing to duplicate project configurations.
For this reason, we rolled out canonical type of deployment environment: “[Staging](/docs/deploy/deploy-environments#staging-environment).” If a project defines both a “Production” environment and a “Staging” environment, then cross-project references from development and “Staging” environments will resolve to “Staging,” whereas only references coming from “Production” environments will resolve to “Production.” In this way, you are guaranteed separation of data environments, without needing to duplicate project configurations.

If you’re interested in beta access to “Staging” environments, let your dbt Labs account representative know!
</detailsToggle>

<detailsToggle alt_header="Does dbt Mesh work if projects are 'duplicated' (dev project <> prod project)?">

The short answer is "no." Cross-project references require that each project `name` be unique in your dbt Cloud account.

Historical limitations required customers to "duplicate" projects so that one actual dbt project (codebase) would map to more than one dbt Cloud project. To that end, we are working to remove the historical limitations that required customers to "duplicate" projects in dbt Cloud — Staging environments for data isolation, environment-level permissions, and environment-level data warehouse connections (coming soon). Once those pieces are in place, it should no longer be necessary to define separate dbt Cloud projects to isolate data environments or permissions.

</detailsToggle>

Expand Down
10 changes: 8 additions & 2 deletions website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,25 @@ keywords:
- unit test, unit tests, unit testing, dag
---

:::note

This functionality is only supported in dbt Core v1.8+ or dbt Cloud accounts that have opted to ["Keep on latest version"](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version).

:::

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.

Now, we are introducing a new type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.
With dbt Core v1.8 and dbt Cloud environments that opt to "Keep on latest version," we have introduced an additional type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.

## Before you begin

- This functionality is only supported in dbt Core v1.8+ or dbt Cloud accounts that have opted to ["Keep on latest version"](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version).
- We currently only support unit testing SQL models.
- We currently only support adding unit tests to models in your _current_ project.
- We currently *don't* support unit testing models that use recursive SQL.
- You must specify all fields in a BigQuery STRUCT in a unit test. You cannot use only a subset of fields in a STRUCT.
- If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](#unit-testing-versioned-models) for more information.
- Unit tests must be defined in a YML file in your `models/` directory.
- Available to dbt Cloud customers who have selected ["Keep on latest version"](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version) and dbt Core v1.8.0 or later.
- Table names must be [aliased](/docs/build/custom-aliases) in order to unit test `join` logic.

Read the [reference doc](/reference/resource-properties/unit-tests) for more details about formatting your unit tests.
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/about-cloud-develop-defer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ By default, dbt follows these rules:

For a clean slate, it's a good practice to drop the development schema at the start and end of your development cycle.

If you require additional controls over production data, create a [Staging evironment](/docs/deploy/deploy-environments#staging-environment) <Lifecycle status='beta' /> and dbt will use that, rather than the Production environment, to resolve `{{ ref() }}` functions.
If you require additional controls over production data, create a [Staging environment](/docs/deploy/deploy-environments#staging-environment) and dbt will use that, rather than the Production environment, to resolve `{{ ref() }}` functions.

## Required setup

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/about-cloud/about-dbt-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make

<Card
title="dbt Cloud IDE"
body="The IDE is the easiest and most efficient way to develop dbt models, allowing you to build, test, run, and version control your dbt projects directly from your browser."
body="The IDE is the easiest and most efficient way to develop dbt models, allowing you to build, test, run, and version control your dbt projects directly from your browser. Use dbt Assist, a powerful AI co-pilot feature that automatically generates documentation, metrics, semantic models, and tests."
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

Expand Down
22 changes: 9 additions & 13 deletions website/docs/docs/cloud/cloud-cli-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,17 @@ You can install the dbt Cloud CLI on the command line by using one of these meth
</details>

<Tabs queryString="install">

<TabItem value="brew" label="macOS (brew)">

Before you begin, make sure you have [Homebrew installed](http://brew.sh/) in your code editor or command line terminal. Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.

1. Verify that you don't already have dbt Core installed by running the following command.
1. Verify that you don't already have dbt Core installed by running the following command:

```bash
which dbt
```

If the output is `dbt not found`, then that confirms you don't have it installed.
:::tip Run `pip uninstall dbt` to uninstall dbt Core
Expand Down Expand Up @@ -109,7 +110,7 @@ Note that if you're using VS Code, you must restart it to pick up modified envir
```
If you don't see this output, check that you've deactivated pyenv or venv and don't have a global dbt version installed.
* Note that you no longer need to run the `dbt deps` command when your environment starts. This step was previously required during initialization. However, you should still run `dbt deps` if you make any changes to your `packages.yml` file.
4. Clone your repository to your local computer using `git clone`. For example, to clone a GitHub repo using HTTPS format, run `git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY`.
Expand Down Expand Up @@ -222,9 +223,7 @@ We recommend using virtual environments (venv) to namespace `cloud-cli`.
## Update dbt Cloud CLI
The following instructions explain how to update the dbt Cloud CLI to the latest version depending on your operating system.
During the public preview period, we recommend updating before filing a bug report. This is because the API is subject to breaking changes.
The following instructions explain how to update the dbt Cloud CLI to the latest version depending on your operating system.
<Tabs>
Expand Down Expand Up @@ -257,16 +256,13 @@ To update:
</Tabs>
## Using VS Code extensions
Visual Studio (VS) Code extensions enhance command line tools by adding extra functionalities. The dbt Cloud CLI is fully compatible with dbt Core, however, it doesn't support some dbt Core APIs required by certain tools, for example, VS Code extensions.
## VS Code extension <Lifecycle status="beta"/>
You can use extensions like [dbt-power-user](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) with the dbt Cloud CLI by following these steps:
Visual Studio (VS) Code extensions enhance command line tools by adding extra functionalities. Power user for dbt Core and dbt Cloud is a common extension used to enhance your development with VS Code.
- [Install](/docs/cloud/cloud-cli-installation?install=brew) it using Homebrew along with dbt Core.
- [Create an alias](#faqs) to run the dbt Cloud CLI as `dbt-cloud`.
You can use the dbt Cloud CLI with the [Power user for dbt Core and dbt Cloud](https://docs.myaltimate.com/arch/beta/) extension, now available in beta for the dbt Cloud CLI.
This setup allows dbt-power-user to continue to work with dbt Core in the background, alongside the dbt Cloud CLI. For more, check the dbt Power User [documentation](https://docs.myaltimate.com/).
To use the extension, you must first [install the dbt Cloud CLI](#install-dbt-cloud-cli).
## FAQs
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/configure-cloud-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Learn how to configure the dbt Cloud CLI for your dbt Cloud project to run dbt c

## Configure the dbt Cloud CLI

Once you install the dbt Cloud CLI, you need to configure it to connect to a dbt Cloud project.
Once you install the dbt Cloud CLI, you need to configure it to connect to a dbt Cloud project.

1. In dbt Cloud, navigate to **Develop** and click **Configure dbt Cloud CLI** to download your `dbt_cloud.yml` credentials file.

Expand Down
29 changes: 29 additions & 0 deletions website/docs/docs/cloud/dbt-assist-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
title: "dbt Assist privacy and data"
sidebar_label: "dbt Assist privacy"
description: "dbt Assist’s powerful AI feature helps you deliver data that works."
---

# dbt Assist privacy and data <Lifecycle status='beta'/>

dbt Labs is committed to protecting your privacy and data. This page provides information about how dbt Labs handles your data when you use dbt Assist.

#### Is my data used by dbt Labs to train AI models?

No, dbt Assist does not use client warehouse data to train any AI models. It uses API calls to an AI provider.

#### Does dbt Labs share my personal data with third parties

dbt Labs only shares client personal information as needed to perform the services, under client instructions, or for legal, tax, or compliance reasons.

#### Does dbt Assist store or use personal data?

The user clicks the AI assist button, and the user does not otherwise enter data.

#### Does dbt Assist access my warehouse data?

dbt Assist utilizes the button click to initiate a search via API of the AI vendor. Such output must be double-checked by the user for completeness and accuracy. This does not involve the use of client’s warehouse data.

#### Can dbt Assist data be deleted upon client written request?

dbt Assist data, aside from usage data, does not persist on dbt Labs systems. Usage data is retained by dbt Labs. dbt Labs does not have possession of any personal or sensitive data. To the extent client identifies personal or sensitive information uploaded by or on behalf of client to dbt Labs systems, such data can be deleted within 30 days of written request.
23 changes: 23 additions & 0 deletions website/docs/docs/cloud/dbt-assist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: "About dbt Assist"
sidebar_label: "About dbt Assist"
description: "dbt Assist’s powerful AI co-pilot feature helps you deliver data that works."
---

# About dbt Assist <Lifecycle status='beta'/>

dbt Assist is a powerful artificial intelligence (AI) co-pilot feature that helps automate development in dbt Cloud, allowing you to focus on delivering data that works. dbt Assist’s AI co-pilot generates documentation, metrics, semantic models, and tests for your dbt SQL and Python models directly in the dbt Cloud IDE, with a click of a button, and helps you accomplish more in less time.

:::tip Beta feature
dbt Assist is a AI tool meant to _help_ developers generate documentation, metrics, semantic models, and tests in dbt Cloud. It's available in beta, in the dbt Cloud IDE only.

To use dbt Assist, you must have an active [dbt Cloud Enterprise account](https://www.getdbt.com/pricing) and agree to use dbt Labs' OpenAI key. [Register your interest](https://docs.google.com/forms/d/e/1FAIpQLScPjRGyrtgfmdY919Pf3kgqI5E95xxPXz-8JoVruw-L9jVtxg/viewform) to join the private beta or reach out to your account team to begin this process.
:::

<Lightbox src="/img/docs/dbt-cloud/cloud-ide/dbt-assist-doc.gif" width="100%" title="Use dbt Assist, a powerful AI feature, to automatically generate tests and documentation in the dbt Cloud IDE." />

## Feedback

Please note: Always review AI-generated code and content as it may produce incorrect results. dbt Assist features and/or functionality may be added or eliminated as part of the beta trial.

To give feedback, please reach out to your dbt Labs account team. We appreciate your feedback and suggestions as we improve dbt Assist.
Loading

0 comments on commit 7945f41

Please sign in to comment.