Skip to content

Commit

Permalink
Merge pull request #448 from dbt-labs/repo-sync
Browse files Browse the repository at this point in the history
REPO SYNC - Public to Private
  • Loading branch information
john-rock authored Feb 15, 2024
2 parents c61870a + fd60564 commit 83441b2
Show file tree
Hide file tree
Showing 8 changed files with 167 additions and 4 deletions.
18 changes: 18 additions & 0 deletions website/docs/docs/collaborate/column-level-lineage.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ When exploring your data products, navigating column lineage allows analytics en

## Caveats

Following are the CLL caveats/limitations.

### SQL parsing

Column-level lineage relies on SQL parsing. Errors can occur when parsing fails or a column's origin is unknown (like with JSON unpacking, lateral joins, and so on). In these cases, lineage may be incomplete and dbt Cloud will provide a warning about it in the column lineage.

<Lightbox src="/img/docs/collaborate/dbt-explorer/example-parsing-error-pill.png" title="Example of warning in the full lineage graph"/>
Expand All @@ -54,3 +58,17 @@ Possible error cases are:
- **Parsing error** &mdash; Error occurs when the SQL is ambiguous or too complex for parsing. An example of ambiguous parsing scenarios are _complex_ lateral joins.
- **Python error** &mdash; Error occurs when a Python model is used within the lineage. Due to the nature of Python models, it's not possible to parse and determine the lineage.
- **Unknown error** &mdash; Error occurs when the lineage can't be determined for an unknown reason. An example of this would be if a dbt best practice is not being followed, like using hardcoded table names instead of `ref` statements.

### Data platform support

CLL in dbt Cloud works with the following data platforms:
- Snowflake
- BigQuery
- Redshift
- Databricks (Unity Catalog)

The following adapters aren't currently supported by CLL in dbt Cloud. More of these platforms will be supported in the future.
- Hive metastore version of Databricks
- Apache Spark
- Starburst/Trino
- Microsoft Fabric
2 changes: 2 additions & 0 deletions website/docs/docs/core/connect-data-platform/mindsdb-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ meta:

The dbt-mindsdb package allows dbt to connect to [MindsDB](https://github.com/mindsdb/mindsdb).

:::

import SetUpPages from '/snippets/_setup-pages-intro.md';

<SetUpPages meta={frontMatter.meta} />
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ As a dbt user, your main focus will be on writing models (i.e. select queries) t
| Load seed files| Often in analytics, raw values need to be mapped to a more readable value (for example, converting a country-code to a country name) or enriched with static or infrequently changing data. These data sources, known as seed files, can be saved as a CSV file in your `project` and loaded into your data warehouse using the `seed` command. Read more about [Seeds](/docs/build/seeds).|
| Snapshot data | Often, records in a data source are mutable, in that they change over time. This can be difficult to handle in analytics if you want to reconstruct historic values. dbt provides a mechanism to snapshot raw data for a point in time, through use of [snapshots](/docs/build/snapshots).|

### Related docs
## Related docs

- [Quickstarts for dbt](/guides)
- [Best practice guides](/best-practices)
Expand Down
8 changes: 6 additions & 2 deletions website/docs/reference/artifacts/dbt-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,19 @@ They could also be used to:

dbt has produced artifacts since the release of dbt-docs in v0.11.0. Starting in dbt v0.19.0, we are committing to a stable and sustainable way of versioning, documenting, and validating dbt artifacts.

## When are artifacts produced?
### When are artifacts produced? <Lifecycle status="team,enterprise"/>

Most dbt commands (and corresponding RPC methods) produce artifacts:
- [semantic manifest](/docs/dbt-cloud-apis/sl-manifest): Lives in the `/target` directory of your dbt project and stores various artifacts (such as compiled models and tests) generated during the execution of your project.
- [semantic manifest](/docs/dbt-cloud-apis/sl-manifest): produced whenever your dbt project is parsed
- [manifest](/reference/artifacts/manifest-json): produced by commands that read and understand your project
- [run results](/reference/artifacts/run-results-json): produced by commands that run, compile, or catalog nodes in your DAG
- [catalog](catalog-json): produced by `docs generate`
- [sources](/reference/artifacts/sources-json): produced by `source freshness`

## Where are artifacts produced?

By default, artifacts are written to the `/target` directory of your dbt project. You can configure the location using the [`target-path`](/reference/project-configs/target-path#configuration).

## Common metadata

All artifacts produced by dbt include a `metadata` dictionary with these properties:
Expand Down
51 changes: 51 additions & 0 deletions website/src/components/expandable/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/* eslint-disable */

import React, { useState } from 'react';
import styles from './styles.module.css';

function slugify(text) {
return text.toString().toLowerCase()
.normalize('NFD') // Normalize to NFD Unicode form
.replace(/[\u0300-\u036f]/g, '') // Remove diacritics
.replace(/\s+/g, '-') // Replace spaces with -
.replace(/[^\w\-]+/g, '') // Remove all non-word chars
.replace(/\-\-+/g, '-') // Replace multiple - with single -
.replace(/^-+/, '') // Trim - from start
.replace(/-+$/, ''); // Trim - from end
}

function expandable({ children, alt_header = null }) {
if(!alt_header) { return null; }
const [isOn, setOn] = useState(false);
// generate a slug from the alt_header
const anchorId = slugify(alt_header);

const handleToggleClick = (event) => {
event.preventDefault();
setOn(current => !current);
};

return (
<div id={anchorId} className={`${styles.expandableContainer} ${styles.local} expandable-anchor anchor`}>
<a
href={`#${anchorId}`}
className={styles.link}
onClick={handleToggleClick}
role="button"
tabIndex="0"
>
<span className={`${styles.toggle} ${isOn ? styles.toggleDown : styles.toggleRight}`}></span>
&nbsp;
<span className={styles.headerText}>{alt_header}</span>
</a>
<div
style={{ display: isOn ? 'block' : 'none' }}
className={styles.body}
>
{children}
</div>
</div>
);
}

export default expandable;
86 changes: 86 additions & 0 deletions website/src/components/expandable/styles.module.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
:local(.link) :local(.headerText) {
content: '';
color: black; /* Black text in normal mode */
text-decoration: none;
transition: text-decoration 0.3s; /* Smooth transition */
font-weight: 600;
margin-bottom: 20px;
}

:local(.link:hover) :local(.headerText),
:local(.link:focus) :local(.headerText) {
text-decoration: underline;
cursor: pointer;
}

:local(.toggle)::before {
content: '';
display: inline-block;
border: solid black; /* Arrow color */
border-width: 0 2px 2px 0; /* Adjust for 'boldness' */
padding: 3px; /* Adjust size */
transform: rotate(45deg); /* Initial right-pointing arrow */
transition: transform 0.3s; /* Smooth transition for toggle icon */
margin-right: 8px;
}

:local(.toggleDown)::before {
transform: rotate(225deg); /* Downward arrow */
}

:local(.toggleRight)::before {
transform: rotate(315deg); /* Right-pointing arrow */
}

:local(.toggle)::before {
border-color: rgb(253, 153, 83); /* Orange arrow color in light mode */
}

/* Adjusting for Light and Dark Modes */
:local(html[data-theme='dark'] .link), :local(html[data-theme='dark'] .headerText) {
color: white; /* White text in dark mode */
}

:local(html[data-theme='dark'] .toggle)::before {
border-color: white; /* White arrow in dark mode */
border-color: rgb(253, 153, 83);
}

.expandableContainer :local(.body) {
margin-top: 10px;
margin-left: .5em;
padding: 10px;
background-color: transparent;
}

:local(html[data-theme='dark'] .link),
:local(html[data-theme='dark'] .headerText) {
color: white; /* White text in dark mode */
}

:local(.body > p:last-child) {
margin-bottom: 0px;

}

:local(.link)::after {
content: "";
display: inline-block;
width: 12px;
height: 12px;
background-image: url('/img/copy.png');
background-size: contain;
background-repeat: no-repeat;
opacity: 0; /* Start with icon hidden */
transition: opacity 0.3s ease-in-out;
margin-left: 8px;
}

:local(.link:not(.toggleOpen)):hover::after {
opacity: 1; /* Only show icon on hover when toggle is not open */
}

.expandableContainer {
margin-bottom: 10px; /* Adjust this value as needed to create space */
}

2 changes: 2 additions & 0 deletions website/src/theme/MDXComponents/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ import dbtEditor from '@site/src/components/dbt-editor';
import Icon from '@site/src/components/icon';
import Lifecycle from '@site/src/components/lifeCycle';
import detailsToggle from '@site/src/components/detailsToggle';
import expandable from '@site/src/components/expandable';

const MDXComponents = {
head: MDXHead,
Expand Down Expand Up @@ -94,5 +95,6 @@ const MDXComponents = {
Icon: Icon,
Lifecycle: Lifecycle,
detailsToggle: detailsToggle,
expandable: expandable,
};
export default MDXComponents;
2 changes: 1 addition & 1 deletion website/static/js/headerLinkCopy.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

// separating function from eventlistener to understand they are two separate things
function copyHeader () {
const headers = document.querySelectorAll("h2.anchor, h3.anchor");
const headers = document.querySelectorAll("h2.anchor, h3.anchor, .expandable-anchor.anchor");

headers.forEach((header) => {
header.style.cursor = "pointer";
Expand Down

0 comments on commit 83441b2

Please sign in to comment.