Skip to content

Commit

Permalink
Merge branch 'main' into batch-metadata-freshness
Browse files Browse the repository at this point in the history
  • Loading branch information
MichelleArk committed Mar 11, 2024
2 parents 2beba0b + 2a99e36 commit 151b131
Show file tree
Hide file tree
Showing 169 changed files with 483 additions and 84 deletions.
1 change: 1 addition & 0 deletions .changes/1.0.0-b1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
## dbt-adapter 1.0.0-b1 - February 27, 2024
1 change: 1 addition & 0 deletions .changes/1.8.0-a2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
## dbt-adapter 1.8.0-a2 - February 28, 2024
1 change: 1 addition & 0 deletions .changes/1.8.0-b1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
## dbt-adapter 1.8.0-b1 - March 01, 2024
18 changes: 18 additions & 0 deletions .github/ISSUE_TEMPLATE/internal-epic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,24 @@ body:
validations:
required: false

- type: textarea
attributes:
label: Documentation
description: |
Provide a list of relevant documentation. Is there a proof of concept?
Does this require and RFCs, ADRs, etc.?
If the documentation exists, link it; if it does not exist yet, reference it descriptively.
Example:
- [ ] RFC for updating connection interface to accept new parameters
- [ ] POC: https://github.com/dbt-labs/dbt-adapters/pull/42
value: |
```[tasklist]
- [ ] Task
```
validations:
required: false

- type: textarea
attributes:
label: Consequences
Expand Down
17 changes: 14 additions & 3 deletions .github/ISSUE_TEMPLATE/internal-feature-implementation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ body:

- type: textarea
attributes:
label: Suggested tests
label: Testing
description: |
Provide scenarios to test. Include both positive and negative tests if possible.
Link to existing similar tests if appropriate.
Expand All @@ -71,13 +71,24 @@ body:

- type: textarea
attributes:
label: Are there any security concerns with these changes?
label: Security
description: |
Are there any security concerns with these changes?
When in doubt, run it by the security team.
placeholder: |
Example: Logging sensitive data
validations:
required: true
required: true

- type: textarea
attributes:
label: Docs
description: |
Are there any docs the will need to be added or updated?
placeholder: |
Example: We need to document how to configure this new authentication method.
validations:
required: true

- type: textarea
attributes:
Expand Down
7 changes: 4 additions & 3 deletions .github/actions/build-hatch/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ inputs:
working-dir:
description: Where to run commands from, supports namespace packaging
default: "./"
artifacts-dir:
archive-name:
description: Where to upload the artifacts
default: "dist"
required: true

runs:
using: composite
Expand All @@ -32,5 +32,6 @@ runs:
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: ${{ inputs.artifacts-dir}}
name: ${{ inputs.archive-name }}
path: ${{ inputs.working-dir }}dist/
retention-days: 3
10 changes: 7 additions & 3 deletions .github/actions/publish-pypi/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Publish - PyPI
description: Publish artifacts saved during build step to PyPI

inputs:
artifacts-dir-name:
archive-name:
description: Where to download the artifacts from
required: true
repository-url:
Expand All @@ -16,8 +16,12 @@ runs:
- name: Download artifacts
uses: actions/download-artifact@v3
with:
name: ${{ inputs.artifacts-dir }}
path: .
name: ${{ inputs.archive-name }}
path: dist/

- name: "[DEBUG]"
run : ls -R
shell: bash

- name: Publish artifacts to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,6 @@ resolves #
### Checklist

- [ ] I have read [the contributing guide](https://github.com/dbt-labs/dbt-adapter/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] I have run this code in development, and it appears to resolve the stated issue
- [ ] This PR includes tests, or tests are not required/relevant for this PR
- [ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
- [ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX
259 changes: 259 additions & 0 deletions .github/workflows/github-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
# **what?**
# Create a new release on GitHub and include any artifacts in the `/dist` directory of the GitHub artifacts store.
#
# Inputs:
# sha: The commit to attach to this release
# version_number: The release version number (i.e. 1.0.0b1, 1.2.3rc2, 1.0.0)
# changelog_path: Path to the changelog file for release notes
# test_run: Test run (Publish release as draft)
#
# **why?**
# Reusable and consistent GitHub release process.
#
# **when?**
# Call after a successful build. Build artifacts should be ready to release and live in a dist/ directory.
#
# This workflow expects the artifacts to already be built and living in the artifact store of the workflow.
#
# Validation Checks
#
# 1. If no release already exists for this commit and version, create the tag and release it to GitHub.
# 2. If a release already exists for this commit, skip creating the release but finish with a success.
# 3. If a release exists for this commit under a different tag, fail.
# 4. If the commit is already associated with a different release, fail.

name: GitHub Release

on:
workflow_call:
inputs:
sha:
description: The commit to attach to this release
required: true
type: string
version_number:
description: The release version number (i.e. 1.0.0b1)
required: true
type: string
changelog_path:
description: Path to the changelog file for release notes
required: true
type: string
test_run:
description: Test run (Publish release as draft)
required: true
type: boolean
archive_name:
description: artifact name to download
required: true
type: string
outputs:
tag:
description: The path to the changelog for this version
value: ${{ jobs.check-release-exists.outputs.tag }}

permissions:
contents: write

env:
REPO_LINK: ${{ github.server_url }}/${{ github.repository }}
NOTIFICATION_PREFIX: "[GitHub Release]"

jobs:
log-inputs:
runs-on: ubuntu-latest
steps:
- name: "[DEBUG] Print Variables"
run: |
echo The last commit sha in the release: ${{ inputs.sha }}
echo The release version number: ${{ inputs.version_number }}
echo Expected Changelog path: ${{ inputs.changelog_path }}
echo Test run: ${{ inputs.test_run }}
echo Repo link: ${{ env.REPO_LINK }}
echo Notification prefix: ${{ env.NOTIFICATION_PREFIX }}
check-release-exists:
runs-on: ubuntu-latest
outputs:
exists: ${{ steps.release_check.outputs.exists }}
draft_exists: ${{ steps.release_check.outputs.draft_exists }}
tag: ${{ steps.set_tag.outputs.tag }}

steps:
- name: "Generate Release Tag"
id: set_tag
run: echo "tag=v${{ inputs.version_number }}" >> $GITHUB_OUTPUT

# When the GitHub CLI doesn't find a release for the given tag, it will exit 1 with a
# message of "release not found". In our case, it's not an actual error, just a
# confirmation that the release does not already exists so we can go ahead and create it.
# The `|| true` makes it so the step does not exit with a non-zero exit code
# Also check if the release already exists is draft state. If it does, and we are not
# testing then we can publish that draft as is. If it's in draft and we are testing, skip the
# release.
- name: "Check If Release Exists For Tag ${{ steps.set_tag.outputs.tag }}"
id: release_check
run: |
output=$((gh release view ${{ steps.set_tag.outputs.tag }} --json isDraft,targetCommitish --repo ${{ env.REPO_LINK }}) 2>&1) || true
if [[ "$output" == "release not found" ]]
then
title="Release for tag ${{ steps.set_tag.outputs.tag }} does not exist."
message="Check passed."
echo "exists=false" >> $GITHUB_OUTPUT
echo "draft_exists=false" >> $GITHUB_OUTPUT
echo "::notice title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
exit 0
fi
commit=$(jq -r '.targetCommitish' <<< "$output")
if [[ $commit != ${{ inputs.sha }} ]]
then
title="Release for tag ${{ steps.set_tag.outputs.tag }} already exists for commit $commit!"
message="Cannot create a new release for commit ${{ inputs.sha }}. Exiting."
echo "::error title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
exit 1
fi
isDraft=$(jq -r '.isDraft' <<< "$output")
if [[ $isDraft == true ]] && [[ ${{ inputs.test_run }} == false ]]
then
title="Release tag ${{ steps.set_tag.outputs.tag }} already associated with the draft release."
message="Release workflow will publish the associated release."
echo "exists=false" >> $GITHUB_OUTPUT
echo "draft_exists=true" >> $GITHUB_OUTPUT
echo "::notice title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
exit 0
fi
title="Release for tag ${{ steps.set_tag.outputs.tag }} already exists."
message="Skip GitHub Release Publishing."
echo "exists=true" >> $GITHUB_OUTPUT
echo "draft_exists=false" >> $GITHUB_OUTPUT
echo "::notice title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ env.REPO_LINK }}

- name: "[DEBUG] Log Job Outputs"
run: |
echo exists: ${{ steps.release_check.outputs.exists }}
echo draft_exists: ${{ steps.release_check.outputs.draft_exists }}
echo tag: ${{ steps.set_tag.outputs.tag }}
skip-github-release:
runs-on: ubuntu-latest
needs: [check-release-exists]
if: needs.check-release-exists.outputs.exists == 'true'

steps:
- name: "Tag Exists, Skip GitHub Release Job"
run: |
echo title="A tag already exists for ${{ needs.check-release-exists.outputs.tag }} and commit."
echo message="Skipping GitHub release."
echo "::notice title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
audit-release-different-commit:
runs-on: ubuntu-latest
needs: [check-release-exists]
if: needs.check-release-exists.outputs.exists == 'false'

steps:
- name: "Check If Release Already Exists For Commit"
uses: cardinalby/[email protected]
id: check_release_commit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
commitSha: ${{ inputs.sha }}
doNotFailIfNotFound: true # returns blank outputs when not found instead of error
searchLimit: 15 # Since we only care about recent releases, speed up the process

- name: "[DEBUG] Print Release Details"
run: |
echo steps.check_release_commit.outputs.id: ${{ steps.check_release_commit.outputs.id }}
echo steps.check_release_commit.outputs.tag_name: ${{ steps.check_release_commit.outputs.tag_name }}
echo steps.check_release_commit.outputs.target_commitish: ${{ steps.check_release_commit.outputs.target_commitish }}
echo steps.check_release_commit.outputs.prerelease: ${{ steps.check_release_commit.outputs.prerelease }}
# Since we already know a release for this tag does not exist, if we find anything it's for the wrong tag, exit
- name: "Check If The Tag Matches The Version Number"
if: steps.check_release_commit.outputs.id != ''
run: |
title="Tag ${{ steps.check_release_commit.outputs.tag_name }} already exists for this commit!"
message="Cannot create a new tag for ${{ needs.check-release-exists.outputs.tag }} for the same commit"
echo "::error title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
exit 1
publish-draft-release:
runs-on: ubuntu-latest
needs: [check-release-exists, audit-release-different-commit]
if: >-
needs.check-release-exists.outputs.draft_exists == 'true' &&
inputs.test_run == false
steps:
- name: "Publish Draft Release - ${{ needs.check-release-exists.outputs.tag }}"
run: |
gh release edit $TAG --draft=false --repo ${{ env.REPO_LINK }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ needs.check-release-exists.outputs.tag }}

create-github-release:
runs-on: ubuntu-latest
needs: [check-release-exists, audit-release-different-commit]
if: needs.check-release-exists.outputs.draft_exists == 'false'

steps:
- name: "Check out repository"
uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}

- name: "Download Artifact ${{ inputs.archive_name }}"
uses: actions/download-artifact@v3
with:
name: ${{ inputs.archive_name }}
path: .

- name: "[DEBUG] Display Structure Of Expected Files"
run: |
ls -R .changes
ls -l dist
- name: "Set Release Type"
id: release_type
run: |
if ${{ contains(inputs.version_number, 'rc') || contains(inputs.version_number, 'b') }}
then
echo Release will be set as pre-release
echo "prerelease=--prerelease" >> $GITHUB_OUTPUT
else
echo This is not a prerelease
fi
- name: "Set As Draft Release"
id: draft
run: |
if [[ ${{ inputs.test_run }} == true ]]
then
echo Release will be published as draft
echo "draft=--draft" >> $GITHUB_OUTPUT
else
echo This is not a draft release
fi
- name: "GitHub Release Workflow Annotation"
run: |
title="Release ${{ needs.check-release-exists.outputs.tag }}"
message="Configuration: ${{ steps.release_type.outputs.prerelease }} ${{ steps.draft.outputs.draft }}"
echo "::notice title=${{ env.NOTIFICATION_PREFIX }}: $title::$message"
- name: "Create New GitHub Release - ${{ needs.check-release-exists.outputs.tag }}"
run: |
gh release create $TAG ./dist/* --title "$TITLE" --notes-file $RELEASE_NOTES --target $COMMIT $PRERELEASE $DRAFT --repo ${{ env.REPO_LINK }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ needs.check-release-exists.outputs.tag }}
TITLE: ${{ github.event.repository.name }} ${{ needs.check-release-exists.outputs.tag }}
RELEASE_NOTES: ${{ inputs.changelog_path }}
COMMIT: ${{ inputs.sha }}
PRERELEASE: ${{ steps.release_type.outputs.prerelease }}
DRAFT: ${{ steps.draft.outputs.draft }}
Loading

0 comments on commit 151b131

Please sign in to comment.