Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DBT Deploy support #74

Merged
merged 7 commits into from
Sep 24, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/e2e/get_bundle_info/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Get Deployment Bundle Info
description: Get deployment bundle info from Astronomer API
inputs:
deployment_id:
description: The deployment ID
required: true
organization_id:
description: The organization ID
required: true
astro_api_token:
description: The Astronomer API token
required: true
astronomer_host:
description: The Astronomer host
required: true
expected_status_code:
description: The expected status code
default: 200

outputs:
desired_bundle_version:
description: The desired bundle version
value: ${{ steps.get-deployment-info.outputs.desired_bundle_version }}
bundle_type:
description: The bundle type
value: ${{ steps.get-deployment-info.outputs.bundle_type }}
updated_at:
description: The timestamp at which the bundle was last updated
value: ${{ steps.get-deployment-info.outputs.updated_at }}

runs:
using: "composite"
steps:
- name: Get Deployment Info
id: get-deployment-info
shell: bash
run: |
STATUS_CODE=$(curl -s -w "%{http_code}" -o response.json -H "Authorization: Bearer ${{ inputs.astro_api_token }}" "https://api.${{ inputs.astronomer_host }}/v1alpha1/organizations/${{ inputs.organization_id }}/deployments/${{ inputs.deployment_id }}/deploys")
if [[ $STATUS_CODE -ne ${{ inputs.expected_status_code }} ]]; then
echo "Failed to get expected status code from GET Deployment API. Status code: $STATUS_CODE"
exit 1
fi
if [[ $(cat response.json | jq -r '.deploys | length') -eq 0 ]]; then
echo "No deploys found for the deployment: ${{ inputs.deployment_id }}"
exit 1
fi

# sort by updatedAt to fetch the latest deploy object
cat response.json | jq '.deploys | sort_by(.updatedAt)' > response_sorted.json

desired_bundle_version=$(cat response_sorted.json | jq -r '.[] | select(.type == "BUNDLE" and .bundles[0].bundleType == "dbt") | .bundles[0].desiredVersion' | head -n 1)
bundle_type=$(cat response_sorted.json | jq -r '.[] | select(.type == "BUNDLE" and .bundles[0].bundleType == "dbt") | .bundles[0].bundleType' | head -n 1)
updated_at=$(cat response_sorted.json | jq -r '.[] | select(.type == "BUNDLE" and .bundles[0].bundleType == "dbt") | .updatedAt' | head -n 1)

echo "desired_bundle_version=$desired_bundle_version" >> $GITHUB_OUTPUT
echo "bundle_type=$bundle_type" >> $GITHUB_OUTPUT
echo "updated_at=$updated_at" >> $GITHUB_OUTPUT
12 changes: 12 additions & 0 deletions .github/workflows/e2e/validate_deployment/action.yaml
Original file line number Diff line number Diff line change
@@ -5,6 +5,9 @@ inputs:
is_dag_only_deploy:
description: Whether the deploy operation was DAG-only
default: false
is_no_deploy:
description: If the deploy test was a no-op
default: false
dag_tarball_version_before:
description: The desired DAG tarball version before the test
required: true
@@ -26,6 +29,15 @@ runs:
shell: bash
run: |

if [[ "${{ inputs.is_no_deploy }}" == "true" ]]; then
if [[ "${{ inputs.dag_tarball_version_before }}" == "${{ inputs.dag_tarball_version_after }}" && "${{ inputs.image_version_before }}" == "${{ inputs.image_version_after }}" ]]; then
echo "Deploy Action validation succeeded: no deploy operation"
exit 0
fi
echo "Deploy Action validation failed: deploy operation was not a no-op"
exit 1
fi

# If it's dag only deploy then validate that only desiredDagTarballVersion was updated
if [[ "${{ inputs.is_dag_only_deploy }}" == "true" ]]; then
if [[ "${{ inputs.dag_tarball_version_before }}" != "${{ inputs.dag_tarball_version_after }}" && "${{ inputs.image_version_before }}" == "${{ inputs.image_version_after }}" ]]; then
Loading