Add vertica profile with test by perttus · Pull Request #540 · astronomer/astronomer-cosmos

perttus · 2023-09-15T12:20:38Z

Description

Added Vertica profile with test

Related Issue(s)

#538

Checklist

I have made corresponding changes to the documentation (if required)
I have added tests that prove my fix is effective or that my feature works

Signed-off-by: Perttu Salonen <perttu.salonen@nitor.com>

netlify · 2023-09-15T12:20:42Z

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name	Link
🔨 Latest commit	`0da9e60`
🔍 Latest deploy log	https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/652fb54f60e95d0008924595

codecov · 2023-09-15T12:52:41Z

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (e09ac6d) 93.15% compared to head (0da9e60) 93.21%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #540      +/-   ##
==========================================
+ Coverage   93.15%   93.21%   +0.06%     
==========================================
  Files          51       53       +2     
  Lines        2059     2079      +20     
==========================================
+ Hits         1918     1938      +20     
  Misses        141      141

Files	Coverage Δ
cosmos/profiles/__init__.py	`96.42% <100.00%> (+0.13%)`	⬆️
cosmos/profiles/vertica/__init__.py	`100.00% <100.00%> (ø)`
cosmos/profiles/vertica/user_pass.py	`100.00% <100.00%> (ø)`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tatiana

Hi @perttus , this looks amazing, thank you very much for your contribution!
Is there any chance you could add a test to cover mock_profile?
We can release this as part of 1.2.0.

Signed-off-by: Perttu Salonen <perttu.salonen@nitor.com>

tatiana · 2023-09-27T13:55:57Z

Hi @perttus, since this is a new feature, we'll aim to launch it as part of the 1.2 release. Thank you very much for your contribution!

tatiana · 2023-10-18T10:38:05Z

Sorry, @perttus , we ended up postponing this release to 1.3. I just solved a conflict, and we'll merge the PR once the tests pass.

perttus · 2023-10-18T13:55:04Z

Sorry, @perttus , we ended up postponing this release to 1.3. I just solved a conflict, and we'll merge the PR once the tests pass.

No worries, I'm glad to see it in 1.3 then 👍
Thanks for resolving the conflict!

tatiana

Thank you very much for the contribution, @perttus !

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 * Add ProfileMapping for Snowflake encrypted private key path by @ivanstillfront in #608

@perttus

Features * Add `ProfileMapping` for Vertica by @perttus in #540 * Add `ProfileMapping` for Snowflake encrypted private key path by @ivanstillfront in #608 * Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in #616

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 * Add ProfileMapping for Snowflake encrypted private key path by @ivanstillfront in #608 * Add DbtDocsGCSOperator for uploading dbt docs to GCS by @jbandoro in #616

@perttus

Features * Add `ProfileMapping` for Vertica by @perttus in #540 * Add `ProfileMapping` for Snowflake encrypted private key path by @ivanstillfront in #608 * Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in #616

tatiana · 2023-10-27T10:04:13Z

@perttus, this change is available in the 1.3.0a1 pre-release of Cosmos:

Thank you very much for your contribution!

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 and #688 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728 * Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648 * Add operator_args full_refresh as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735 Enhancements * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730 * Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731 * Add aws_session_token for Athena mapping by @benjamin-awd in #663 Others * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 * Speed up integration tests by @jbandoro in #732

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 and #688 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728 * Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648 * Add operator_args full_refresh as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735 Enhancements * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730 * Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731 * Add aws_session_token for Athena mapping by @benjamin-awd in #663 Others * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 * Speed up integration tests by @jbandoro in #732

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in #733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in #728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in #755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in #540, #688 and #741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in #608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in #616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in #648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in #735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in #768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in #730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in #663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in #758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in #759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in #772 **Others** * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Speed up integration tests by @jbandoro in #732 * Fix README quickstart link in by @RNHTTR in #776 * Add package location to work with hatchling 1.19.0 by @jbandoro in #761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in #766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in #757 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 and #779 * pre-commit updates in #775, #770, #762

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in astronomer#755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in astronomer#608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in astronomer#616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in astronomer#648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in astronomer#623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in astronomer#735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in astronomer#768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in astronomer#736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in astronomer#758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in astronomer#759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in astronomer#772 **Others** * Replace flake8 for Ruff by @joppevos in astronomer#743 * Reduce code complexity to 8 by @joppevos in astronomer#738 * Speed up integration tests by @jbandoro in astronomer#732 * Fix README quickstart link in by @RNHTTR in astronomer#776 * Add package location to work with hatchling 1.19.0 by @jbandoro in astronomer#761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in astronomer#766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757 * Update conflict matrix between Airflow and dbt versions by @tatiana in astronomer#731 and astronomer#779 * pre-commit updates in astronomer#775, astronomer#770, astronomer#762

@perttus

Features * Add ProfileMapping for Vertica by @perttus in astronomer#540 and astronomer#688 * Add ProfileMapping for Snowflake encrypted private key path by @ivanstillfront in astronomer#608 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add DbtDocsGCSOperator for uploading dbt docs to GCS by @jbandoro in astronomer#616 Others Rebased on changes released on 1.2.5 (1.3.0a1 was based on 1.2.4)

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in astronomer#755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in astronomer#608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in astronomer#616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in astronomer#648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in astronomer#623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in astronomer#735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in astronomer#768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in astronomer#736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in astronomer#758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in astronomer#759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in astronomer#772 **Others** * Replace flake8 for Ruff by @joppevos in astronomer#743 * Reduce code complexity to 8 by @joppevos in astronomer#738 * Speed up integration tests by @jbandoro in astronomer#732 * Fix README quickstart link in by @RNHTTR in astronomer#776 * Add package location to work with hatchling 1.19.0 by @jbandoro in astronomer#761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in astronomer#766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757 * Update conflict matrix between Airflow and dbt versions by @tatiana in astronomer#731 and astronomer#779 * pre-commit updates in astronomer#775, astronomer#770, astronomer#762

added vertica profile with test

9af20f2

Signed-off-by: Perttu Salonen <perttu.salonen@nitor.com>

perttus requested a review from a team as a code owner September 15, 2023 12:20

perttus requested a review from a team September 15, 2023 12:20

perttus had a problem deploying to external September 15, 2023 12:20 — with GitHub Actions Error

🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

200477b

pre-commit-ci Bot temporarily deployed to external September 15, 2023 12:22 Inactive

tatiana reviewed Sep 15, 2023

View reviewed changes

tatiana added this to the 1.2.0 milestone Sep 15, 2023

added test to cover mock_profile

033f204

Signed-off-by: Perttu Salonen <perttu.salonen@nitor.com>

perttus temporarily deployed to external September 18, 2023 11:44 — with GitHub Actions Inactive

Merge branch 'main' into vertica-profile

08837b3

tatiana temporarily deployed to external September 27, 2023 13:55 — with GitHub Actions Inactive

tatiana modified the milestones: 1.2.0, 1.3.0 Oct 10, 2023

Merge branch 'main' into vertica-profile

0da9e60

tatiana temporarily deployed to external October 18, 2023 10:37 — with GitHub Actions Inactive

tatiana changed the title ~~added vertica profile with test~~ Add vertica profile with test Oct 18, 2023

tatiana approved these changes Oct 18, 2023

View reviewed changes

tatiana merged commit c82e6bc into astronomer:main Oct 18, 2023

tatiana added a commit that referenced this pull request Oct 25, 2023

Release 1.3.0a1

c0a86d4

Features * Add ProfileMapping for Vertica by @perttus in #540 * Add ProfileMapping for Snowflake encrypted private key path by @ivanstillfront in #608

tatiana mentioned this pull request Oct 25, 2023

Release 1.3.0a1 #628

Merged

tatiana mentioned this pull request Nov 23, 2023

Release 1.3.0a2 #711

Merged

tatiana mentioned this pull request Dec 7, 2023

Release 1.3.0a3 #753

Merged

tatiana mentioned this pull request Jan 4, 2024

Release 1.3.0 #781

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vertica profile with test#540

Add vertica profile with test#540
tatiana merged 5 commits into
astronomer:mainfrom
perttus:vertica-profile

perttus commented Sep 15, 2023

Uh oh!

netlify Bot commented Sep 15, 2023 •

edited

Loading

Uh oh!

codecov Bot commented Sep 15, 2023 •

edited

Loading

Uh oh!

tatiana left a comment

Uh oh!

tatiana commented Sep 27, 2023

Uh oh!

tatiana commented Oct 18, 2023

Uh oh!

perttus commented Oct 18, 2023

Uh oh!

tatiana left a comment

Uh oh!

tatiana commented Oct 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

perttus commented Sep 15, 2023

Description

Related Issue(s)

Checklist

Uh oh!

netlify Bot commented Sep 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Uh oh!

codecov Bot commented Sep 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tatiana left a comment

Choose a reason for hiding this comment

Uh oh!

tatiana commented Sep 27, 2023

Uh oh!

tatiana commented Oct 18, 2023

Uh oh!

perttus commented Oct 18, 2023

Uh oh!

tatiana left a comment

Choose a reason for hiding this comment

Uh oh!

tatiana commented Oct 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

netlify Bot commented Sep 15, 2023 •

edited

Loading

codecov Bot commented Sep 15, 2023 •

edited

Loading