Skip to content

Fix incorrect temporary directory creation in VirtualenvOperator init#500

Merged
tatiana merged 1 commit into
mainfrom
issue-484
Aug 30, 2023
Merged

Fix incorrect temporary directory creation in VirtualenvOperator init#500
tatiana merged 1 commit into
mainfrom
issue-484

Conversation

@tatiana
Copy link
Copy Markdown
Collaborator

@tatiana tatiana commented Aug 30, 2023

As part of adding MyPy checks, we introduced an issue of creating temporary directories during the VirtualenvOperator initialization. This leads to Cosmos creating unnecessary empty temporary directories in the user scheduler during parse time.

Fixes: #484

@tatiana tatiana requested a review from a team as a code owner August 30, 2023 15:28
@tatiana tatiana requested a review from a team August 30, 2023 15:28
@netlify
Copy link
Copy Markdown

netlify Bot commented Aug 30, 2023

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name Link
🔨 Latest commit 6590b80
🔍 Latest deploy log https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/64ef6022974c080008c3b51f

@tatiana tatiana temporarily deployed to internal August 30, 2023 15:28 — with GitHub Actions Inactive
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 30, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.59%. Comparing base (53dbb38) to head (6590b80).
Report is 618 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #500   +/-   ##
=======================================
  Coverage   91.59%   91.59%           
=======================================
  Files          50       50           
  Lines        1857     1857           
=======================================
  Hits         1701     1701           
  Misses        156      156           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tatiana tatiana merged commit 9387c45 into main Aug 30, 2023
@tatiana tatiana deleted the issue-484 branch August 30, 2023 16:03
@tatiana tatiana added this to the 1.1.0 milestone Aug 31, 2023
@tatiana tatiana mentioned this pull request Sep 6, 2023
tatiana added a commit that referenced this pull request Sep 6, 2023
**Features**

* Support dbt global flags (via dbt_cmd_global_flags in operator_args)
by @tatiana in #469
* Support parsing DAGs when there are no connections by @jlaneve in #489

**Enhancements**

* Hide sensitive field when using BigQuery keyfile_dict profile mapping
by @jbandoro in #471
* Consistent Airflow Dataset URIs, inlets and outlets with `Openlineage
package <https://pypi.org/project/openlineage-integration-common/>`_ by
@tatiana in #485. `Read more
<https://astronomer.github.io/astronomer-cosmos/configuration/lineage.html>`_.
* Refactor ``LoadMethod.DBT_LS`` to run from a temporary directory with
symbolic links by @tatiana in #488
* Run ``dbt deps`` when using ``LoadMethod.DBT_LS`` by @DanMawdsleyBA in
#481
* Update Cosmos log color to purple by @harels in #494
* Change operators to log ``dbt`` commands output as opposed to
recording to XCom by @tatiana in #513

**Bug fixes**

* Fix bug on select node add exclude selector subset ids logic by
@jensenity in #463
* Refactor dbt ls to run from a temporary directory, to avoid Read-only
file system errors during DAG parsing, by @tatiana in #414
* Fix profile_config arg in DbtKubernetesBaseOperator by @david-mag in
#505
* Fix SnowflakePrivateKeyPemProfileMapping private_key reference by
@nacpacheco in #501
* Fix incorrect temporary directory creation in VirtualenvOperator init
by @tatiana in #500
* Fix log propagation issue by @tatiana in #498
* Fix PostgresUserPasswordProfileMapping to retrieve port from
connection by @jlneve in #511

**Others**

* Docs: Fix RenderConfig load argument by @jbandoro in #466
* Enable CI integration tests from external forks by @tatiana in #458
* Improve CI tests runtime by @tatiana in #457
* Change CI to run coverage after tests pass by @tatiana in #461
* Fix forks code revision in code coverage by @tatiana in #472
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #467
* Drop support to Python 3.7 in the CI test matrix by @harels in #490
* Add Airflow 2.7 to the CI test matrix by @tatiana in #487
* Add MyPy type checks to CI since we exceeded pre-commit disk quota
usage by @tatiana in #510
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Disk space leak due to empty folders creation

2 participants