Skip to content

Install requirements.txt by default during dev env spin up#1099

Merged
tatiana merged 2 commits into
astronomer:mainfrom
CorsettiS:Improve-dev-experience
Jul 17, 2024
Merged

Install requirements.txt by default during dev env spin up#1099
tatiana merged 2 commits into
astronomer:mainfrom
CorsettiS:Improve-dev-experience

Conversation

@CorsettiS
Copy link
Copy Markdown
Contributor

@CorsettiS CorsettiS commented Jul 17, 2024

Description

I bumped the astronomer image to 11.6 since it uses the newest airflow, incorporating tatiana's change in airflow version for datasets and I am installing requirements.txt by default in the image, since if the user wants to add some custom package, he needs to change not only the requirements.txt file but also the Dockerfile. With this only adding dependencies to requirements.txt will already install them in the image for dev development.

Related Issue(s)

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jul 17, 2024
@netlify
Copy link
Copy Markdown

netlify Bot commented Jul 17, 2024

Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name Link
🔨 Latest commit 0f6228a
🔍 Latest deploy log https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/66985501273b110008dd3d7c

@dosubot dosubot Bot added area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc execution:docker Related to Docker execution environment labels Jul 17, 2024
@tatiana
Copy link
Copy Markdown
Collaborator

tatiana commented Jul 17, 2024

Thanks for fixing this, @CorsettiS !

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jul 17, 2024
@tatiana tatiana merged commit 87b2b14 into astronomer:main Jul 17, 2024
@tatiana tatiana added this to the Cosmos 1.6.0 milestone Jul 17, 2024
@corsettigyg corsettigyg deleted the Improve-dev-experience branch July 18, 2024 06:40
@pankajkoti pankajkoti mentioned this pull request Jul 31, 2024
pankajkoti added a commit that referenced this pull request Aug 20, 2024
New Features

* Add support for loading manifest from cloud stores using Airflow
Object Storage by @pankajkoti in #1109
* Cache ``package-lock.yml`` file by @pankajastro in #1086
* Support persisting the ``LoadMode.VIRTUALENV`` directory by @tatiana
in #1079
* Add support to store and fetch ``dbt ls`` cache in remote stores by
@pankajkoti in #1147
* Add default source nodes rendering by @arojasb3 in #1107
* Add Teradata ``ProfileMapping`` by @sc250072 in #1077

Enhancements

* Add ``DatabricksOauthProfileMapping`` profile by @CorsettiS in #1091
* Use ``dbt ls`` as the default parser when ``profile_config`` is
provided by @pankajastro in #1101
* Add task owner to dbt operators by @wornjs in #1082
* Extend Cosmos custom selector to support + when using paths and tags
by @mvictoria in #1150
* Simplify logging by @dwreeves in #1108

Bug fixes

* Fix Teradata ``ProfileMapping`` target invalid issue by @sc250072 in
#1088
* Fix empty tag in case of custom parser by @pankajastro in #1100
* Fix ``dbt deps`` of ``LoadMode.DBT_LS`` should use
``ProjectConfig.dbt_vars`` by @tatiana in #1114
* Fix import handling by lazy loading hooks introduced in PR #1109 by
@dwreeves in #1132
* Fix Airflow 2.10 regression and add Airflow 2.10 in test matrix by
@pankajastro in #1162

Docs

* Fix typo in azure-container-instance docs by @pankajastro in #1106
* Use Airflow trademark as it has been registered by @pankajastro in
#1105

Others

* Run some example DAGs in Kubernetes execution mode in CI by
@pankajastro in #1127
* Install requirements.txt by default during dev env spin up by
@@CorsettiS in #1099
* Remove ``DbtGraph.current_version`` dead code by @tatiana in #1111
* Disable test for Airflow-2.5 and Python-3.11 combination in CI by
@pankajastro in #1124
* Pre-commit hook updates in #1074, #1113, #1125, #1144, #1154,  #1167

---------

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com>
tatiana added a commit that referenced this pull request Sep 18, 2025
[Giovanni Corsetti
Silva](https://www.linkedin.com/in/giovanni-corsetti/) (@corsettigyg
@CorsettiS @giovannicorsetti) is a Core Data Platform Data Engineer at
[Get Your Guide](https://www.getyourguide.com/). He's a Brazilian based
in Berlin (Germany) and has been an early adopter of Cosmos, using it
regularly throughout his career.

Not only has he been using Cosmos since the early stages, but he has
consistently improved Cosmos since April 2023:

<img width="494" height="249" alt="Screenshot 2025-09-17 at 16 39 49"
src="https://github.com/user-attachments/assets/ff9b0055-7df6-47a4-8efa-83a30d5aee99"
/>
] m. ,
<img width="492" height="252" alt="Screenshot 2025-09-17 at 16 40 11"
src="https://github.com/user-attachments/assets/8402d883-eff4-46aa-9ae7-afe35bd39a6e"
/>

<img width="499" height="256" alt="Screenshot 2025-09-17 at 16 39 23"
src="https://github.com/user-attachments/assets/c6b84245-aae1-46e6-99e3-552553619dd9"
/>

His contributions include new features, enhancements, bug fixes and also
improvements on the local dev setup, as it can see in some of the many
contributions he's done to the project so far:

* #1787
* #1761
* #1695 
* #1693 
* #1663
* #1571
* #1449
* #1099
* #1091
* #419
* #402
* #355 
* #354 
* #225 

Additionally, he has been interacting with users in the #airflow-dbt
Slack channel in a very collaborative and supportive way.

We want to promote him as a Cosmos committer and maintainer for all
these, recognising his constant efforts and achievements towards our
community. Thank you very much, @corsettigyg!

From this moment, you have write permissions on the repo:
<img width="884" height="93" alt="Screenshot 2025-09-18 at 14 34 07"
src="https://github.com/user-attachments/assets/abcc8c42-87cf-4b32-a7be-81127190d164"
/>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc execution:docker Related to Docker execution environment lgtm This PR has been approved by a maintainer size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants