Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIP-72: Add support for fetching variables and connections in Supervisor #44229

Merged
merged 1 commit into from
Nov 21, 2024

Conversation

kaxil
Copy link
Member

@kaxil kaxil commented Nov 21, 2024

  • Updated VariableOperations and ConnectionOperations in Client:
    • Added get methods for fetching variable and connection details.
  • Refactored communication protocol (comms.py):
    • Unified result models (ConnectionResult, VariableResult, XComResult) extending auto-generated models.
    • Renamed ReadXCom to GetXCom for consistency with other request models (GetConnection, GetVariable).
  • Updated WatchedSubprocess in supervisor.py:
    • Integrated handle_requests to process GetVariable and GetConnection messages.

and some minor refactors -- check inline comments


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@kaxil kaxil added the area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK label Nov 21, 2024
@kaxil kaxil requested a review from ashb November 21, 2024 01:39
@kaxil kaxil force-pushed the add-variables-connections-task-sdk branch from 0175650 to 85d1bb0 Compare November 21, 2024 01:41
@kaxil kaxil force-pushed the add-variables-connections-task-sdk branch from 85d1bb0 to 96068da Compare November 21, 2024 01:53
Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review only - I need to look at test_handle_request on a bigger screen

task_sdk/tests/api/test_client.py Outdated Show resolved Hide resolved
task_sdk/tests/execution_time/test_supervisor.py Outdated Show resolved Hide resolved
@kaxil kaxil force-pushed the add-variables-connections-task-sdk branch from 96068da to c6e14bb Compare November 21, 2024 11:08
@kaxil kaxil force-pushed the add-variables-connections-task-sdk branch 2 times, most recently from d283723 to 42ddd22 Compare November 21, 2024 16:06
- Updated `VariableOperations` and `ConnectionOperations` in `Client`:
  - Added `get` methods for fetching variable and connection details.
- Refactored communication protocol (`comms.py`):
  - Unified result models (`ConnectionResult`, `VariableResult`, `XComResult`) extending auto-generated models.
  - Renamed `ReadXCom` to `GetXCom` for consistency with other request models (`GetConnection`, `GetVariable`).
- Updated `WatchedSubprocess` in `supervisor.py`:
  - Integrated `handle_requests` to process `GetVariable` and `GetConnection` messages.

and some minor refactors
@kaxil kaxil force-pushed the add-variables-connections-task-sdk branch from 42ddd22 to bfba2b9 Compare November 21, 2024 16:45
@kaxil kaxil merged commit 1307e37 into apache:main Nov 21, 2024
42 checks passed
@kaxil kaxil deleted the add-variables-connections-task-sdk branch November 21, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK area:task-sdk
Development

Successfully merging this pull request may close these issues.

2 participants