Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate public endpoint Clear Task Instances to FastAPI #44220

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

omkar-foss
Copy link
Collaborator

closes: #43751
related: #42370

This migrates the Clear Task Instances API from api_connexion to api_fastapi.

@omkar-foss omkar-foss self-assigned this Nov 20, 2024
@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Nov 20, 2024
@omkar-foss omkar-foss added legacy api Whether legacy API changes should be allowed in PR and removed area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Nov 20, 2024
@omkar-foss omkar-foss closed this Nov 20, 2024
@omkar-foss omkar-foss reopened this Nov 20, 2024
@omkar-foss omkar-foss marked this pull request as ready for review November 20, 2024 17:49
@omkar-foss
Copy link
Collaborator Author

Resolving conflicts from recent PR merges into main.

@omkar-foss
Copy link
Collaborator Author

Resolving conflicts from recent PR merges into main.

This is done, PR synced with main and all conflicts resolved ✅

@omkar-foss
Copy link
Collaborator Author

All checks passed, PR ready to review ✅

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

A few suggestions (only #44220 (comment) requires change), we can merge after :)

airflow/api_fastapi/common/types.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/datamodels/task_instances.py Outdated Show resolved Hide resolved
Comment on lines +41 to +47
@pytest.fixture(scope="module")
def dagbag():
from airflow.models import DagBag

dagbag_instance = DagBag(include_examples=True, read_dags_from_db=False)
dagbag_instance.sync_to_db()
return dagbag_instance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you put the fixture at the conftest.py level, do you mind also using it in other tests, and replace.

Maybe test_dag in test_dag_sources.py can reuse this fixture:

@pytest.fixture
def test_dag(dagbag):
    return dagbag.dags[TEST_DAG_ID]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this works, but if you don't mind, shall I pick this up in a separate PR? Have some other cleanup to do in tests so can do it together if that's okay.

@omkar-foss
Copy link
Collaborator Author

omkar-foss commented Nov 21, 2024

Looking good!

A few suggestions (only #44220 (comment) requires change), we can merge after :)

Thanks @pierrejeambrun, I've made the changes as per your suggestions and resolved the corresponding conversations. Also rebased with main. Please have a look, thanks! 😁

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One extra nit, can be merged after that.

class TaskInstanceReferenceCollectionResponse(BaseModel):
"""Task Instance Reference collection serializer for responses."""

task_instances: list[TaskInstanceReferenceResponse]
Copy link
Member

@pierrejeambrun pierrejeambrun Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency with other Collection Datamodel, can you add total_entries: int as well please. (and in the response as well)

It's not here in the legacy one but I think it should be added to stay consistent with the rest of the API.

task_id: str
dag_run_id: str = Field(validation_alias="run_id")
dag_id: str
logical_date: datetime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I'm not sure we want or need logical_date to be here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
legacy api Whether legacy API changes should be allowed in PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AIP-84 Post Clear Task Instances
3 participants