AIP-84 Migrate public endpoints from api_connexion to the new rest api

[pierrejeambrun]

Body

#42366 is taking care of migrating private endpoints.

This issue focus on migrating all the public ones. Ideally they shouldn't change too much and be feature rich compare to airflow 2.x.

An exemple PR can be found here #42196

Tasks (total ~ 81 endpoints)

Backfill (6 endpoints) @dstandish

• list_backfills AIP-84 Migrate backfill API to fast api #43496 @dstandish
• get_backfill AIP-84 Migrate backfill API to fast api #43496 @dstandish
• pause_backfill AIP-84 Migrate backfill API to fast api #43496 @dstandish
• unpause_backfill AIP-84 Migrate backfill API to fast api #43496 @dstandish
• cancel_backfill AIP-84 Migrate backfill API to fast api #43496 @dstandish
• create_backfill AIP-84 Migrate backfill API to fast api #43496 @dstandish

Config (2 endpoints) @rawwar

• AIP-84 Migrate Config related public endpoint to FastAPI  #42745 @jason810496

Connection @bugraoz93 (6 endpoints)

• api_connexion/endpoints/connection_endpoint.py delete_connection AIP-84 Migrate the public endpoint Delete Connection to FastAPI #42559 @bugraoz93
• api_connexion/endpoints/connection_endpoint.py get_connection AIP-84 Migrate the public endpoint Get Connection to FastAPI #42590 @rawwar
• api_connexion/endpoints/connection_endpoint.py get_connections AIP-84 Migrate the public endpoint Get Connections to FastAPI  #42591 @bugraoz93
• api_connexion/endpoints/connection_endpoint.py patch_connection AIP-84 Migrate the public endpoint Patch Connection to FastAPI #42592 @bugraoz93
• api_connexion/endpoints/connection_endpoint.py post_connection AIP-84 Migrate the public endpoint Post Connection to FastAPI  #42593 @bugraoz93
• api_connexion/endpoints/connection_endpoint.py test_connection AIP-84 Migrate the public endpoint Test Connection to FastAPI #42594 @bugraoz93

DAG (5 endpoints)

• api_connexion/endpoints/dag_endpoint.py get_dag_details AIP-84 Migrate the public endpoint DAG Details to FastAPI #42453 @omkar-foss
• api_connexion/endpoints/dag_endpoint.py patch_dag AIP-84 Migrate the public endpoint Update DAG to FastAPI #42468 @pierrejeambrun
• api_connexion/endpoints/dag_endpoint.py patch_dags AIP-84 Migrate the public endpoint Batch Update DAG to FastAPI #42544 @pierrejeambrun
• api_connexion/endpoints/dag_endpoint.py get_dag AIP-84 Migrate the public endpoint Get DAG to FastAPI #42652 @omkar-foss
• api_connexion/endpoints/dag_endpoint.py delete_dag AIP-84 Migrate the public endpoint Delete DAG to FastAPI #42650 @omkar-foss

Dag Parsing (1 endpoint) @prabhusneha

Dag Runs @rawwar (9 endpoints)

• All dag run endpointsapi_connexion/endpoints/dag_run_endpoint.py AIP-84 Migrate Dag Run related public endpoint to FastAPI  #42701 @rawwar

Dag Source (1 endpoint)

• AIP-84 Migrate the public endpoint Get DAG Source to FastAPI #42876 @omkar-foss

Dag Stats (1 endpoint)

• AIP-84 Migrate the public endpoint Get DAG Stats to FastAPI #42877 @omkar-foss

Dag Warnings (1 endpoint) @rawwar

• AIP-84 Migrate DagWarning public endpoint to FastAPI #42748 @rawwar

Dataset / Assets (10 endpoints) @amoghrajesh

• get_assets AIP-84 Migrating GET ASSETS Legacy API to fastAPI #43783 @amoghrajesh
• get_asset AIP 84: Migrate GET one ASSET legacy API to fast API #43825 @amoghrajesh
• get_upstream_asset_events AIP-84: Migrating GET Dataset events for DAG runs api to fastAPI #43874 @amoghrajesh
• get_asset_events AIP 84: Migrate GET ASSET EVENTS legacy API to fast API  #43881 @vatsrahul1001
• create_asset_event AIP 84: Migrate POST ASSET EVENT legacy API to fast API.  #43984 @vatsrahul1001
• get_asset_queued_events AIP-84: Migrating GET queued asset events for assets to FASTAPI #44139 @vatsrahul1001
• delete_asset_queued_events AIP-84: Migrating DELETE queued asset events for assets to FASTAPI #44138 @vatsrahul1001
• get_dag_asset_queued_events AIP-84: Migrating GET queued asset events for DAG to fastAPI #44124 @amoghrajesh
• delete_dag_asset_queued_events AIP-84: Migrating DELETE queued asset events for DAG to fastAPI #44129 @amoghrajesh
• delete_dag_asset_queued_event AIP-84: Migrating DELETE a queued asset events for DAG to fastAPI #44130 @amoghrajesh
• get_dag_asset_queued_event AIP-84: Migrating GET one queued asset events for DAG to fastAPI #44128 @amoghrajesh

Event Logs (2 endpoints)

• AIP-84 Migrate the public endpoint Get Event Logs to FastAPI #43326 @jason810496
• AIP-84 Migrate the public endpoint Get Event Log to FastAPI #43327 @jason810496

Extra Link (1 endpoint) @prabhusneha

Monitor (2 endpoints)

• AIP-84 Migrate public get Health info to FastAPI #42937 @bbovenzi
• AIP-84 Migrate the public endpoint Get Airflow Version Info to FastAPI #42879 @omkar-foss

Import Error (2 endpoints) @jason810496

• AIP-84 Migrate public endpoint Get Import Error to FastAPI #43594 @jason810496
  AIP-84 Migrate public endpoint to list import errors to FastAPI #43595 @jason810496

Log (1 endpoint) @utkarsharma2

Plugin (1 endpoint)

• AIP-84 Get Plugins #43125 @pierrejeambrun

Pool (5 endpoints) @pierrejeambrun

• AIP-84 Delete Pool #43165 @pierrejeambrun
• AIP-84 Get Pool #43221 @pierrejeambrun
• AIP-84 Get Pools #43223 @pierrejeambrun
• AIP-84 Patch Pool #43266 @pierrejeambrun
• AIP-84 Post Pool #43317 @pierrejeambrun

Provider (1 endpoint)

• AIP-84 Get Providers #43159 @pierrejeambrun

Task (2 endpoints) @omkar-foss

• AIP-84 Migrate the public endpoint Get Task to FastAPI #42874 @omkar-foss
• AIP-84 Migrate the public endpoint Get Tasks to FastAPI #42875 @omkar-foss

Task Instance (15 endpoints) @pierrejeambrun

• AIP-84 Get Task Instance #43485 @pierrejeambrun
• AIP-84 Get Mapped Task Instance #43548 @pierrejeambrun
• AIP-84 List Mapped Task Instances #43642 @pierrejeambrun
• AIP-84 Get Task Instances #43748 @pierrejeambrun
• AIP-84 Get Task Instance Dependencies #43749 @pierrejeambrun
• AIP-84 Get Mapped Task Instance Dependencies #43750 @pierrejeambrun
• AIP-84 Get Task Instances Batch #43756 @pierrejeambrun
• AIP-84 Post Clear Task Instances #43751 @omkar-foss
• AIP-84 Patch Task Instance #43753 @omkar-foss
• AIP-84 Patch Mapped Task Instance #43754 @omkar-foss
• AIP-84 Set Task Instances State #43752 @omkar-foss
• get_task_instance_try_details AIP-84 Get Task Instance Try Details #43675 @kandharvishnu
• get_mapped_task_instance_try_details AIP-84 Get Mapped Task Instance Try Details #44206 @kandharvishnu
• get_task_instance_tries @kandharvishnu
• get_mapped_task_instance_tries @kandharvishnu

Variables (5 endpoints)

• AIP-84 Delete Variable #42798 @pierrejeambrun
• AIP-84 Get Variable #42834 @pierrejeambrun
• AIP-84 Patch Variable #42929 @pierrejeambrun
• AIP-84 Post variable #42948 @pierrejeambrun
• AIP-84 Get Variables #43083 @pierrejeambrun

XCom (2 endpoints)

• AIP-84 Migrate XCom get entry public endpoint to FastAPI #42978 @michaeljs-c
• AIP-84 Migrate XCom get entries public endpoint to FastAPI #42980 @michaeljs-c

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[omkar-foss]

Trimming down views.py is very tempting 😄

May I pick this up? If yes, please assign this issue to me. Thanks!

[pierrejeambrun]

Hello @omkar-foss,

views.py are mostly 'rendered' front end that will get replaced by the new UI react, in views.py you also find private/UI REST route, most of them are with the /object/ prefix, such as def grid_data. If you want to work on those UI dedicated endpoints you can that would be part of #42366.

This issue focuses on the public endpoints, those are located under the api_connexion folder.

In anycase, just mention the endpoint you are working on, so people do not do the same ones, and I will update the description to track that :). I think we should do one at the time.

[omkar-foss]

Hi @pierrejeambrun, thank you for explaining it to me, much appreciate it.

I'll start migrating the public endpoints, beginning with migration of the DAG Details API (get_dag_details) in api_connexion/endpoints/dag_endpoint.py.

Since this is a meta issue (missed that previously!), I'll create the issue/PR corresponding to the endpoint(s) that I'll be working on and link to this issue (like usual).

P.S: views.py will have to wait :)

[bugraoz93]

Thanks for creating the ticket @pierrejeambrun! Let me take connection endpoints

[pierrejeambrun]

Great, thanks @bugraoz93

[bugraoz93]

Thanks for assigning! Let me save you from regular updates :) I created issues for all the endpoints in connections.

[pierrejeambrun]

Great, thanks @bugraoz93. I just updated the meta issue with those new additions. Let me know if you plan to work on all of them so I can assign you, or just some of them so I can label the others with good first issue and let other contributors grab them :)

[bugraoz93]

I am planning to work on all of them @pierrejeambrun. if you can assign them to me, that would be great! Thanks :)

[omkar-foss]

Hey folks! After #42631 (open for review), I can pick up these two next in order:

1. get_dag - AIP-84 Migrate the public endpoint Get DAG to FastAPI #42652
2. delete_dag - AIP-84 Migrate the public endpoint Delete DAG to FastAPI #42650

With these 2 done, I suppose we should have the dags.py migration complete. When you find some time, please update this meta and assign these to me, thank you! 😃

[bbovenzi]

@omkar-foss Assigned. Thanks for picking these issues up!

[bbovenzi]

@prabhusneha I see you're assigned to Logs. What is the status of getting the logs for a task instance? That would be a useful one to get in the UI soon!

[utkarsharma2]

@prabhusneha I see you're assigned to Logs. What is the status of getting the logs for a task instance? That would be a useful one to get in the UI soon!

Hey, @bbovenzi I'm taking care of task instance logs endpoint, I'm mostly done with it, but there are a couple of test cases that require setting users in fastapi which requires a little more work. I'll keep you posted.

[phanikumv]

Thanks @utkarsharma2 (updated the description accordingly).