Initial POC of GRPC Internal API #25094
Conversation
boring-cyborg
bot
added
area:CLI
area:dev-tools
area:Scheduler
airflow/dag_processing/processor.py
Outdated
There was a problem hiding this comment.
Only needed to avoid errors for callbacks. I did not want to complicate it passing the "right" callbacks so "fake" callbacks I am passing now generate a lot of exceptions as dags/dag runs are not found,
airflow/dag_processing/processor.py
Outdated
There was a problem hiding this comment.
This is the only change needed for "DB Aware" classes with logic to make them GRPC Aware"
There was a problem hiding this comment.
Nice Mypy stub :). ❤️ to createors of mypy-protobuf.
airflow/internal_api/server.py
Outdated
There was a problem hiding this comment.
This "servicer" has to be implemented for every "DB Aware class" to forward gRPC calls to the right "DB enabled" implementation.
airflow/models/dagbag.py
Outdated
There was a problem hiding this comment.
This is the "small" caveat that we will have to change slightly. GRPC runs in non-main thread so signals are not working for it, we will have to implement timeout a bit differently.
There was a problem hiding this comment.
This method (load a dag from a file) should never be used remotely anyway, should it?
potiuk
force-pushed
the
internal-api-poc
branch
from
72164de to
40a0683
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
github-actions
bot
added
the
stale
|
Not stale. |
github-actions
bot
removed
the
stale
|
Closing as the founding PR for AIP-44 was already merged #27892 |
2 participants
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.