-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[REFACTOR] Introduce and modernize ffi system #17920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…ptional for as This helps us to simplify the API deps while still leverage compact memory of Optional<T> for storing data structures and parameter passing.
|
The changes started from an initial code module implementation in collaboration with @junrushao one year ago. Then it independently evolved driven by the needs of a full upgrade in tree, while Upgrade NoteSome upgrade note of the dependent code
As we bring followup changes to adapt to the new FFI, it might be useful to first rebase to this particular commit, that contains redirection sccafolds, e.g. PackedFunc is redirected to ffi::Function, get the code to compile then add following comments. |
|
If there are issues with rebase merge(maybe a gh issue), we can also do squash merge if needed so the commit track back to the pr |
MasterJH5574
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Selectively went through the PR
yzh119
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking forward to seeing tvm-ffi evolution!
This PR modernizes the FFI foundation of the project and introduce a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi) based on our lessons in the past few years. It implements a modern version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md) that unifies the packed function call and object systems. Summary of the change: - A dedicated clean Any/AnyView that can store strong and weak references of items - Function(previously PackedFunc) system built on top of the Any/AnyView - A minimal C API that backs the overall calls. We are stabilizing the API with a goal to bring clean, stable FFI conventions for both compiled and registered code - A rewrite of core python binding and generated code based on the module - Update existing code and test cases to the new module - Latest dlpack support The new module brings many benefits thanks to the cleaner design, to name a few: - Any can support both POD types(int) and object types. - Containers (e.g. Array) can now also contain Any value, e.g. now `Array<int>` is supported, no need for boxed types - Error handling now upgrades to object-based, allowing cleaner traceback across languages - Map now preserves insertion orders - Path toward isolated stabilize minimum core ABI/API foundation module - Type traits based design that cleanly defines how values interact with Any system - Automatic conversion of different types based on traits if needed Because FFI upgrade is at heart of the project, the change touches every component of the system. Importantly, this is an upgrade of the ABI so the change is not backward compatible. The code compiled under the old FFI won't work under the new one. We did provide example ABI translation (e.g. LegacyTVMArgValueToFFIAny) functions for compatibility. The PR tries to leave files in their old places while creating redirections. The goal is to have the first milestone landed and infrastructure in place, so we can do further refactors to complete features and cleanup legacy code as trackable PRs. As of now, python binding and compiled code are under the new convention while RPC and some other bindings still relies on legacy ABI translation. We will work on upgrades in the coming PRs, including areas such as reflection, phasing out legacy redirections etc.
|
#18226 contains a followup that modularize the ffi module into its own standalone python package |
This PR modernizes the FFI foundation of the project and introduce a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi) based on our lessons in the past few years. It implements a modern version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md) that unifies the packed function call and object systems. Summary of the change: - A dedicated clean Any/AnyView that can store strong and weak references of items - Function(previously PackedFunc) system built on top of the Any/AnyView - A minimal C API that backs the overall calls. We are stabilizing the API with a goal to bring clean, stable FFI conventions for both compiled and registered code - A rewrite of core python binding and generated code based on the module - Update existing code and test cases to the new module - Latest dlpack support The new module brings many benefits thanks to the cleaner design, to name a few: - Any can support both POD types(int) and object types. - Containers (e.g. Array) can now also contain Any value, e.g. now `Array<int>` is supported, no need for boxed types - Error handling now upgrades to object-based, allowing cleaner traceback across languages - Map now preserves insertion orders - Path toward isolated stabilize minimum core ABI/API foundation module - Type traits based design that cleanly defines how values interact with Any system - Automatic conversion of different types based on traits if needed Because FFI upgrade is at heart of the project, the change touches every component of the system. Importantly, this is an upgrade of the ABI so the change is not backward compatible. The code compiled under the old FFI won't work under the new one. We did provide example ABI translation (e.g. LegacyTVMArgValueToFFIAny) functions for compatibility. The PR tries to leave files in their old places while creating redirections. The goal is to have the first milestone landed and infrastructure in place, so we can do further refactors to complete features and cleanup legacy code as trackable PRs. As of now, python binding and compiled code are under the new convention while RPC and some other bindings still relies on legacy ABI translation. We will work on upgrades in the coming PRs, including areas such as reflection, phasing out legacy redirections etc.
This PR modernizes the FFI foundation of the project and introduce a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi) based on our lessons in the past few years. It implements a modern version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md) that unifies the packed function call and object systems. Summary of the change: - A dedicated clean Any/AnyView that can store strong and weak references of items - Function(previously PackedFunc) system built on top of the Any/AnyView - A minimal C API that backs the overall calls. We are stabilizing the API with a goal to bring clean, stable FFI conventions for both compiled and registered code - A rewrite of core python binding and generated code based on the module - Update existing code and test cases to the new module - Latest dlpack support The new module brings many benefits thanks to the cleaner design, to name a few: - Any can support both POD types(int) and object types. - Containers (e.g. Array) can now also contain Any value, e.g. now `Array<int>` is supported, no need for boxed types - Error handling now upgrades to object-based, allowing cleaner traceback across languages - Map now preserves insertion orders - Path toward isolated stabilize minimum core ABI/API foundation module - Type traits based design that cleanly defines how values interact with Any system - Automatic conversion of different types based on traits if needed Because FFI upgrade is at heart of the project, the change touches every component of the system. Importantly, this is an upgrade of the ABI so the change is not backward compatible. The code compiled under the old FFI won't work under the new one. We did provide example ABI translation (e.g. LegacyTVMArgValueToFFIAny) functions for compatibility. The PR tries to leave files in their old places while creating redirections. The goal is to have the first milestone landed and infrastructure in place, so we can do further refactors to complete features and cleanup legacy code as trackable PRs. As of now, python binding and compiled code are under the new convention while RPC and some other bindings still relies on legacy ABI translation. We will work on upgrades in the coming PRs, including areas such as reflection, phasing out legacy redirections etc. Co-authored-by: Junru Shao <[email protected]>
This PR modernizes the FFI foundation of the project and introduce a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi) based on our lessons in the past few years. It implements a modern version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md) that unifies the packed function call and object systems. Summary of the change: - A dedicated clean Any/AnyView that can store strong and weak references of items - Function(previously PackedFunc) system built on top of the Any/AnyView - A minimal C API that backs the overall calls. We are stabilizing the API with a goal to bring clean, stable FFI conventions for both compiled and registered code - A rewrite of core python binding and generated code based on the module - Update existing code and test cases to the new module - Latest dlpack support The new module brings many benefits thanks to the cleaner design, to name a few: - Any can support both POD types(int) and object types. - Containers (e.g. Array) can now also contain Any value, e.g. now `Array<int>` is supported, no need for boxed types - Error handling now upgrades to object-based, allowing cleaner traceback across languages - Map now preserves insertion orders - Path toward isolated stabilize minimum core ABI/API foundation module - Type traits based design that cleanly defines how values interact with Any system - Automatic conversion of different types based on traits if needed Because FFI upgrade is at heart of the project, the change touches every component of the system. Importantly, this is an upgrade of the ABI so the change is not backward compatible. The code compiled under the old FFI won't work under the new one. We did provide example ABI translation (e.g. LegacyTVMArgValueToFFIAny) functions for compatibility. The PR tries to leave files in their old places while creating redirections. The goal is to have the first milestone landed and infrastructure in place, so we can do further refactors to complete features and cleanup legacy code as trackable PRs. As of now, python binding and compiled code are under the new convention while RPC and some other bindings still relies on legacy ABI translation. We will work on upgrades in the coming PRs, including areas such as reflection, phasing out legacy redirections etc.
This PR modernizes the FFI foundation of the project and introduce a new minimal and lightweight module tvm ffi based on our lessons in the past few years. It implements a modern version of the Unified Packed and Object RFC that unifies the packed function call and object systems.
Summary of the change:
The new module brings many benefits thanks to the cleaner design, to name a few:
Array<int>is supported, no need for boxed typesBecause FFI upgrade is at heart of the project, the change touches every component of the system. Importantly, this is an upgrade of the ABI so the change is not backward compatible. The code compiled under the old FFI won't work under the new one. We did provide example ABI translation (e.g. LegacyTVMArgValueToFFIAny) functions for compatibility.
The PR tries to leave files in their old places while creating redirections. The goal is to have the first milestone landed and infrastructure in place, so we can do further refactors to complete features and cleanup legacy code as trackable PRs. As of now, python binding and compiled code are under the new convention while RPC and some other bindings still relies on legacy ABI translation. We will work on upgrades in the coming PRs, including areas such as reflection, phasing out legacy redirections etc.