-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
☂️ [vm/ffi] Native assets feature #50565
Comments
Hi, as a current user of option number 4 above, this sounds really enticing. Right now I am using CMake for Linux/Windows/Android and Ruby (with inline bash) for macOS/iOS to download & extract binaries and it is a significant pain point. A few questions:
|
Hi @GregoryConrad, nice to meet you!
Thanks for bearing with us in the mean time! 🙃
We will start with dynamic loading: In AOT ( I have not explored using the native linker to do dynamic loading. I'm not sure if that will be of any benefit in any Dart or Flutter usage scenario. @GregoryConrad do you have a scenario in mind where you would prefer using dynamic loading?
I cannot comment on that. We'd want to ship the feature when it's ready, not earlier.
We're in discussions about that, but likely yes. We have two options, and might want to support both:
Best case scenario we could have a unified API for interacting with the C library on native platforms and a to WASM compiled library on the web: Further discussion can be on that issue. |
I don't actually have a case for using dynamic linking over dynamic loading, or vice versa. When I said:
I was more concerned with making sure we would be able to choose between static linking and dynamic linking/loading per platform. If someone else has a use case, however, I would be happy to hear it.
Although I like the idea here a lot, I foresee some issues with this approach. Especially in my project, some custom build steps and quite a few other package dependencies (besides just And thanks for the rest of your input! Much appreciated. |
Just for the record, as mentioned in my flutter issue, I think the workflow for writing C/C++ plugins should be made simpler.
One way this could be achieved is to use CMakePresets.json to configure cmake for each platform. |
@eladmaimoni Plugins are a Flutter-specific concept, as is the template you are referring to. |
The build for native assets would be disconnected from the Flutter native build. Flutter would only receive a list of static and dynamic libraries that need to be bundle.
Even using CMake would be an abstraction in this approach. We haven't settled yet on whether we mandate compilers, but with trying to support Go and Rust as well, constraining C/C++ to a single build system would also be constraining. How various compilers would be supported (for example with Dart packages that wrap them) is still up for discussion. The way that this is achieved is that we interact through a CLI (likely
This is trivial once the build is an arbitrary Dart script. One of my examples is a completely cross platform CMake build.
Native assets is a completely different approach. Zero Flutter template "boilerplate" will be included. These will be Dart standalone packages that will have their compiled dynamic libraries properly build and bundled with Flutter apps. |
I know I've mentioned this in other contexts, but we'll need to be very careful about getting this right; the ecosystem effects if we don't have a robust way to manage the toolchain dependencies automatically could be very unpleasant. |
If you are familiar with Rust, is what you are describing here similar to
I almost wonder if there should be a standard way to package native assets at all then (other than being able to upload to pub.dev, which would be great); if we have build scripts then we can decide which way works best for our projects based on our own build systems (e.g., However, one obvious drawback is that the build script could access anything on the system. I am not too worried about this issue though, as any Flutter package today can do the same without any warning to the user and other ecosystems such as Rust haven't had many troubles with this--you just simply need to trust the packages you use. |
The VM can read native asset mappings from kernel. Previous CLs already added support to embed native asset mappings for one-shot compilation. This CL adds support for adding native assets mappings to kernel files created by the frontend_server with the incremental compiler. The frontend_server accepts a `--native-assets=<uri>` at startup and accepts a `native-assets <uri>` message on stdin as compilation command. The frontend_server caches the compiled native assets library. When a `reset` command is sent to request a full dill from the incremental compiler, the native assets mapping is taken from the cache and added to the final dill file. Split of DartSDK & flutter_tools prototype to land separately. TEST=pkg/frontend_server/test/native_assets_test.dart Bug: #49803 Bug: #50565 Change-Id: I6e15f177564b8a962e81261815e951e7c9525513 Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282101 Reviewed-by: Jens Johansen <[email protected]> Commit-Queue: Daco Harkes <[email protected]>
Thank you for the information and excited to see those improvements!
I see, then I may run it on my own machine (at the same time it would be great to see google's numbers to check whether I did something wrong).
Not yet, but I am planning to do some benchmarking after finishing most of the WIP flutter_rust_bridge 2.0 ;) |
EDIT: Moved to a separate issue #54132 Original content: Hi, when running P.S. I am (currently) using Shell output:
Also posted as a separate issue (since I am not sure whether it belongs to this thread or should be another separate thread): #54132 |
Lets keep this thread as the umbrella issue and discuss specifics in specific issues. |
Adds documentation for the native assets feature. * dart-lang/sdk#50565 --------- Co-authored-by: Marya <[email protected]> Co-authored-by: Parker Lougheed <[email protected]>
Native assets support in Dart standalone is available in Dart 3.2 behind an experimental flag:
dart --enable-experiment=native-assets run [path/to/script.dart]
.This is a high level "feature" issue that we can refer to from various places.
The current way of bundling native code with Dart packages is far from ideal, we will introduce "Native Assets" to Dart.
Problem
As of right now, one can do one (or more) of the following:
dart run <your-package>/setup.dart
see webcrypto, orbin/setup.dart
download the native library, orflutter create --template=plugin_ffi --platform=...
, orThis leads to pain for both package developers (options 1-4) and package users (option 1-2). Moreover, there is currently no way to make a package with native code that works both for flutter and Dart standalone.
High level solution
We introduce the concept of "Native Assets" to the Dart eco system.
A native asset is a native library (compiled from native source code).
Three components of the "Native Assets" solution in Dart:
dart
andflutter
tooling to link the above two together.Try now
Native assets support in Dart standalone is available in Dart 3.2 behind an experimental flag :
dart --enable-experiment=native-assets run [path/to/script.dart]
.Native assets support in Flutter is in development. Progress can be tracked in flutter/flutter#129757.
An example on how to use native assets can be found here.
Checklist
FfiNative
s #49803FfiNative
s more concise #50097FfiNative
for static linking #49418FfiNative
return types treatment inconsistent withasFunction
#49518dart pub global run
#56044--enable-experiment=native-assets
)The text was updated successfully, but these errors were encountered: