-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: fix the rust build #27
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Generated by Cargo | ||
# will have compiled files and executables | ||
debug/ | ||
target/ | ||
|
||
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries | ||
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html | ||
Cargo.lock | ||
|
||
# These are backup files generated by rustfmt | ||
**/*.rs.bk | ||
|
||
# MSVC Windows builds of rustc generate these, which store debugging information | ||
*.pdb | ||
|
||
# Cache directory from updater cli. | ||
updater_cache |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
[workspace] | ||
members = ["cli", "library"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# Updater library | ||
|
||
This is the C/Rust side of the Shorebird code push system. This is built | ||
in Rust with a C API for easy calling from other languages, most notably | ||
for linking into libflutter.so. | ||
|
||
# Parts | ||
* cli: Test the updater library via the Rust API (for development). | ||
* dart_cli: Test ffi wrapping of updater library. | ||
* library: The rust library that does the actual update work. | ||
|
||
# TODO: | ||
* Remove all non-MVP code. | ||
* Add an async API. | ||
* Add support for "channels" (e.g. beta, stable, etc). | ||
* Write tests for state management. | ||
* Make state management/filesystem management atomic (and tested). | ||
* Move updater values out of the params into post body? | ||
* Support hashing values and check them? | ||
* Add "validate" command to validate state. | ||
* Write a mode that runs the updater first and then launches whatever is downloaded? | ||
* Use cbindgen to generate the C api header file. | ||
https://github.com/eqrion/cbindgen/blob/master/docs.md | ||
|
||
|
||
# Rust | ||
We use normal rust idioms (e.g. Result) inside the library and then bridge those | ||
to C via an explicit stable C API (explicit enums, null pointers for optional | ||
arguments, etc). The reason for this is that it lets the Rust code feel natural | ||
and also gives us maximum flexibility in the future for exposing more in the C | ||
API without having to refactor the internals of the library. | ||
|
||
## Notes | ||
* https://github.com/RubberDuckEng/safe_wren has an example of building a rust library and exposing it with a C api. | ||
|
||
## Other update systems | ||
* https://theupdateframework.io/ | ||
* https://fuchsia.dev/fuchsia-src/concepts/packages/software_update_system |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
To replicate the updater demo, you'll need a copy of the Flutter Engine. | ||
|
||
These are _not_ how Shorebird will work, but this is what I hacked together | ||
for the demo video. Writing these down so others can replicate if desired. | ||
|
||
# Building the updater library for Android | ||
|
||
The best way I found was to install: | ||
https://github.com/bbqsrc/cargo-ndk | ||
|
||
``` | ||
rustup install beta | ||
cargo +beta install cargo-ndk | ||
rustup +beta target add \ | ||
aarch64-linux-android \ | ||
armv7-linux-androideabi \ | ||
x86_64-linux-android \ | ||
i686-linux-android | ||
cargo +beta ndk --target aarch64-linux-android build --release | ||
``` | ||
|
||
# Setting up to build the Flutter Engine: | ||
|
||
https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment | ||
https://github.com/flutter/flutter/wiki/Compiling-the-engine | ||
|
||
I would consider building a clean engine first and testing that you have | ||
that working before trying Shorebird's modified engine. | ||
|
||
The hacked up version of the engine used for my demo can be found here: | ||
https://github.com/shorebirdtech/engine/tree/codepush | ||
|
||
# Symlink in the Rust binaries | ||
|
||
I symlinked the results of the rust build into the engine/src directory: | ||
|
||
``` | ||
cd flutter | ||
mkdir updater | ||
cd updater | ||
ln -s $HOME/Documents/GitHub/shorebird_private/shorebird/updater/library/include/updater.h | ||
mkdir android_aarch64 | ||
cd android_aarch64 | ||
ln -s $HOME/Documents/GitHub/shorebird_private/shorebird/updater/target/aarch64-linux-android/release/libupdater.a | ||
``` | ||
|
||
# Building Flutter Engine | ||
|
||
``` | ||
./flutter/tools/gn --android --android-cpu arm64 --runtime-mode=release | ||
ninja -C out/android_release_arm64 | ||
``` | ||
|
||
The linking step for android_release_arm64 is _much_ longer than other platforms | ||
we may need to use unopt or debug builds for faster iteration. | ||
|
||
I also add `&& say "done"` to the end of the ninja command so I know when it's | ||
done (because it takes minutes). | ||
|
||
# Running the updater | ||
|
||
From updater_demo: | ||
|
||
``` | ||
flutter run --local-engine-src-path $HOME/Documents/GitHub/engine/src --local-engine=android_release_arm64 --release | ||
``` | ||
|
||
Only need to do that once, once it's installed on the phone then you don't | ||
need `flutter run` anymore. | ||
|
||
# Building the replacement libraries | ||
|
||
For the demo I used "android.a" and "android.b" which were just copies of | ||
libapp.so files which Flutter had built for me. | ||
|
||
Once you've built the Flutter app in the way you want it: | ||
|
||
``` | ||
cp build/app/intermediates/stripped_native_libs/release/out/lib/arm64-v8a/libapp.so android.a | ||
``` | ||
|
||
You could dig them out of the apk, but that intermediate directory should be | ||
the correct file and is much easier. | ||
|
||
`flutter build apk -t lib/main_b.dart` should build the app in the way I used | ||
in my demo (I built it with `flutter run` and modifying main.dart directly, but | ||
that command should work too). | ||
|
||
# shorebird command line | ||
|
||
I hadn't yet modified `shorebird` to include the `publisher` functionality, | ||
so I had this in my path: | ||
|
||
``` | ||
#!/bin/bash | ||
dart run $HOME/Documents/Github/shorebird_private/shorebird/updater/publisher/bin/publisher.dart publish $2 | ||
``` | ||
|
||
The right solution is to remove the old shorebird functionality and integrate publisher. | ||
|
||
# Ports | ||
|
||
Because updater_server as running locally I also had to forward ports from my | ||
host into the emulator: | ||
|
||
``` | ||
adb reverse tcp:8080 tcp:8080 | ||
``` | ||
|
||
# Running the updater_server | ||
|
||
In a separate terminal: | ||
|
||
``` | ||
cd shorebird/updater/updater_server | ||
dart run | ||
``` | ||
|
||
# The demo | ||
|
||
The demo was then just launching the app on the emulator (manually) | ||
and then using the `shorebird` command line to publish the new libraries | ||
to change what code the app ran: | ||
|
||
``` | ||
shorebird publish android.a | ||
shorebird publish android.b | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
[package] | ||
name = "cli" | ||
version = "0.1.0" | ||
edition = "2021" | ||
|
||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html | ||
|
||
[dependencies] | ||
clap = { version = "4.1.6", features = ["derive"] } | ||
updater = { path = "../library" } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
extern crate updater; | ||
|
||
use clap::{Parser, Subcommand}; | ||
|
||
#[derive(Parser)] | ||
#[command(author, version, about, long_about = None, arg_required_else_help=true)] | ||
struct Cli { | ||
#[command(subcommand)] | ||
command: Option<Commands>, | ||
} | ||
|
||
#[derive(Subcommand)] | ||
enum Commands { | ||
Check {}, | ||
Current {}, | ||
Update {}, | ||
} | ||
|
||
fn main() { | ||
let cli = Cli::parse(); | ||
|
||
let config = updater::AppConfig { | ||
client_id: "demo".to_string(), | ||
cache_dir: None, | ||
// base_url: "http://localhost:8080", | ||
// channel: "stable", | ||
}; | ||
|
||
// You can check for the existence of subcommands, and if found use their | ||
// matches just as you would the top level cmd | ||
match &cli.command { | ||
Some(Commands::Check {}) => { | ||
let needs_update = updater::check_for_update(&config); | ||
println!("Checking for update..."); | ||
if needs_update { | ||
println!("Update needed."); | ||
} else { | ||
println!("No update needed."); | ||
} | ||
} | ||
Some(Commands::Current {}) => { | ||
let version = updater::active_version(&config); | ||
println!("Current version info:"); | ||
match version { | ||
Some(v) => { | ||
println!("path: {:?}", v.path); | ||
println!("hash: {:?}", v.hash); | ||
println!("version: {:?}", v.version); | ||
} | ||
None => { | ||
println!("None"); | ||
} | ||
} | ||
} | ||
Some(Commands::Update {}) => { | ||
let status = updater::update(&config); | ||
println!("Update: {}", status); | ||
} | ||
None => {} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# https://dart.dev/guides/libraries/private-files | ||
# Created by `dart pub` | ||
.dart_tool/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Command line application to test ffi wrapping of updater library. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# This file configures the static analysis results for your project (errors, | ||
# warnings, and lints). | ||
# | ||
# This enables the 'recommended' set of lints from `package:lints`. | ||
# This set helps identify many issues that may lead to problems when running | ||
# or consuming Dart code, and enforces writing Dart using a single, idiomatic | ||
# style and format. | ||
# | ||
# If you want a smaller set of lints you can change this to specify | ||
# 'package:lints/core.yaml'. These are just the most critical lints | ||
# (the recommended set includes the core lints). | ||
# The core lints are also what is used by pub.dev for scoring packages. | ||
|
||
include: package:lints/recommended.yaml | ||
|
||
# Uncomment the following section to specify additional rules. | ||
|
||
# linter: | ||
# rules: | ||
# - camel_case_types | ||
|
||
# analyzer: | ||
# exclude: | ||
# - path/to/excluded/files/** | ||
|
||
# For more information about the core and recommended set of lints, see | ||
# https://dart.dev/go/core-lints | ||
|
||
# For additional information about configuring this file, see | ||
# https://dart.dev/guides/language/analysis-options |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of this should hopefully be able to go away very soon :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct.