Skip to content

Commit

Permalink
chore: Fix the rust build (#27)
Browse files Browse the repository at this point in the history
  • Loading branch information
eseidel authored Mar 6, 2023
1 parent debc601 commit c13091b
Show file tree
Hide file tree
Showing 21 changed files with 967 additions and 3 deletions.
17 changes: 17 additions & 0 deletions updater/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Generated by Cargo
# will have compiled files and executables
debug/
target/

# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk

# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb

# Cache directory from updater cli.
updater_cache
2 changes: 2 additions & 0 deletions updater/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[workspace]
members = ["cli", "library"]
38 changes: 38 additions & 0 deletions updater/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Updater library

This is the C/Rust side of the Shorebird code push system. This is built
in Rust with a C API for easy calling from other languages, most notably
for linking into libflutter.so.

# Parts
* cli: Test the updater library via the Rust API (for development).
* dart_cli: Test ffi wrapping of updater library.
* library: The rust library that does the actual update work.

# TODO:
* Remove all non-MVP code.
* Add an async API.
* Add support for "channels" (e.g. beta, stable, etc).
* Write tests for state management.
* Make state management/filesystem management atomic (and tested).
* Move updater values out of the params into post body?
* Support hashing values and check them?
* Add "validate" command to validate state.
* Write a mode that runs the updater first and then launches whatever is downloaded?
* Use cbindgen to generate the C api header file.
https://github.com/eqrion/cbindgen/blob/master/docs.md


# Rust
We use normal rust idioms (e.g. Result) inside the library and then bridge those
to C via an explicit stable C API (explicit enums, null pointers for optional
arguments, etc). The reason for this is that it lets the Rust code feel natural
and also gives us maximum flexibility in the future for exposing more in the C
API without having to refactor the internals of the library.

## Notes
* https://github.com/RubberDuckEng/safe_wren has an example of building a rust library and exposing it with a C api.

## Other update systems
* https://theupdateframework.io/
* https://fuchsia.dev/fuchsia-src/concepts/packages/software_update_system
128 changes: 128 additions & 0 deletions updater/UPDATER_DEMO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
To replicate the updater demo, you'll need a copy of the Flutter Engine.

These are _not_ how Shorebird will work, but this is what I hacked together
for the demo video. Writing these down so others can replicate if desired.

# Building the updater library for Android

The best way I found was to install:
https://github.com/bbqsrc/cargo-ndk

```
rustup install beta
cargo +beta install cargo-ndk
rustup +beta target add \
aarch64-linux-android \
armv7-linux-androideabi \
x86_64-linux-android \
i686-linux-android
cargo +beta ndk --target aarch64-linux-android build --release
```

# Setting up to build the Flutter Engine:

https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment
https://github.com/flutter/flutter/wiki/Compiling-the-engine

I would consider building a clean engine first and testing that you have
that working before trying Shorebird's modified engine.

The hacked up version of the engine used for my demo can be found here:
https://github.com/shorebirdtech/engine/tree/codepush

# Symlink in the Rust binaries

I symlinked the results of the rust build into the engine/src directory:

```
cd flutter
mkdir updater
cd updater
ln -s $HOME/Documents/GitHub/shorebird_private/shorebird/updater/library/include/updater.h
mkdir android_aarch64
cd android_aarch64
ln -s $HOME/Documents/GitHub/shorebird_private/shorebird/updater/target/aarch64-linux-android/release/libupdater.a
```

# Building Flutter Engine

```
./flutter/tools/gn --android --android-cpu arm64 --runtime-mode=release
ninja -C out/android_release_arm64
```

The linking step for android_release_arm64 is _much_ longer than other platforms
we may need to use unopt or debug builds for faster iteration.

I also add `&& say "done"` to the end of the ninja command so I know when it's
done (because it takes minutes).

# Running the updater

From updater_demo:

```
flutter run --local-engine-src-path $HOME/Documents/GitHub/engine/src --local-engine=android_release_arm64 --release
```

Only need to do that once, once it's installed on the phone then you don't
need `flutter run` anymore.

# Building the replacement libraries

For the demo I used "android.a" and "android.b" which were just copies of
libapp.so files which Flutter had built for me.

Once you've built the Flutter app in the way you want it:

```
cp build/app/intermediates/stripped_native_libs/release/out/lib/arm64-v8a/libapp.so android.a
```

You could dig them out of the apk, but that intermediate directory should be
the correct file and is much easier.

`flutter build apk -t lib/main_b.dart` should build the app in the way I used
in my demo (I built it with `flutter run` and modifying main.dart directly, but
that command should work too).

# shorebird command line

I hadn't yet modified `shorebird` to include the `publisher` functionality,
so I had this in my path:

```
#!/bin/bash
dart run $HOME/Documents/Github/shorebird_private/shorebird/updater/publisher/bin/publisher.dart publish $2
```

The right solution is to remove the old shorebird functionality and integrate publisher.

# Ports

Because updater_server as running locally I also had to forward ports from my
host into the emulator:

```
adb reverse tcp:8080 tcp:8080
```

# Running the updater_server

In a separate terminal:

```
cd shorebird/updater/updater_server
dart run
```

# The demo

The demo was then just launching the app on the emulator (manually)
and then using the `shorebird` command line to publish the new libraries
to change what code the app ran:

```
shorebird publish android.a
shorebird publish android.b
```
10 changes: 10 additions & 0 deletions updater/cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[package]
name = "cli"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
clap = { version = "4.1.6", features = ["derive"] }
updater = { path = "../library" }
61 changes: 61 additions & 0 deletions updater/cli/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
extern crate updater;

use clap::{Parser, Subcommand};

#[derive(Parser)]
#[command(author, version, about, long_about = None, arg_required_else_help=true)]
struct Cli {
#[command(subcommand)]
command: Option<Commands>,
}

#[derive(Subcommand)]
enum Commands {
Check {},
Current {},
Update {},
}

fn main() {
let cli = Cli::parse();

let config = updater::AppConfig {
client_id: "demo".to_string(),
cache_dir: None,
// base_url: "http://localhost:8080",
// channel: "stable",
};

// You can check for the existence of subcommands, and if found use their
// matches just as you would the top level cmd
match &cli.command {
Some(Commands::Check {}) => {
let needs_update = updater::check_for_update(&config);
println!("Checking for update...");
if needs_update {
println!("Update needed.");
} else {
println!("No update needed.");
}
}
Some(Commands::Current {}) => {
let version = updater::active_version(&config);
println!("Current version info:");
match version {
Some(v) => {
println!("path: {:?}", v.path);
println!("hash: {:?}", v.hash);
println!("version: {:?}", v.version);
}
None => {
println!("None");
}
}
}
Some(Commands::Update {}) => {
let status = updater::update(&config);
println!("Update: {}", status);
}
None => {}
}
}
3 changes: 3 additions & 0 deletions updater/dart_cli/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# https://dart.dev/guides/libraries/private-files
# Created by `dart pub`
.dart_tool/
1 change: 1 addition & 0 deletions updater/dart_cli/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Command line application to test ffi wrapping of updater library.
30 changes: 30 additions & 0 deletions updater/dart_cli/analysis_options.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# This file configures the static analysis results for your project (errors,
# warnings, and lints).
#
# This enables the 'recommended' set of lints from `package:lints`.
# This set helps identify many issues that may lead to problems when running
# or consuming Dart code, and enforces writing Dart using a single, idiomatic
# style and format.
#
# If you want a smaller set of lints you can change this to specify
# 'package:lints/core.yaml'. These are just the most critical lints
# (the recommended set includes the core lints).
# The core lints are also what is used by pub.dev for scoring packages.

include: package:lints/recommended.yaml

# Uncomment the following section to specify additional rules.

# linter:
# rules:
# - camel_case_types

# analyzer:
# exclude:
# - path/to/excluded/files/**

# For more information about the core and recommended set of lints, see
# https://dart.dev/go/core-lints

# For additional information about configuring this file, see
# https://dart.dev/guides/language/analysis-options
Loading

0 comments on commit c13091b

Please sign in to comment.