Allow creating a flashable zip file that would restore apps' data

[Atrate]

It'd be nice if AM had a feature that allowed it to create a large flashable zip with all user apps' data that could be flashed when changing ROMs to restore all apps' data quickly and painlessly.

Feature inspired by Migrate

[MuntashirAkon]

I have to admit, I've no idea how to flush user apps through recovery. Flashing system apps is not a problem but user apps, I've never done that before. If you've any resources on how to do that or any example flashable zip file, please share it here.

[Atrate]

Migrate-generated zips do the following, as far as I know:

• install Migrate as a (system, I think) app
• copy its contents with APK backups to a temporary directory in /data.

Then, after rebooting, the Migrate app can take over the restoration process.

I know it it somehow possible to install user apps through recovery mode, though, as some years back i used Titanium Backup to make such a zip, which restored apps from recovery alone.

[MuntashirAkon]

Migrate-generated zips do the following, as far as I know:

• install Migrate as a (system, I think) app
• copy its contents with APK backups to a temporary directory in /data.

Then, after rebooting, the Migrate app can take over the restoration process.

I'm not going to follow this scheme as it's not a convenient process. Besides, you can just install AM and restore all your (or selected) apps using batch operations.

I know it it somehow possible to install user apps through recovery mode, though, as some years back i used Titanium Backup to make such a zip, which restored apps from recovery alone.

If you can find such example, please share it. If it's possible, it will be both easy and convenient process.

[Atrate]

OAndBackupX also has such a feature in its' roadmap so we may try to see if their implementation makes sense for AM.

This is the updater-script used in the flashable zip by TB (for an example audio recorder app):

 ui_print("");
show_progress(2,0);
run_program("/sbin/busybox", "mount", "/data");
run_program("/sbin/busybox", "mount", "/system");
run_program("/sbin/busybox", "mkdir", "/data/data/.titanium_backup_data_restore");
set_perm(0, 0, 0555, "/data/data/.titanium_backup_data_restore");
set_progress(0);
ui_print("Extracting data: com.dimowner.audiorecorder");
package_extract_file("com.dimowner.audiorecorder-20200813-071213.properties", "/data/data/.titanium_backup_data_restore/com.dimowner.audiorecorder-20200813-071213.properties");
set_perm(0, 0, 0444, "/data/data/.titanium_backup_data_restore/com.dimowner.audiorecorder-20200813-071213.properties");
package_extract_file("com.dimowner.audiorecorder-20200813-071213.tar.gz", "/data/data/.titanium_backup_data_restore/com.dimowner.audiorecorder-20200813-071213.tar.gz");
set_perm(0, 0, 0444, "/data/data/.titanium_backup_data_restore/com.dimowner.audiorecorder-20200813-071213.tar.gz");
ui_print("Installing app: com.dimowner.audiorecorder");
delete("/data/app/com.dimowner.audiorecorder.apk");
delete("/data/app/com.dimowner.audiorecorder-2.apk");
package_extract_file("com.dimowner.audiorecorder-0dc8bd4abd4ee0381e42b357743a7fcf.apk", "/data/app/com.dimowner.audiorecorder-1.apk");
set_perm(0, 0, 0644, "/data/app/com.dimowner.audiorecorder-1.apk");
set_progress(1);
ui_print("Installing app: com.keramidas.TitaniumBackup");
delete("/data/app/com.keramidas.TitaniumBackup.apk");
delete("/data/app/com.keramidas.TitaniumBackup-2.apk");
package_extract_file("com.keramidas.TitaniumBackup.apk", "/data/app/com.keramidas.TitaniumBackup-1.apk");
set_perm(0, 0, 0644, "/data/app/com.keramidas.TitaniumBackup-1.apk");
set_progress(2);
unmount("/data");
unmount("/system");
ui_print("Done !");
ui_print("Will reboot in 5 seconds...");
run_program("/sbin/busybox", "sleep", "5");
run_program("/sbin/reboot");

It also, of course, carries with itself a large obfuscated update-binary file.

It seems as if PM automatically handled installing apps if they are put in /data/app/?

[MuntashirAkon]

It seems as if PM automatically handled installing apps if they are put in /data/app/?

That's interesting. I wonder if it's true.

It seems somebody wondered about it as well: https://forum.xda-developers.com/showthread.php?t=2017608

[MuntashirAkon]

This creates another problem: How do I restore patched data like the odex file in this method?

[MuntashirAkon]

Related code in Android frameworks: https://cs.android.com/android/platform/superproject/+/master:frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

[Atrate]

This creates another problem: How do I restore patched data like the odex file in this method?

One way would be to make it a two-stage process, wherein the system boots after 1 flash, reboots and reflashes automatically through TWRP, which I think is possible, but might be too unconvenient compared to simple batch restore

[MuntashirAkon]

I think if the system detects an app installed at /data/app/, it can detect app installed at /data/app/package.name-number_or_16_digit_based64_encoded_string/ as well. (Documentation suggests that number is used before Android 8 (Oreo). After that, 16 digit based64 encoded stuff is used. The code you've shared earlier suggests that TB is using the Oreo (and previous versions) style install method.) But I have to test this theory before I can proceed. If it works, though, the backup and restore process will be similar to #30 and I can simply add a utility function to create a flashable zip from the existing backup files.

One way would be to make it a two-stage process, wherein the system boots after 1 flash, reboots and reflashes automatically through TWRP, which I think is possible, but might be too unconvenient compared to simple batch restore

Another issue that I can think of is the platform dependency. If you patch an app's odex file in arm64, you cannot use the same odex file in x86_64 platform (the oat/arm64/ folder inside the app's code path will remain as is but the system will not use the odex file since it will be looking for odex file at oat/x86_64/). Same is true for library paths (lib and lib-32) as well. This is an issue with #30 as well.

One possible solution this problem is to detect the current platform and copy only the apk files (or skip restoring code path in #30) if it's different.

[Atrate]

One possible solution this problem is to detect the current platform and copy only the apk files (or skip restoring code path in #30 ) if it's different.

Seems like the only sensible idea, possibly also show the user a warning at that.

[MuntashirAkon]

API 30 made things even more complicated and I don't think it would be appropriate to install (user) packages directly from recovery. The following workflow should work but it might need another reboot after the flash:

During flash

• Restore data
• Store source files and misc.am.tsv to some place else
• Install AM as a system app
• Leave some indications which will inform AM that the restore process is still on-going

After reboot

• Launch AM on boot
• Install apps, restore sources
• Restore contents of misc.am.tsv
• Reboot again

Complications

• Multiple users not supported
• Zip file size cannot be greater than 4GB
• Large APK size could be problematic due to lack of internal storage

There might be other complications which I don't know yet due to lack of experience.