Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rando ANDZA0000: Unable to open app.apk as a zip archive #6067

Closed
jonpryor opened this issue Jul 1, 2021 · 3 comments
Closed

Rando ANDZA0000: Unable to open app.apk as a zip archive #6067

jonpryor opened this issue Jul 1, 2021 · 3 comments
Assignees
Labels
Area: App+Library Build Issues when building Library projects or Application projects. bug Component does not function as intended.
Milestone

Comments

@jonpryor
Copy link
Member

jonpryor commented Jul 1, 2021

It's our favorite, hysenbugs!

There is no repro!

Context:

Sometimes, the DebuggerAttribute"Full", False, True) unit test (and others?) will fail:

error ANDZA0000: Unable to open '/Users/runner/work/1/s/bin/TestRelease/temp/DebuggerAttribute_Full_False_True/obj/Debug/android/bin/com.xamarin.debuggerattribute.apk' as zip archive

The cause of the error is that zipalign doesn't like the file:

$ "$HOME/android-toolchain/sdk/build-tools/28.0.3/zipalign" -p 4 bin/Debug/com.xamarin.debuggerattribute.apk app-Signed.apk
Unable to open 'bin/Debug/com.xamarin.debuggerattribute.apk' as zip archive

The file does exist; it is attached here as com.xamarin.debuggerattribute.zip.

Aside: zipalign source: https://android.googlesource.com/platform/build.git/+/refs/heads/master/tools/zipalign/

Lots of spitballing and investigation later, and we we see that zipdetails -vv doesn't like it:

% zipdetails -vv com.xamarin.debuggerattribute.zip # or .apk…
…
00C06E 000004 50 4B 03 04 LOCAL HEADER #8       04034B50
00C072 000001 00          Extract Zip Spec      00 '0.0'
00C073 000001 00          Extract OS            00 'MS-DOS'
00C074 000002 00 00       General Purpose Flag  0000
00C076 000002 00 00       Compression Method    0000 'Stored'
00C078 000004 00 00 21 00 Last Mod Time         00210000 'Tue Jan  1 00:00:00 1980'
00C07C 000004 02 19 FA 61 CRC                   61FA1902
00C080 000004 BC 06 00 00 Compressed Length     000006BC
00C084 000004 BC 06 00 00 Uncompressed Length   000006BC
00C088 000002 0E 00       Filename Length       000E
00C08A 000002 02 00       Extra Length          0002
00C08C 00000E 72 65 73 6F Filename              'resources.arsc'
              75 72 63 65
              73 2E 61 72
              73 63
00C09A 000002 00 00       Malformed Extra Data  00 00
00C09C 0006BC ...         PAYLOAD


Unexpecded END at offset 0000C758, value 00000000
Done

It appears that either the resources.arsc entry, or the following classes.dex entry, is somehow broken; 7zip reports:

% 7z t -bb3 com.xamarin.debuggerattribute.apk
…
ERROR: Headers Error : classes.dex
…
Sub items Errors: 1

Archives with Errors: 1

Sub items Errors: 1

Something isn't kosher here.

@grendello eventually concludes:

local header for classes.dex is corrupted
the CD entry for it is fine
that's why listing the contents works fine

it appears the entry is preceded (or the previous entry followed) by two additional bytes
which then escalates to 9 extra bytes before the subsequent entries, but it appears to badly affect just the first entry with the extra leading bytes

It looks like this is due to something we're doing, but we're not sure what, why, or how to fix it.

@jonpryor jonpryor added Area: App+Library Build Issues when building Library projects or Application projects. needs-triage Issues that need to be assigned. labels Jul 1, 2021
@grendello
Copy link
Contributor

Other ways to diagnose:

$ unzip -t com.xamarin.debuggerattribute.apk 
Archive:  com.xamarin.debuggerattribute.apk
    testing: AndroidManifest.xml      OK
    testing: res/drawable-mdpi-v4/icon.png   OK
    testing: res/drawable-hdpi-v4/icon.png   OK
    testing: res/drawable-xhdpi-v4/icon.png   OK
    testing: res/drawable-xxhdpi-v4/icon.png   OK
    testing: res/drawable-xxxhdpi-v4/icon.png   OK
    testing: res/layout/main.xml      OK
    testing: resources.arsc           OK
file #9:  bad zipfile offset (local header sig):  51032
    testing: lib/arm64-v8a/libxamarin-app.so   OK
    testing: lib/x86/libxamarin-app.so   OK
    testing: lib/arm64-v8a/libSystem.IO.Compression.Native.so   OK
    testing: lib/arm64-v8a/libSystem.Native.so   OK
    testing: lib/arm64-v8a/libSystem.Security.Cryptography.Native.Android.so   OK
    testing: lib/arm64-v8a/libmonosgen-2.0.so   OK
    testing: lib/arm64-v8a/libmonodroid.so   OK
    testing: lib/arm64-v8a/libxamarin-debug-app-helper.so   OK
    testing: lib/x86/libSystem.IO.Compression.Native.so   OK
    testing: lib/x86/libSystem.Native.so   OK
    testing: lib/x86/libSystem.Security.Cryptography.Native.Android.so   OK
    testing: lib/x86/libmonosgen-2.0.so   OK
    testing: lib/x86/libmonodroid.so   OK
    testing: lib/x86/libxamarin-debug-app-helper.so   OK
At least one error was detected in com.xamarin.debuggerattribute.apk.

unzip -t doesn't give name of the broken entry, alas.

zipinfo -vv shows us there are extra bytes preceding classes.dex (or following the previous entry, resources.arsc. @dellis1972 notes that the broken entry follows data we add to the apk by copying it from another zip archive creted by aapt2

Central directory entry #9:
---------------------------

  There are an extra 2 bytes preceding this file.

  classes.dex

  offset of local header from start of archive:   51032
                                                  (000000000000C758h) bytes
  file system or operating system of origin:      Unix
  version of encoding software:                   6.3
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT
  minimum software version required to extract:   2.0
  compression method:                             deflated
  compression sub-type (deflation):               maximum
  file security status:                           not encrypted
  extended local header:                          no
  file last modified on (DOS date/time):          2021 Jul 1 08:50:32
  32-bit CRC value (hex):                         ec5ca9fd
  compressed size:                                72278 bytes
  uncompressed size:                              317776 bytes
  length of filename:                             11 characters
  length of extra field:                          0 bytes
  length of file comment:                         0 characters
  disk number on which file begins:               disk 1
  apparent file type:                             binary
  Unix file attributes (100644 octal):            -rw-r--r--
  MS-DOS file attributes (00 hex):                none

  There is no file comment.

Central directory entry #10:
---------------------------

  There are an extra 9 bytes preceding this file.

  lib/arm64-v8a/libxamarin-app.so

@jonpryor jonpryor removed the needs-triage Issues that need to be assigned. label Jul 1, 2021
@jonpryor jonpryor added this to the .NET 7 milestone Jul 1, 2021
dellis1972 added a commit to dellis1972/xamarin-android that referenced this issue Jul 2, 2021
dellis1972 added a commit to dellis1972/xamarin-android that referenced this issue Jul 6, 2021
@jonpryor
Copy link
Member Author

We have another apparent instance of this bug: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4979360&view=ms.vss-test-web.build-test-results-tab&runId=23547392&resultId=100040&paneView=attachments

/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2320,2):
error ANDZA0000: Unable to open '…/CheckOldResourceDesignerWithWrongCasingIsRemoved(False)/obj/Debug/android/bin/com.xamarin.checkoldresourcedesignerwithwrongcasingisremoved.apk' as zip archive

dellis1972 added a commit to dellis1972/xamarin-android that referenced this issue Aug 5, 2021
@jpobst jpobst added the bug Component does not function as intended. label Sep 6, 2022
@jonpryor jonpryor modified the milestones: .NET 7, .NET 8 Sep 12, 2022
jonpryor pushed a commit that referenced this issue Feb 28, 2023
…7759)

Fixes? #6067

Context: https://gist.github.com/grendello/55b9a74c80c611dd48b726ee6f16d3c9
Context: dotnet/android-libzipsharp#125

Context: #7622

Changes: https://github.com/xamarin/monodroid/compare/50faac94c6a0c27864564829ac83f3988c82f8ef...602aca98245744882a129206b79b5a5e093dae44

  * xamarin/monodroid@602aca982: Bump androidtools for new LibZipSharp (xamarin/monodroid#1287)
  * xamarin/monodroid@1f52d5873: [tools/msbuild] Deploy *resources.dll from PackageReference (xamarin/monodroid#1276)

Changes: xamarin/androidtools@f11d163...c0bcb66

  * xamarin/androidtools@c0bcb66: Bump xamarin-android-tools for new LibZipSharp (xamarin/androidtools#377)

Changes: dotnet/android-tools@099fd95...dbe42bf

  * dotnet/android-tools@dbe42bf: Bump LibZipSharp version (dotnet/android-tools#204)

Sometimes in our tests we see an error from LibZipSharp which claims
a file cannot be opened as a valid ZIP archive:

	error ANDZA0000: Unable to open '…/example.apk' as zip archive

The reason for this is ZIP format corruption happening sometimes in
the `<BuildApk/>` task.

The way the process works is that we first run `aapt2` to produce
an APK archive named `packaged_resources`, which we then copy to the
destination APK (using standard `File.Copy()`) and open it to append
our content.  The first item we always append is the `classes.dex`
file which is also the **only** corrupted entry in the broken APK
files.

ZIP files contain two records describing each entry in the archive:

 1. in the Central Directory (located at the end of the file)

 2. local header in the ZIP data stream, offset of which is contained
    in the above Central Directory record.

After close examination, it appears that the broken APK files contain
(1) for `classes.dex` but not (2).  Instead of (2) we see a *copy*
of `packaged_resource` package's Central Directory record:

	% zipinfo -vv example.apk
	…
	Central directory entry #9:
	---------------------------

	  There are an extra 2 bytes preceding this file.

	  classes.dex

	  offset of local header from start of archive:   21680
	…

In particular, "There are an extra 2 bytes preceding this file" is a
sign that something "isn't right".  If we use `hexdump` to look at
the 128 bytes starting at offset 21680:

	% hexdump -C -s 21680 -n 128 example.apk 
	000054b0  50 4b 01 02 00 00 00 00  00 00 08 00 00 00 21 00  |PK............!.|
	000054c0  5e b2 00 5e cc 03 00 00  5c 0b 00 00 13 00 00 00  |^..^....\.......|
	000054d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 41 6e  |..............An|
	000054e0  64 72 6f 69 64 4d 61 6e  69 66 65 73 74 2e 78 6d  |droidManifest.xm|
	000054f0  6c 50 4b 01 02 00 00 00  00 00 00 00 00 00 00 21  |lPK............!|
	00005500  00 5c 71 d9 26 d2 05 00  00 d2 05 00 00 1d 00 00  |.\q.&...........|
	00005510  00 00 00 00 00 00 00 00  00 00 00 fd 03 00 00 72  |...............r|
	00005520  65 73 2f 64 72 61 77 61  62 6c 65 2d 6d 64 70 69  |es/drawable-mdpi|

we see that it starts with "PK", the common magic signature for zip
archives, and the following bytes { 0x1 0x2 } indicate that this is a
*central directory* entry, while it *should* be { 0x3, 0x4 } for a
*local header* entry.

The corruption may be caused by invalid stream position after we open
the `packaged_resources` copy and append `classes.dex` entry to it;
see dotnet/android-libzipsharp#125 for details.

We believe that the changes to `ZipArchive.stream_callback()` in
dotnet/android-libzipsharp#125 alongside additional calls to `Flush()` after
adding `classex.dex` **may** fix the problem.

If it doesn't, we should consider copying data from
`packaged_resources` to the final APK entry by entry by decompressing
them from the original archive and adding them to the destination one.
@jonpryor
Copy link
Member Author

We think (hope? pray?) that this was fixed by PR #7759.

@ghost ghost locked as resolved and limited conversation to collaborators Apr 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Area: App+Library Build Issues when building Library projects or Application projects. bug Component does not function as intended.
Projects
None yet
Development

No branches or pull requests

5 participants