Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix reading slightly incorrect Zip files #68106

Merged
merged 7 commits into from
Apr 20, 2022
Merged

Fix reading slightly incorrect Zip files #68106

merged 7 commits into from
Apr 20, 2022

Conversation

danmoseley
Copy link
Member

Fix #49580

@ghost ghost assigned danmoseley Apr 16, 2022
@ghost
Copy link

ghost commented Apr 16, 2022

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

Issue Details

Fix #49580

Author: danmoseley
Assignees: danmoseley
Labels:

area-System.IO.Compression
Milestone: -

@danmoseley
Copy link
Member Author

Also fix new SC test that will only pass elevated. cc @deeprobin -- our PR validation runs elevated, I believe.

@danmoseley
Copy link
Member Author

Updated with full annotations added for the test zip, making clear what's going on.

@danmoseley
Copy link
Member Author

failures are #67349

@danmoseley
Copy link
Member Author

BTW @dotnet/area-system-io-compression are we aware of any public corpus of test zip files? Clearly, there is some variation in spec interpretations. If there was such a corpus, I could imagine we write a script to unzip them all with "main" and also with 6.0 and verify the same outputs/errors. That would reduce the risk of the #1094 experience we had.

deeprobin
deeprobin approved these changes Apr 19, 2022
View reviewed changes
Copy link
Contributor

@deeprobin deeprobin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits.

Looks overall good to me

src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated Show resolved Hide resolved
adamsitnik
adamsitnik approved these changes Apr 20, 2022
View reviewed changes
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, I've left some comments, but all of them are just nits.

@danmoseley thanks a lot!

src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Outdated
Assert.Equal(6, entry.CompressedLength); // it should have used 32-bit size
}

private static readonly byte[] s_tinyZip64 =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: based on the description provided for the tests above, I think that "slightlyIncorrect" would be a better name than "tiny" here

src/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Show resolved Hide resolved
src/libraries/System.IO.Compression/src/System/IO/Compression/ZipBlocks.cs Outdated Show resolved Hide resolved
src/libraries/System.IO.Compression/tests/ZipArchive/zip_InvalidParametersAndStrangeFiles.cs Show resolved Hide resolved
@danmoseley
Copy link
Member Author

@dotnet/dnceng seems an issue in dotnet-install.sh?

2022-04-20T16:20:22.1681480Z ##[section]Starting: Restore and Build Product
2022-04-20T16:20:22.1693590Z ==============================================================================
2022-04-20T16:20:22.1694320Z Task         : Command line
2022-04-20T16:20:22.1694920Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-20T16:20:22.1695490Z Version      : 2.201.1
2022-04-20T16:20:22.1695890Z Author       : Microsoft Corporation
2022-04-20T16:20:22.1696510Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-20T16:20:22.1697200Z ==============================================================================
2022-04-20T16:20:22.5454730Z Generating script.
2022-04-20T16:20:22.5776320Z Script contents:
2022-04-20T16:20:22.5837500Z ./build.sh -subset libs+libs.tests   -configuration Debug -ci -arch x64 -framework net7.0  -testscope innerloop      /p:ArchiveTests=true
2022-04-20T16:20:22.5927300Z ========================== Starting Command Output ===========================
2022-04-20T16:20:22.6362330Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/7198096b-97d2-4e9e-b9f3-8d34e7d82013.sh
2022-04-20T16:20:24.1231070Z __DistroRid: osx-x64
2022-04-20T16:20:24.5786000Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-04-20T16:21:23.4178630Z Attempting to install dotnet from public_location.
2022-04-20T16:21:23.4381070Z /Users/runner/work/1/s//.dotnet/dotnet-install.sh: line 2271: syntax error: unexpected end of file

@ChadNedzlek
Copy link
Member

@danmoseley. It does look like there is a bug with it, but that tool isn't written/maintained by dnceng. If you click the "report infrastructure issue" we could try and get it assigned to the right people.

@danmoseley danmoseley merged commit 9a4cdce into dotnet:main Apr 20, 2022
@danmoseley danmoseley deleted the zip64 branch April 20, 2022 19:07
@danmoseley
Copy link
Member Author

My bad, thanks @ChadNedzlek and @missymessa

@missymessa
Copy link
Member

My bad, thanks @ChadNedzlek and @missymessa

No worries, thanks for using the new features :)

directhex pushed a commit to directhex/runtime that referenced this pull request Apr 21, 2022
@danmoseley @deeprobin
@adamsitnik @directhex
Fix reading slightly incorrect Zip files (dotnet#68106)
55d0cb6 
* Add test that fails

* fix

* Fix test failing unelevated

* annotate

* overflow

* Apply suggestions from code review

Co-authored-by: Robin Lindner <[email protected]>

* Apply suggestions from code review

Co-authored-by: Adam Sitnik <[email protected]>

Co-authored-by: Robin Lindner <[email protected]>
Co-authored-by: Adam Sitnik <[email protected]>
@ghost ghost locked as resolved and limited conversation to collaborators May 21, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@gfoidl gfoidl gfoidl left review comments

@adamsitnik adamsitnik adamsitnik approved these changes

@deeprobin deeprobin deeprobin approved these changes

@carlossanlop carlossanlop Awaiting requested review from carlossanlop

Assignees

@danmoseley danmoseley

Labels
area-System.IO.Compression
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

"A local file header is corrupt" error occurs while unpacking the ZIP archive
6 participants
@danmoseley @ChadNedzlek @missymessa @gfoidl @adamsitnik @deeprobin