Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix reading from a large compressed file using external process #161

Merged
merged 3 commits into from
Jun 12, 2024

Conversation

marcelm
Copy link
Collaborator

@marcelm marcelm commented Jun 4, 2024

Previously, only the uncompressed data was "large", but issue #160 is only triggered if the compressed data is large (in this case, larger than 128 kB), which apparently exceeds some input buffer.

This only ensures the tests are failing, no fix yet.

This includes an attempt at a fix where the process is first terminated and then the reader thread is told to stop.

See #160

It doesn’t make sense to require a sleep() because real usage of the library
will not sleep either.
There was already a test for this, but only the uncompressed data was
"large". Issue #160 is only triggered if the *compressed* data is
large (in this case, larger than 128 kB), which apparently exceeds
some input buffer.

See #160
@marcelm marcelm changed the title Test reading from a larger compressed file Fix reading from a large compressed file using external process Jun 4, 2024
@marcelm
Copy link
Collaborator Author

marcelm commented Jun 4, 2024

I pushed another commit that appears to fix the problem: By terminating the process first (and the thread afterwards), the problem can be circumvented.

@rhpvorderman
Copy link
Collaborator

Looks good to me! I am sorry I did not catch this during development of the feature.

@gcflymoto
Copy link

Hey no worries! Thank you for the quick support and the excellent library!

@rhpvorderman rhpvorderman merged commit 538cc1b into main Jun 12, 2024
18 checks passed
@rhpvorderman rhpvorderman deleted the large-piped branch June 12, 2024 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants