Skip to content

Avoid eagerly reading input streams in -r#16857

Merged
charliermarsh merged 1 commit intomainfrom
charlie/empty
Nov 26, 2025
Merged

Avoid eagerly reading input streams in -r#16857
charliermarsh merged 1 commit intomainfrom
charlie/empty

Conversation

@charliermarsh
Copy link
Member

Summary

I think the comment should explain it.

Closes #16856.

@charliermarsh charliermarsh added the bug Something isn't working label Nov 26, 2025
@charliermarsh charliermarsh marked this pull request as ready for review November 26, 2025 03:30
Comment on lines +72 to +74
// `-r <( cat requirements.lock | grep -v nvidia | grep -v torch==)` or similar, in
// which case, reading the input would consume the stream, and only `requirements.txt`
// format is supported anyway.)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "only requirements.txt format is supported anyway"? It seems like you could do this with another file?

@zanieb
Copy link
Member

zanieb commented Nov 26, 2025

Not blocking the bug fix, but... why not just hold on to the contents in memory if we need to sniff it? It seems a little weird to constrain streamed contents to requirements.txt?

@zanieb
Copy link
Member

zanieb commented Nov 26, 2025

How does this relate to #16855 ?

@zanieb
Copy link
Member

zanieb commented Nov 26, 2025

I presume this regressed in #16805 / #16744 ?

@charliermarsh
Copy link
Member Author

Yes, it regressed there; #16855 fixed one manifestation of this issue, but the problem is more general; yes, we should be storing the contents in-memory but this brings us back into parity with the code prior to those changes and fixes the regression.

@charliermarsh charliermarsh merged commit 4d747f6 into main Nov 26, 2025
102 checks passed
@charliermarsh charliermarsh deleted the charlie/empty branch November 26, 2025 03:55
zanieb added a commit that referenced this pull request Nov 26, 2025
…data scripts" (#16861)

Reverts #16805 /
#16744

This also invalidates

- #16855
- #16857 

There's probably a way we can make this work, but detecting whether a
file is safe to read repeatedly is non-trivial, `is_file` returns `true`
for `/dev/stdin` on macOS so the approach from #16857 is not sufficient.
I spent a while trying to add `is_char_device` detection for macOS but
unfortunately that didn't work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0.9.12 breaks installs from requirements.lock

2 participants