Emulate non-blocking STDIN console on Windows#14947
Merged
straight-shoota merged 3 commits intocrystal-lang:masterfrom Sep 5, 2024
Merged
Emulate non-blocking STDIN console on Windows#14947straight-shoota merged 3 commits intocrystal-lang:masterfrom
STDIN console on Windows#14947straight-shoota merged 3 commits intocrystal-lang:masterfrom
Conversation
HertzDevil
added
kind:feature
topic:stdlib:concurrency
platform:windows
…tdin-non-blocking
straight-shoota
approved these changes
Sep 4, 2024
straight-shoota
added a commit
that referenced
this pull request
Sep 16, 2024
The following should print the compiler help message to the file `foo.txt`:
```crystal
File.open("foo.txt", "w") do |f|
Process.exec("crystal", output: f)
end
```
It used to work on Windows in Crystal 1.12, but is now broken since 1.13. This is because `LibC._wexecvp` only inherits file descriptors in the C runtime, not arbitrary Win32 file handles; since we stopped calling `LibC._open_osfhandle`, the C runtime knows nothing about any reopened standard streams in Win32. Thus the above merely prints the help message to the standard output.
This PR creates the missing C file descriptors right before `LibC._wexecvp`. It also fixes a different regression of #14947 where reconfiguring `STDIN.blocking` always fails.
Co-authored-by: Johannes Müller <straightshoota@gmail.com>
straight-shoota
added a commit
that referenced
this pull request
Sep 17, 2024
The following should print the compiler help message to the file `foo.txt`:
```crystal
File.open("foo.txt", "w") do |f|
Process.exec("crystal", output: f)
end
```
It used to work on Windows in Crystal 1.12, but is now broken since 1.13. This is because `LibC._wexecvp` only inherits file descriptors in the C runtime, not arbitrary Win32 file handles; since we stopped calling `LibC._open_osfhandle`, the C runtime knows nothing about any reopened standard streams in Win32. Thus the above merely prints the help message to the standard output.
This PR creates the missing C file descriptors right before `LibC._wexecvp`. It also fixes a different regression of #14947 where reconfiguring `STDIN.blocking` always fails.
Co-authored-by: Johannes Müller <straightshoota@gmail.com>
1 task
straight-shoota
pushed a commit
that referenced
this pull request
Oct 31, 2024
`Crystal::System::FileDescriptor#@@reader_thread` is initialized before `Crystal::System::Fiber::RESERVED_STACK_SIZE` which creates a race condition. Regression from #14947
CTC97
pushed a commit
to CTC97/crystal
that referenced
this pull request
Nov 9, 2024
`Crystal::System::FileDescriptor#@@reader_thread` is initialized before `Crystal::System::Fiber::RESERVED_STACK_SIZE` which creates a race condition. Regression from crystal-lang#14947
straight-shoota
pushed a commit
that referenced
this pull request
Feb 17, 2026
…g syscall (#15871) Some syscalls can block the current thread in certain circumstances, for example: - `open(2)` when opening a FIFO, pipe or character device until another end is connected (from another thread or process); - `getaddrinfo(3)` until a DNS response (or error, or timeout) is received. This patch introduces a mechanism to declare the scheduler as "doing a syscall" which the monitor thread (SYSMON) can detect on its next iteration and will try to move the scheduler to another thread, so that only the fiber doing the syscall will be blocked, and the other fibers can be resumed. Usually, the syscall should terminate _before_ the monitor thread notices (for example opening a regular file), so the impact on performance is an atomic STORE + atomic CAS per syscall. At worst, a thread will be blocked for 10ms (SYSMON frequency). For example the updated opening FIFO file spec takes ~11ms to complete. It works for the MT execution contexts _and_ the ST context. It doesn't invalidate the ST guarantee that fibers in the context will never run in parallel: the blocked fiber is blocked on a syscall and will be re-enqueued _immediately_ after the syscall has completed; also the syscalls don't invoke callbacks that would execute crystal code, so AFAICT fibers still won't run in parallel (please correct me if I'm wrong). ## NOTES The isolated context expects to block, so the `#syscall(&)` method is a no-op there. There are probably other blocking syscalls that we might want to consider. For example, reading from STDIN on Windows could be greatly simplified. Another example is `flock` that is currently retried every 100ms when it doesn't block the current thread. We might want to be able to actively detach a scheduler when calling `#syscall(&)`, so we could try once (non-blocking) then on failure detach the scheduler and try again (blocking) without waiting for SYSMON to notice. ## EXAMPLE The following example blocks the current thread, yet the spawned fiber keeps ticking every second. Remove the `Fiber.syscall` wrapper, and the fiber won't even start! ```crystal # bin/crystal foo.cr -Dpreview_mt -Dexecution_context spawn do loop do sleep 1.second puts "tick" end end Fiber.syscall do Thread.sleep(5.seconds) end ``` ## FOLLOW UP We plan to use this in the future to rework and simplify use cases in the stdlib. For example: - Polling event loops could support blocking file descriptors, so we could stop setting `O_NONBLOCK` on standard descriptors (shared), including pipes to spawned processes (#16353). - Same on Windows where console streams don't support OVERLAPPED (#14576, #14947).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves part of #14576.
Thread::ConditionVariableis now used again.There is no opt-out mechanism; while technically possible, there is probably no real benefit to it.