Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only) #133631

flba-eb · 2024-11-29T14:05:23Z

Changes of this pull request:

Refactor code for qnx nto targets to share more code in file nto_qnx.rs
Add support for an additional network stack on nto qnx 7.1.

QNX 7.1 supports two network stacks:
1. io-pkt, which is default
2. io-sock, which is optional on 7.1 but default in QNX 8.0
As one can see in the io-sock migration notes, this changes the libc API in a way similar to e.g. linux-gnu vs. linux-musl.

This change adds a new target which has a different value for target_env, so that e.g. libc can distinguish between both APIs.
Add initial support for QNX 8.0, thanks to AkhilTThomas. As it turned out, the problem with forking many processes still exists in QNX 8.0. Because if this, we are now using it for any QNX version (i.e. not check for target_env anymore).

@rustbot label +O-neutrino
CC: @jonathanpallant @japaric @gh-tr @AkhilTThomas

rustbot · 2024-11-29T14:05:31Z

rustbot has assigned @pnkfelix.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

jonathanpallant · 2024-11-29T14:20:41Z

So, first up, excellent! The AWS QNX Neutrino 7.1.0 AMIs appears to use iosock and so I have to keep copying over libsocket.so.3 to run Rust code there. With this change hopefully I won't.

However, I note that there are currently no five-part-triples in the target list, so this would be the first. Perhaps aarch64-unknown-nto-qnx710_iosock would be better.

Will there be a x86_64-unknown-nto-qnx710_iosock to match?

flba-eb · 2024-11-29T14:26:54Z

So, first up, excellent! The AWS QNX Neutrino 7.1.0 AMIs appears to use iosock and so I have to keep copying over libsocket.so.3 to run Rust code there. With this change hopefully I won't.

Beware that version 3 and 4 have different APIs -- they may seem to work, but there might be subtle issues up to undefined behavior.

However, I note that there are currently no five-part-triples in the target list, so this would be the first. Perhaps aarch64-unknown-nto-qnx710_iosock would be better.

Very good idea -- I found it confusing that target_env uses an underscore while the target name does not. I will rename it!

Will there be a x86_64-unknown-nto-qnx710_iosock to match?

Not planned so far, but should be quite easy to do. How big is the need for it?

jonathanpallant · 2024-11-29T14:29:28Z

How big is the need for it?

No idea. Who was asking for this target?

src/doc/rustc/src/platform-support/nto-qnx.md

jonathanpallant · 2024-11-29T15:16:25Z

I was thinking to use an obvious dummy value rather than panicking if the environment variable isn't set. But maybe the project people have a better idea.

Are there another other targets that need to read environment variables?

madsmtm · 2025-01-24T13:49:57Z

Lesson learned: Define new target first, with minimal (or no) other changes, and create a PR for it. All other changes like refactoring and documentation should came later and separately.

Well I think it needs to go into the platform docs, but you should mark it as having only libcore support (because libstd requires cc-rs and libc and a bunch of other updates that can't happen until they know about the target).

I've opened #135992 to fix that.

workingjubilee · 2025-01-24T20:27:07Z

My last material question was #133631 (comment) which was because I thought I might have had an idea for how we could do without the additional target definition. Alas, it is not so.

Anyway, these maintainers know what the policy is. The PR description does not need a bunch of additional noise.

@bors r+ rollup

bors · 2025-01-24T20:27:10Z

📌 Commit 0462826 has been approved by workingjubilee

It is now in the queue for this repository.

…rt, r=workingjubilee Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only) Changes of this pull request: 1. Refactor code for qnx nto targets to share more code in file `nto_qnx.rs` 1. Add support for an additional network stack on nto qnx 7.1. QNX 7.1 supports two network stacks: 1. `io-pkt`, which is default 2. `io-sock`, which is optional on 7.1 but default in QNX 8.0 As one can see in the [io-sock migration notes](https://www.qnx.com/developers/docs/7.1/index.html#com.qnx.doc.neutrino.io_sock/topic/migrate_app.html), this changes the libc API in a way similar to e.g. linux-gnu vs. linux-musl. This change adds a new target which has a different value for `target_env`, so that e.g. libc can distinguish between both APIs. 2. Add initial support for QNX 8.0, thanks to AkhilTThomas. As it turned out, the problem with forking many processes still exists in QNX 8.0. Because if this, we are now using it for any QNX version (i.e. not check for `target_env` anymore).

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#133631 (Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only)) - rust-lang#133951 (Make the wasm_c_abi future compat warning a hard error) - rust-lang#134283 (fix(libtest): Deprecate '--logfile') - rust-lang#134679 (Windows: remove readonly files) - rust-lang#135635 (Move `std::io::pipe` code into its own file) - rust-lang#135842 (TRPL: more backward-compatible Edition changes) - rust-lang#135953 (ci.py: check the return code in `run-local`) - rust-lang#136031 (Expand polonius MIR dump) r? `@ghost` `@rustbot` modify labels: rollup

…rt, r=workingjubilee Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only) Changes of this pull request: 1. Refactor code for qnx nto targets to share more code in file `nto_qnx.rs` 1. Add support for an additional network stack on nto qnx 7.1. QNX 7.1 supports two network stacks: 1. `io-pkt`, which is default 2. `io-sock`, which is optional on 7.1 but default in QNX 8.0 As one can see in the [io-sock migration notes](https://www.qnx.com/developers/docs/7.1/index.html#com.qnx.doc.neutrino.io_sock/topic/migrate_app.html), this changes the libc API in a way similar to e.g. linux-gnu vs. linux-musl. This change adds a new target which has a different value for `target_env`, so that e.g. libc can distinguish between both APIs. 2. Add initial support for QNX 8.0, thanks to AkhilTThomas. As it turned out, the problem with forking many processes still exists in QNX 8.0. Because if this, we are now using it for any QNX version (i.e. not check for `target_env` anymore).

Rollup of 8 pull requests Successful merges: - rust-lang#133631 (Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only)) - rust-lang#134358 (compiler: Set `target_abi = "ilp32e"` on all riscv32e targets) - rust-lang#135764 (Fix tests on LLVM 20) - rust-lang#135812 (Fix GDB `OsString` provider on Windows ) - rust-lang#135842 (TRPL: more backward-compatible Edition changes) - rust-lang#135946 (Remove extra whitespace from rustdoc breadcrumbs for copypasting) - rust-lang#135953 (ci.py: check the return code in `run-local`) - rust-lang#136019 (Add an `unchecked_div` alias to the `Div<NonZero<_>>` impls) Failed merges: - rust-lang#136037 (Mark all NuttX targets as tier 3 target and support the standard library) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 7 pull requests Successful merges: - rust-lang#133631 (Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only)) - rust-lang#134358 (compiler: Set `target_abi = "ilp32e"` on all riscv32e targets) - rust-lang#135812 (Fix GDB `OsString` provider on Windows ) - rust-lang#135842 (TRPL: more backward-compatible Edition changes) - rust-lang#135946 (Remove extra whitespace from rustdoc breadcrumbs for copypasting) - rust-lang#135953 (ci.py: check the return code in `run-local`) - rust-lang#136019 (Add an `unchecked_div` alias to the `Div<NonZero<_>>` impls) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#133631 - flba-eb:add_nto_qnx71_iosock_support, r=workingjubilee Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only) Changes of this pull request: 1. Refactor code for qnx nto targets to share more code in file `nto_qnx.rs` 1. Add support for an additional network stack on nto qnx 7.1. QNX 7.1 supports two network stacks: 1. `io-pkt`, which is default 2. `io-sock`, which is optional on 7.1 but default in QNX 8.0 As one can see in the [io-sock migration notes](https://www.qnx.com/developers/docs/7.1/index.html#com.qnx.doc.neutrino.io_sock/topic/migrate_app.html), this changes the libc API in a way similar to e.g. linux-gnu vs. linux-musl. This change adds a new target which has a different value for `target_env`, so that e.g. libc can distinguish between both APIs. 2. Add initial support for QNX 8.0, thanks to AkhilTThomas. As it turned out, the problem with forking many processes still exists in QNX 8.0. Because if this, we are now using it for any QNX version (i.e. not check for `target_env` anymore).

elahav · 2025-01-30T15:18:55Z

What is the problem with forking many processes? How many is "many"? QNX supports up to 4094 simultaneous processes (procnto takes 1, so that's 4093), each with up to 32K threads. Are you really hitting a limit?

flba-eb · 2025-01-30T15:42:57Z

@elahav : Are you referring to the changed comment in file library/std/src/sys/pal/unix/process/process_unix.rs? If you have read the comment and the linked page, what exactly is unclear, how can we improve it? We do not hit a limit.

elahav · 2025-01-30T15:50:20Z

I was just looking at the description of the MR, which says "As it turned out, the problem with forking many processes still exists in QNX 8.0". I went to the file you mentioned, and the comment is indeed clear. The code was changed in 8.0 to ignore EBADF on file duplication during fork(). While it is still a user error, we decided it's better to let the fork() succeed. Of course, you won't be able to predict whether the fd being changed from another thread is open in the child.

Did you run into the problem, or is it just based on the documentation? If so, it's possible the documentation is out of date.

flba-eb · 2025-01-30T16:04:26Z

The problem was reliably reproducible with 7.1 when running the test suite of Rusts stdlib (which forks a lot of processes with high frequency, one process per test case). I never tested that on 8.0 without our workaround. We changed the comment based on the documentation of 8.0. Do you think this should be fixed in 8.0 or could the behavior have change which would require a change in Rusts stdlib?

elahav · 2025-01-30T16:08:46Z

You can keep the loop if it makes it easier to have common code between 7.1 and 8.0. It should not be needed, but it doesn't do any harm.

Note that posix_spawn() can still return EBDAF for reasons mentioned by POSIX.

Improve documentation when adding a new target rust-lang#133631 (comment) shows that it can be a bit difficult process-wise to add a new target. I've added a bit of text to the docs, suggesting that users add the target defintion/spec first, and later work on `std` support. I also found that we have two places where we document how to add a new target. I've linked these for now, but they should probably be merged somehow in the future. `@rustbot` label A-docs r? compiler CC `@workingjubilee` who's worked a lot on target specs IIRC.

Rollup merge of rust-lang#135992 - madsmtm:new-target-docs, r=jieyouxu Improve documentation when adding a new target rust-lang#133631 (comment) shows that it can be a bit difficult process-wise to add a new target. I've added a bit of text to the docs, suggesting that users add the target defintion/spec first, and later work on `std` support. I also found that we have two places where we document how to add a new target. I've linked these for now, but they should probably be merged somehow in the future. `@rustbot` label A-docs r? compiler CC `@workingjubilee` who's worked a lot on target specs IIRC.

Improve documentation when adding a new target rust-lang/rust#133631 (comment) shows that it can be a bit difficult process-wise to add a new target. I've added a bit of text to the docs, suggesting that users add the target defintion/spec first, and later work on `std` support. I also found that we have two places where we document how to add a new target. I've linked these for now, but they should probably be merged somehow in the future. `@rustbot` label A-docs r? compiler CC `@workingjubilee` who's worked a lot on target specs IIRC.

rustbot assigned pnkfelix Nov 29, 2024