Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to Solaris #1152

Closed
psumbera opened this issue Nov 15, 2019 · 35 comments · Fixed by #1808
Closed

Port to Solaris #1152

psumbera opened this issue Nov 15, 2019 · 35 comments · Fixed by #1808

Comments

@psumbera
Copy link
Contributor

Currently mio is using for Solaris epoll interface. But Solaris doesn't have this interface. Only Illumos has it.

There are two things to be done:

  • To distinguish between Solaris and Illumos.
  • Implement selector on Solaris (probably via even ports API)
@Thomasdezeeuw
Copy link
Collaborator

So you're saying that the Solaris port never worked?

@psumbera do you use Solaris and are you willing to contribute such a port? Otherwise we maybe should just drop support for Solaris if it never worked in the first place.

/cc @carllerche

@carllerche
Copy link
Member

Well, we need some way to test it in CI... none of the mio devs have a solaris setup.

@psumbera
Copy link
Contributor Author

Mio is used by Rust. And Solaris really needs Rust to build Firefox. From Rust version 1.38 something changed and it now fails to link on Solaris because of this issue (though I know Mio was used also with previous Rust versions without the issue).

I'm Solaris engineer so I have access to Solaris. Unfortunately I'm not very good with Rust. But anyway I will try to come with something or will seek for help.

@Thomasdezeeuw
Copy link
Collaborator

Can you try testing Mio (by running cargo test) and give us an output of the error, then we can try and fix it.

Also could you help with setting up the CI for Solaris? Going forward we would like to a bit stricter about what platforms we support and having a CI for it is one of them.

@psumbera
Copy link
Contributor Author

Can you try testing Mio (by running cargo test) and give us an output of the error, then we can try and fix it.

error: linking with `gcc` failed: exit code: 1
  |
  = note: "gcc" "-m64" "-L" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.11v52vx52e6g1ahq.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.13gmn7nbr8fz58ef.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.1a0zdiqerop42dtc.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.1qz1pata0l5g4pvq.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.1vxfmru39gxz4ar3.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.1xciivo4r5pr1dsq.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.24zcpn44b12eo1td.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.25r1lhkoly7a984g.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.2l3vzd2qkdu4ithc.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.2rlwld8q5u3rzw4m.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.2zklttukxouc1zzb.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.348stzlifohl7ig9.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.375xutsz6sjyrar4.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.39xu1wql7p9t2w6.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.4ap6wzhjmmr89gni.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.4dgjk7iptqfgxwiu.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.4kqcii0nyyrp2t8o.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.4txpvbe1msmro5ki.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.4zymg0dpk7z686tz.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.503g90dslmwg1yit.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.58y8hj1dfye6cfhp.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.f7q89bdlkfageun.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.jtqk5a7j6hdi865.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.no53nngwdccutsq.rcgu.o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.wwpf7khpch25zyy.rcgu.o" "-o" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc" "/builds/psumbera/mio-test/target/debug/examples/udp_server-2bc75a83567111bc.2c8h1obrgnxnb2gw.rcgu.o" "-Wl,-zignore" "-nodefaultlibs" "-L" "/builds/psumbera/mio-test/target/debug/deps" "-L" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib" "-Wl,-Bstatic" "/builds/psumbera/mio-test/target/debug/deps/libmio-13bc6fc3f9b5e16b.rlib" "/builds/psumbera/mio-test/target/debug/deps/liblibc-96233ac8f4948085.rlib" "/builds/psumbera/mio-test/target/debug/deps/liblog-56b3bee101ff990c.rlib" "/builds/psumbera/mio-test/target/debug/deps/libcfg_if-f7c1f14feae8d52e.rlib" "-Wl,--start-group" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libstd-8db4654fdc908441.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libpanic_unwind-3bfcf78ba2f0a654.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libbacktrace-7e6fbe207ebada1c.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libbacktrace_sys-37dec1614f68e8aa.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/librustc_demangle-7a6232bee123d2d4.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libhashbrown-4b510aad541d18fe.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/librustc_std_workspace_alloc-58508acde8e63d53.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libunwind-766e9e902a9094b7.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libcfg_if-2e9ddb3347410ede.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/liblibc-ca073c74dcf43c77.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/liballoc-92a9c38060cc44cf.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/librustc_std_workspace_core-953de5af53a4fd9f.rlib" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libcore-5fd40412d92f6541.rlib" "-Wl,--end-group" "/builds/psumbera/rustc-1.37.0/lib/rustlib/sparcv9-sun-solaris/lib/libcompiler_builtins-ba711d886e9bdfdf.rlib" "-Wl,-Bdynamic" "-lsocket" "-lposix4" "-lpthread" "-lresolv" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread"
  = note: Undefined            first referenced
           symbol                  in file
          epoll_wait                          /builds/psumbera/mio-test/target/debug/deps/libmio-13bc6fc3f9b5e16b.rlib(mio-13bc6fc3f9b5e16b.5d4rzgaw272n8cti.rcgu.o)
          epoll_ctl                           /builds/psumbera/mio-test/target/debug/deps/libmio-13bc6fc3f9b5e16b.rlib(mio-13bc6fc3f9b5e16b.5d4rzgaw272n8cti.rcgu.o)
          epoll_create1                       /builds/psumbera/mio-test/target/debug/deps/libmio-13bc6fc3f9b5e16b.rlib(mio-13bc6fc3f9b5e16b.5d4rzgaw272n8cti.rcgu.o)
          ld: fatal: symbol referencing errors
          collect2: error: ld returned 1 exit status


error: aborting due to previous error

I'm trying to add 'src/sys/unix/selector/evport.rs' which would add support via Solaris even ports API.

My question is also whether it wouldn't be enough to add support for other platforms via select()?

Also could you help with setting up the CI for Solaris? Going forward we would like to a bit stricter about what platforms we support and having a CI for it is one of them.

This is different problem we are aware of. As for Solaris right now there is for free available only old image of Solaris 11.4. So if you have virtual environment where to run it (e.g. VirtualBox) I can do some setup. But as you can see see I'm now locked with Rust 1.37 till we solve Mio problem...

@Thomasdezeeuw
Copy link
Collaborator

I think the problem is that rust's libc wrapper has the epoll function defined here: https://github.com/rust-lang/libc/blob/bf877ba7123d8d1f990b70a8dda306942c9a93b0/src/unix/solarish/mod.rs#L2171-L2196. So currently Mio builds against these functions, but as you said (and comment in the libc code) these aren't present.

If you remove target_os = solaris from the targets here:

#[cfg(any(target_os = "linux", target_os = "android", target_os = "solaris"))]
mod epoll;
#[cfg(any(target_os = "linux", target_os = "android", target_os = "solaris"))]
pub use self::epoll::{event, Event, Events, Selector};
. And create a new file in that directory for Solaris using select(), or whatever functions are appropriate, you can use it with the following target #[cfg(target_os = "solaris")]. Much like the mod and pub use statements in that file. Something like:

#[cfg(any(target_os = "linux", target_os = "android"))]
mod epoll;

#[cfg(any(target_os = "linux", target_os = "android"))]
pub use self::epoll::{event, Event, Events, Selector};

#[cfg(target_os = "solaris")]
mod evport;

#[cfg(target_os = "solaris")]
pub use self::evport::{event, Event, Events, Selector};

// kqueue...

In src/sys/unix/selector/evport.rs you would need to define the correct types and module as exported above.

Also could you update

mio/src/poll.rs

Line 168 in ac0515c

/// | Solaris | [epoll] |
.

@psumbera
Copy link
Contributor Author

I have started my work (see here):
psumbera@f6284af

But now I'm getting:

error[E0277]: `*mut std::ffi::c_void` cannot be sent between threads safely
  --> tests/poll.rs:16:5
   |
16 |     assert_send::<Events>();
   |     ^^^^^^^^^^^^^^^^^^^^^ `*mut std::ffi::c_void` cannot be sent between threads safely
   |
   = help: within `libc::unix::solarish::port_event`, the trait `std::marker::Send` is not implemented for `*mut std::ffi::c_void`
   = note: required because it appears within the type `libc::unix::solarish::port_event`
   = note: required because of the requirements on the impl of `std::marker::Send` for `std::ptr::Unique<libc::unix::solarish::port_event>`
   = note: required because it appears within the type `alloc::raw_vec::RawVec<libc::unix::solarish::port_event>`
   = note: required because it appears within the type `std::vec::Vec<libc::unix::solarish::port_event>`
   = note: required because it appears within the type `mio::event::events::Events`
note: required by `util::assert_send`
  --> tests/util/mod.rs:110:1
   |
110| pub fn assert_send<T: Send>() {}
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Any idea about this one? Thank you!

@Thomasdezeeuw
Copy link
Collaborator

I guess port_event contains a *mut c_void field? In rust raw pointers don't implement the Send trait, which is the trait that defines that a type may be send across threads. You'll need to define your own Event type that wraps the port_event type and implement Send and Sync on it:

pub struct Event(libc::port_event);

unsafe impl Send for Event {}
unsafe impl Sync for Event {}

Note however you must manually ensure that the libc::port_event may be send over thread bounds. If this is not allowed we'll need to discuss if we want to drop the Send guarantee from Event.

@psumbera
Copy link
Contributor Author

I guess port_event contains a *mut c_void field?

You are right:

    pub struct port_event {
        pub portev_events: ::c_int,
        pub portev_source: ::c_ushort,
        pub portev_pad: ::c_ushort,
        pub portev_object: ::uintptr_t,
        pub portev_user: *mut ::c_void,
    }

portev_user are user data. It doesn't have to be pointer (man page says: "user defined cookie"). It's set via port_associate(). I do set there token. Is that right?

I tried quick hack by modifying it to ::c_ulong in libc rust wrapper and now I get only following error:

error: `core::slice::<impl [T]>::len` is not yet stable as a const fn
  --> tests/unix_datagram.rs:21:26
   |
21 | const DATA1_LEN: usize = DATA1.len();
   |                          ^^^^^^^^^^^
   |
   = help: add `#![feature(const_slice_len)]` to the crate attributes to enable

Any clue?

In rust raw pointers don't implement the Send trait, which is the trait that defines that a type may be send across threads. You'll need to define your own Event type that wraps the port_event type and implement Send and Sync on it:

pub struct Event(libc::port_event);

unsafe impl Send for Event {}
unsafe impl Sync for Event {}

Sorry I'm no good enough with Rust yet. I tried:

--- a/src/sys/unix/selector/evport.rs
+++ b/src/sys/unix/selector/evport.rs
@@ -113,7 +113,11 @@ impl Drop for Selector {
     }
 }

-pub type Event = libc::port_event;
+pub struct Event(libc::port_event);
+
+unsafe impl Send for Event {}
+unsafe impl Sync for Event {}
+
 pub type Events = Vec<Event>;

 pub mod event {

And I get now:

error[E0308]: mismatched types
  --> src/sys/unix/selector/evport.rs:62:37
   |
62 |         syscall!(port_getn(self.ep, events.as_mut_ptr(), events.capacity() as u32, &mut nget as *mut u32, timeout))
   |                                     ^^^^^^^^^^^^^^^^^^^ expected struct `libc::unix::solarish::port_event`, found struct `sys::unix::selector::evport::Event`
   |
   = note: expected type `*mut libc::unix::solarish::port_event`
              found type `*mut sys::unix::selector::evport::Event`

error[E0609]: no field `portev_user` on type `&sys::unix::selector::evport::Event`
   --> src/sys/unix/selector/evport.rs:128:21
    |
128 |         Token(event.portev_user as usize)
    |                     ^^^^^^^^^^^

Can you please hint me little bit more?

Note however you must manually ensure that the libc::port_event may be send over thread bounds. If this is not allowed we'll need to discuss if we want to drop the Send guarantee from Event.

I hope there is no problem. Event port API is marked as thread safe.
Or am I missing something?

@Thomasdezeeuw
Copy link
Collaborator

portev_user are user data. It doesn't have to be pointer (man page says: "user defined cookie"). It's set via port_associate(). I do set there token. Is that right?

Yes then it means you can implement Send for it and it can be used in Token.

I tried quick hack by modifying it to ::c_ulong in libc rust wrapper and now I get only following error:

error: `core::slice::<impl [T]>::len` is not yet stable as a const fn
  --> tests/unix_datagram.rs:21:26
   |
21 | const DATA1_LEN: usize = DATA1.len();
   |                          ^^^^^^^^^^^
   |
   = help: add `#![feature(const_slice_len)]` to the crate attributes to enable

Any clue?

My guess it that you're using an older rust compiler version. Can you update to a newer version? Preferably 1.39. If not you can change those constants to literal values (that of the length of DATA1) for the time being.

error[E0308]: mismatched types
  --> src/sys/unix/selector/evport.rs:62:37
   |
62 |         syscall!(port_getn(self.ep, events.as_mut_ptr(), events.capacity() as u32, &mut nget as *mut u32, timeout))
   |                                     ^^^^^^^^^^^^^^^^^^^ expected struct `libc::unix::solarish::port_event`, found struct `sys::unix::selector::evport::Event`
   |
   = note: expected type `*mut libc::unix::solarish::port_event`
              found type `*mut sys::unix::selector::evport::Event`

error[E0609]: no field `portev_user` on type `&sys::unix::selector::evport::Event`
   --> src/sys/unix/selector/evport.rs:128:21
    |
128 |         Token(event.portev_user as usize)
    |                     ^^^^^^^^^^^

Can you please hint me little bit more?

The first problem is that now we're using different types. port_getn expects a pointer to an array of port_event structures, but we pass a pointer to the Event structs (which we defined). As these are different types Rust won't allow us to do that. I think the easier way forward is to remove the test in tests/poll.rs which caused the problem in the first place. We'll figure out later the best way to provide an Send implementation.

So revert the suggestion I made and go back to Event being a type alias for port_event. This should also resolve the second error.

Note however you must manually ensure that the libc::port_event may be send over thread bounds. If this is not allowed we'll need to discuss if we want to drop the Send guarantee from Event.

I hope there is no problem. Event port API is marked as thread safe.
Or am I missing something?

This is safe, see the first part of my comment

@psumbera
Copy link
Contributor Author

Ok. With my modification for Rust libc: port_event item portev_user *mut ::c_void to ::c_ulong

and

--- a/tests/unix_datagram.rs
+++ b/tests/unix_datagram.rs
@@ -18,8 +18,8 @@ use util::{

 const DATA1: &[u8] = b"Hello same host!";
 const DATA2: &[u8] = b"Why hello mio!";
-const DATA1_LEN: usize = DATA1.len();
-const DATA2_LEN: usize = DATA2.len();
+const DATA1_LEN: usize = 16;
+const DATA2_LEN: usize = 14;
 const DEFAULT_BUF_SIZE: usize = 64;
 const TEST_DIR: &str = "mio_unix_datagram_tests";
 const TOKEN_1: Token = Token(0);

I have following test failures:

     Running target/debug/deps/mio-f560c53dfa705f78

running 3 tests
test poll::as_raw_fd ... ok
test sys::unix::uds::tests::abstract_address ... ok
test sys::unix::uds::tests::pathname_address ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/close_on_drop-3d9cbb23646a15fe

running 1 test
test close_on_drop ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/event_source-2b115c80abbcee13

running 1 test
test assert_event_source_implemented_for ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/interests-5ec5456f296032de

running 3 tests
test bit_or ... ok
test is_tests ... ok
test fmt_debug ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target/debug/deps/poll-40df02e7a89b5311

running 9 tests
test is_send_and_sync ... ok
test poll_registration ... ok
test poll_erroneous_registration ... ok
test drop_cancels_interest_and_shuts_down ... ok
test registry_behind_arc ... ok
test zero_duration_polls_events ... ok
test poll_closes_fd ... ok
test add_then_drop ... FAILED
test run_once_with_nothing ... FAILED

failures:

---- add_then_drop stdout ----
thread 'add_then_drop' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 62, kind: Other, message: "Timer expired" }', src/libcore/result.rs:999:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---- run_once_with_nothing stdout ----
thread 'run_once_with_nothing' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 62, kind: Other, message: "Timer expired" }', src/libcore/result.rs:999:5


failures:
    add_then_drop
    run_once_with_nothing

test result: FAILED. 7 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

@Thomasdezeeuw
Copy link
Collaborator

If you add RUST_BACKTRACE=1 (or full) to the env vars it logs a backtrace of the error. To run a single test you can add the test name after the command, for example RUST_BACKTRACE=1 cargo test add_then_drop.

For the error itself, I don't know what the source could be. It seems specific to the Solaris port though, as I've never seen the error for Mio before.

Also if you can't find the source of the error it's fine to ignore the test for a while by adding the #[ignore] attribute, see:

mio/tests/tcp_stream.rs

Lines 358 to 360 in d64a303

#[test]
#[ignore = "This test is flaky, it doesn't always receive an event after shutting down the write side"]
fn shutdown_write() {
.

@psumbera
Copy link
Contributor Author

I resolved the issue with 'add_then_drop' like this:
psumbera@e67a3da

But now I'm puzzled with 'register_deregister' test. port_dissociate() returns ENOENT at the end. The issue seems to be that I need to re-associate fd every time I get event for it from port_getn(). Man page says:

       PORT_SOURCE_FD events represent a transition in the poll(2) status of a
       given file descriptor. Once an event is delivered, the file  descriptor
       is  no longer associated with the port. A file descriptor is associated
       (or re-associated) with a port using the port_associate(3C) function.

And I don't do it now. Isn't this needed for epoll too?

@Thomasdezeeuw
Copy link
Collaborator

On epoll the registration is kept alive (kernel side) even when an event is delivered. It seems that ports doesn't do that. So you would have to reregister after each event to simulate level triggers, or after an operation returned would block or didn't fill a buffer to simulate edge trigger (default in Mio). We currently face a similar problem on Windows.

For now; on each I/O operation, e.g. read or write, call reregister to ensure another event will come. This would simulate level triggers but that is fine for now. You might need to ignore ENOENT errors however since the object may be used without it ever being registered. That should fix the register_deregister test.

If it does not you can ignore the ENOENT error in deregister, since the object is no longer registered in any case.

@psumbera
Copy link
Contributor Author

For now; on each I/O operation, e.g. read or write, call reregister to ensure another event will come. This would simulate level triggers but that is fine for now.

How should I do it? Are there some mio read and write function I can modify?

@Thomasdezeeuw
Copy link
Collaborator

I'm afraid you'll need to manually change all methods of sys::unix::TcpStream, sys::unix::TcpListener, etc.

@psumbera
Copy link
Contributor Author

Ok.

There might be also possibility to use poll(7d):
https://docs.oracle.com/cd/E19253-01/816-5177/6mbbc4g9n/index.html

I will have look at it. Thank you!

@Thomasdezeeuw
Copy link
Collaborator

There might be also possibility to use poll(7d):
https://docs.oracle.com/cd/E19253-01/816-5177/6mbbc4g9n/index.html

That contains a note:

"The /dev/poll device, associated driver and corresponding manpages may be removed in a future Solaris release"

It seems to be deprecated and soon to be removed, so let's not depend on that.

@psumbera
Copy link
Contributor Author

"The /dev/poll device, associated driver and corresponding manpages may be removed in a future Solaris release"

It seems to be deprecated and soon to be removed, so let's not depend on that.

As a insider I don't think it will be removed anytime soon... In fact Illuomos epoll interface is using it:
https://git.wegmueller.it/Illumos/illumos-joyent/commit/a5eb7107f06a6e23e8e77e8d3a84c1ff90a73ac6

BTW. How can open and write into the file using libc functions in Rust?

@Thomasdezeeuw
Copy link
Collaborator

As a insider I don't think it will be removed anytime soon... In fact Illuomos epoll interface is using it:
https://git.wegmueller.it/Illumos/illumos-joyent/commit/a5eb7107f06a6e23e8e77e8d3a84c1ff90a73ac6

I see... then I think we could consider it.

BTW. How can open and write into the file using libc functions in Rust?

All libc functions are exported in the libc crate (docs: https://docs.rs/libc/0.2.65/libc). So open would be libc::open(..), read libc::read(..) etc. If you can however just use the std::fs::File type which is a safe wrapper around a file descriptor on Unix.

@psumbera
Copy link
Contributor Author

All libc functions are exported in the libc crate (docs: https://docs.rs/libc/0.2.65/libc). So open would be libc::open(..), read libc::read(..) etc.

Sorry, I don't seem to be able even open file...

error[E0308]: mismatched types
  --> src/sys/unix/selector/devpoll.rs:24:23
   |
24 |         syscall!(open("/dev/poll", O_RDWR)).map(|ep| Selector {
   |                       ^^^^^^^^^^^ expected i8, found str
   |
   = note: expected type `*const i8`
              found type `&'static str`

Also not sure where is open defined. Is it here?
https://github.com/rust-lang/libc/blob/77aa3e536811dcb3a379174f8ff41d49acb36f87/src/unix/mod.rs#L699
(seems to be for macos?)

@Thomasdezeeuw
Copy link
Collaborator

That is because in C a str is just a pointer to a slice of chars (i8 here). In rust a &str is a slice of valid UTF-8 characters, that is Rust's char (which is not the same as C's char).

Easiest way around it is using File.

use std::fs::File;

File::open("/dev/poll").map(|file| {
    let fd = file.into_raw_fd(); // Get the raw file descriptor.
})

@psumbera
Copy link
Contributor Author

I'm lost again. Can you please help me wit this?

   Compiling mio v0.7.0 (/builds/psumbera/rust-mio)
error[E0599]: no method named `as_ref` found for type `libc::unix::pollfd` in the current scope
  --> src/sys/unix/selector/devpoll.rs:79:14
   |
79 |             .as_ref()
   |              ^^^^^^

The relevant code is:

    pub fn register(&self, fd: RawFd, token: Token, interests: Interests) -> io::Result<()> {

        let buf = libc::pollfd {
               fd: fd,
               events: interests_to_evport(interests) as i16,
               revents: usize::from(token) as i16,
        };
        let buf = buf
            .as_ref()
            .map(|s| s as *const _)
            .unwrap_or(ptr::null_mut());

        syscall!(write(self.ep, buf, 8)).map(|_| ())
    }

@Thomasdezeeuw
Copy link
Collaborator

This works:

    pub fn register(&self, fd: RawFd, token: Token, interests: Interests) -> io::Result<()> {
        let buf = libc::pollfd {
               fd: fd,
               events: interests_to_evport(interests) as i16,
               revents: usize::from(token) as i16,
        };

        syscall!(write(self.ep, &buf, 8)).map(|_| ()) // Use &buf here.
    }

The original code mapped an Option<Duration> to raw pointer if Some or a null pointer if None, but this is not required for this code since buf is always something.

@psumbera
Copy link
Contributor Author

Thank you very much for helping me! Now I get this:

error[E0308]: mismatched types
  --> src/sys/unix/selector/devpoll.rs:79:33
   |
79 |         syscall!(write(self.ep, &buf, 8)).map(|_| ())
   |                                 ^^^^ expected enum `core::ffi::c_void`, found struct `libc::unix::pollfd`
   |
   = note: expected type `*const core::ffi::c_void`
              found type `&libc::unix::pollfd`

@Thomasdezeeuw
Copy link
Collaborator

Try &buf as *const _ or &buf as *const c_void (you might need to use c_void from libc/std lib), both these line will convert a pointer to to pollfd into a pointer to C's void.

@psumbera
Copy link
Contributor Author

Try &buf as *const _ or &buf as *const c_void (you might need to use c_void from libc/std lib), both these line will convert a pointer to to pollfd into a pointer to C's void.

Following did the trick:

syscall!(write(self.ep, &buf as *const _ as *const libc::c_void, 8)).map(|_| ())

But it seems that /dev/poll was wrong path. The main problem is missing place where to store token.

I tried to store lowest 3 bits of token into events field of struct pollfd (there are no more unused bits). With this it starts to work somehow...

So should I go back to Solaris ports? If yes, can you please show me some place where to do reregistration (e.g. in sys::unix::TcpStream)?

@Thomasdezeeuw
Copy link
Collaborator

But it seems that /dev/poll was wrong path. The main problem is missing place where to store token.

This might be because Rust str isn't null terminated, which is required by the open call.

I tried to store lowest 3 bits of token into events field of struct pollfd (there are no more unused bits). With this it starts to work somehow...

Doesn't it have any user defined fields? We need at least 64 bits (on 64 bits architectures).

So should I go back to Solaris ports? If yes, can you please show me some place where to do reregistration (e.g. in sys::unix::TcpStream)?

I don't know enough about Solaris to answer this. Maybe you can look at what libuv uses?

@psumbera
Copy link
Contributor Author

But it seems that /dev/poll was wrong path. The main problem is missing place where to store token.
This might be because Rust str isn't null terminated, which is required by the open call.

No. We don't understand each other. I have no problem with file opening. I can open it. I meant that Solaris /dev/poll (https://docs.oracle.com/cd/E19253-01/816-5177/6mbbc4g9n/index.html) doesn't allow to store any user defined value (token).

I tried to store lowest 3 bits of token into events field of struct pollfd (there are no more unused bits). With this it starts to work somehow...

Doesn't it have any user defined fields? We need at least 64 bits (on 64 bits architectures).

No it doesn't have it.

So should I go back to Solaris ports? If yes, can you please show me some place where to do reregistration (e.g. in sys::unix::TcpStream)?

I don't know enough about Solaris to answer this. Maybe you can look at what libuv uses?

libuv uses Solaris event ports:

https://github.com/libuv/libuv/blob/040543eebf4983b1459a1e0e0e26dae68b80cc28/src/unix/sunos.c#L179

And after each read they seem to call port_associate() again:

https://github.com/libuv/libuv/blob/040543eebf4983b1459a1e0e0e26dae68b80cc28/src/unix/sunos.c#L457

@Thomasdezeeuw
Copy link
Collaborator

If /dev/poll can't store any user defined data in the events than it can't be used for Mio. If Solaris event ports does support user defined data than it would be an option. I'm not much help on deciding what options are available as I don't use Solaris.

As for calling port_associate, the Windows implementation also re-registers on I/O operations, see for example TcpStream::read and the try_io! macro it uses.

@Thomasdezeeuw Thomasdezeeuw changed the title mio should provide selector implementation for Solaris via even ports API Port to Solaris Dec 6, 2019
@psumbera
Copy link
Contributor Author

Yes, my colleague proposed fix here: #1263

@Thomasdezeeuw
Copy link
Collaborator

Pr #1263 didn't go anywhere so I'm going to close this for. If people are interested in supporting Solaris (and maintaining it!) we can reopen this issue.

Thomasdezeeuw added a commit to Thomasdezeeuw/mio that referenced this issue Nov 7, 2021
We never really supported Solaris, we pretended it implemented epoll,
but it never did see tokio-rs#1152. As no
one ever committed to being a maintainer for the port I'm removing it
now with this commit.

Instead replace it with illumuos on the CI, which we do support (as it
supports epoll) and for which we do have maintainers.
Thomasdezeeuw added a commit to Thomasdezeeuw/mio that referenced this issue Nov 7, 2021
We never really supported Solaris, we pretended it implemented epoll,
but it never did see tokio-rs#1152. As no
one ever committed to being a maintainer for the port I'm removing it
now with this commit.

Instead replace it with illumuos on the CI, which we do support (as it
supports epoll) and for which we do have maintainers.
Thomasdezeeuw added a commit that referenced this issue Nov 7, 2021
We never really supported Solaris, we pretended it implemented epoll,
but it never did see #1152. As no
one ever committed to being a maintainer for the port I'm removing it
now with this commit.

Instead replace it with illumuos on the CI, which we do support (as it
supports epoll) and for which we do have maintainers.
@gco
Copy link

gco commented Dec 12, 2021

There really does need to be generic poll/select support as a fallback and that would handle most all fringe Unix platforms. It's also testable on the Tier 1 platforms. It's extremely frustrating that this library is causing so much grief on Solaris.

@Thomasdezeeuw
Copy link
Collaborator

There really does need to be generic poll/select support as a fallback and that would handle most all fringe Unix platforms.

Not really, we support a pretty broad range of Unix platforms already without a poll/select fallback. Furthermore the design of poll/select is very different from epoll/kqueue's design, with in turn decided Mio's design, so implementing a poll/select fallback in a safe way isn't as easy as you might think.

It's also testable on the Tier 1 platforms. It's extremely frustrating that this library is causing so much grief on Solaris.

Porting to a new platform requires more then just code, it also requires maintainers. In #1263 an attempt was made, but no-one stepped forward and to finish the port and commit to maintaining it. But anyone is welcome to finish the work!

bdbai added a commit to YtFlow/mio-noafd that referenced this issue Feb 9, 2022
commit 5234b5f
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 13 12:57:29 2021 +0100

    Release v0.8.0

commit 41a494b
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 13 12:59:43 2021 +0100

    Fix Clippy warning

commit a8c5756
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 12:45:33 2021 +0100

    Add changelog for v0.8

commit 7029a35
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 12:34:33 2021 +0100

    Add v0.7.14 change log

    From commit 064af84

commit dca2134
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 11:26:23 2021 +0100

    Fix feature flags for some tests files

    The test util module requires both the "os-poll" and "net" features.

commit b9f089b
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 11:19:09 2021 +0100

    Remove cfg attributes for Solaris

    We never really supported Solaris, we pretended it implemented epoll,
    but it never did see tokio-rs#1152. As no
    one ever committed to being a maintainer for the port I'm removing it
    now with this commit.

    Instead replace it with illumuos on the CI, which we do support (as it
    supports epoll) and for which we do have maintainers.

commit 7d86108
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 11:53:32 2021 +0100

    Add section about raw fd to portability guidelines

commit 3be5811
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Nov 7 12:09:53 2021 +0100

    Add note about short receive on datagram sockets

    Talking about the differences between OSs.

commit 3ca57f3
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 6 15:20:54 2021 +0100

    Document unconnected TcpStream returned by TcpStream::connect

commit 47cf59c
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 6 15:04:56 2021 +0100

    Deregister connection before dropping it in TCP example

commit 05009e4
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 6 14:53:45 2021 +0100

    Document that Mio report OOB data in Event::is_readable

    Reporting Out-of-band (OOB) as readable it could leave applications open
    to DoS attacks. However because Mio uses edge-triggers most applications
    won't actually be effected.

commit 44666e8
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 6 14:41:54 2021 +0100

    Fix match_like_matches_macro Clippy lint

    We've updated our MSVR since the comment above it.

commit f8695a7
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Nov 6 14:39:42 2021 +0100

    Update Rustc nightly version in CI

commit f4b9252
Author: Ben Noordhuis <[email protected]>
Date:   Sun Oct 10 16:45:25 2021 +0200

    Add sys::unix::SocketAddr::as_abstract_namespace()

    Fixes tokio-rs#1517.

commit 04e0ca4
Author: Thomas de Zeeuw <[email protected]>
Date:   Tue Sep 28 18:16:01 2021 +0200

    Update change log with v0.7.x releases

    Contains the work in the following commits:
     * v0.7.8 20b7298.
     * v0.7.9 07bc32f.
     * v0.7.10 b7006d7.
     * v0.7.11 772c692.
     * v0.7.12 7adfb75.
     * v0.7.13 75f41fb.

commit e55ec59
Author: Thomas de Zeeuw <[email protected]>
Date:   Thu Oct 7 20:21:22 2021 +0200

    Install nightly Rust on CI for install cargo-hack

commit 499004f
Author: Thomas de Zeeuw <[email protected]>
Date:   Thu Oct 7 20:14:36 2021 +0200

    Install Cargo-hack using nightly on CI

    Cargo-hack's (transient) dependency bitflags has updated its MSRV.

commit e9e91ff
Author: Thomas de Zeeuw <[email protected]>
Date:   Tue Sep 28 19:59:15 2021 +0200

    Fix Clippy warnings on Windows

    Seems this isn't check on the CI.

commit b48cce6
Author: Thomas de Zeeuw <[email protected]>
Date:   Tue Sep 28 19:51:19 2021 +0200

    Fix Clippy warnings

commit 37aec3e
Author: Thomas de Zeeuw <[email protected]>
Date:   Tue Sep 28 19:45:26 2021 +0200

    Fix dead_code warnings for Windows

commit 02e9be4
Author: Thomas de Zeeuw <[email protected]>
Date:   Tue Sep 28 19:32:35 2021 +0200

    Remove TcpSocket type

    The socket2 crate provide all the functionality and more. Furthermore
    supporting all socket options is beyond the scope of Mio.

    The easier migration is to the socket2 crate, using the Socket or
    SockRef types.

    The migration for Tokio is tracked in
    tokio-rs/tokio#4135.

commit d4ce420
Author: Rémi Lauzier <[email protected]>
Date:   Tue Jul 6 14:21:17 2021 -0400

    Update dev-dependencies

commit fbcc849
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Jul 3 12:44:57 2021 +0200

    Change port in connect_error

    Hopefully this port is actually not used.

    Also check Event::is_write_closed since we expect that to be true.

commit bfbcd9d
Author: Jake Shadle <[email protected]>
Date:   Fri Jul 2 15:17:17 2021 +0200

    Move wine from unsupported

commit 21ddf94
Author: Ivan Enderlin <[email protected]>
Date:   Tue Jun 22 22:36:09 2021 +0200

    chore: Make Clippy happy (bis).

commit 6d62f5d
Author: Ivan Enderlin <[email protected]>
Date:   Mon Jun 21 16:41:21 2021 +0200

    chore: Make Clippy happy.

commit 6eb1efa
Author: Ivan Enderlin <[email protected]>
Date:   Mon Jun 21 16:22:16 2021 +0200

    feat: Move `poll::selector` to `Registry::selector`.

commit 441367b
Author: Thomas de Zeeuw <[email protected]>
Date:   Sun Jun 13 00:33:17 2021 +0200

    Fix Selector::try_clone

    Calls fcntl F_DUPFD_CLOEXEC expects two arguments; the command
    (F_DUPFD_CLOEXEC) and an argument for the command. In this case an lower
    bound for the resulting file descriptor. Because we didn't provide a
    value it would take whatever value was left in the register from
    whatever code used it before the system call.

    This caused Waker::new to fail, see issue
    tokio-rs#1497.

commit cbcaedf
Author: Thomas de Zeeuw <[email protected]>
Date:   Sat Jun 12 20:39:12 2021 +0200

    Set FD_CLOEXEC flag on duplicated kqueue Poll

    Same as commit c52635c, but for kqueue.

commit c52635c
Author: Tim Zhang <[email protected]>
Date:   Tue May 25 11:40:54 2021 +0800

    Set the close-on-exec flag for the duplicate epoll_fd

    The close-on-exec flag (FD_CLOEXEC; see fcntl(2)) for the
    duplicate descriptor created by dup(2) is off.

    We can use fcntl + F_DUPFD_CLOEXEC to dup the epoll_fd to fix this
    issue.

    Fixes: tokio-rs/tokio#3809

    Signed-off-by: Tim Zhang <[email protected]>

commit 2246ffb
Author: Taiki Endo <[email protected]>
Date:   Sun May 23 16:06:15 2021 +0900

    Use ubuntu-18.04 instead of ubuntu-16.04

commit 0cfba5d
Author: cdcode <[email protected]>
Date:   Sun Jun 6 22:42:26 2021 +0100

    Small spelling correction in example

commit 22e8858
Author: Thomas de Zeeuw <[email protected]>
Date:   Thu May 13 17:09:57 2021 +0200

    Update outdated comment

commit 607a12f
Author: Thomas de Zeeuw <[email protected]>
Date:   Mon May 10 12:10:28 2021 +0200

    Replace x86_64-sun-solaris with x86_64-pc-solaris

    rust-lang/rust#82216 removed the
    x86_64-sun-solaris target from rustup, changing it to use
    x86_64-pc-solaris instead.

    Related issues:
     * rust-lang/rust#85098

commit 27a6a3c
Author: Thomas de Zeeuw <[email protected]>
Date:   Mon May 10 11:56:41 2021 +0200

    Avoid cast pointers to usize in windows::NamedPipe

    Changes the Inner::ptr_from_* methods to use ptr::wrapping_sub rather
    then casting to usize.

commit e316b21
Author: Thomas de Zeeuw <[email protected]>
Date:   Wed May 5 12:13:47 2021 +0200

    Replace offset constants with methods in Windows NamedPipe

commit 9e13732
Author: Thomas de Zeeuw <[email protected]>
Date:   Mon Apr 12 20:26:53 2021 +0200

    Reorder NamedPipe fields

    Moving the Overlapped fields to the start to make it easier to determine
    the offsets and hopefully incur less breakage once external fields
    change size.

    Note that the Overlapped fields internally uses miow::Overlapped, which
    in turn is a OVERLAPPED struct as found in the winapi crate and has a
    stable layout (as defined by the Windows API).

commit db0d74c
Author: Thomas de Zeeuw <[email protected]>
Date:   Mon Apr 12 20:03:24 2021 +0200

    Remove unsound offset_of macro

    And replace it with constants that define the offsets to the fields.
    It's not a pretty solution, but it's one without UB.

commit 1667a70
Author: Rob Ede <[email protected]>
Date:   Thu Apr 1 17:01:01 2021 +0100

    remove manual doc versioning
@Thomasdezeeuw
Copy link
Collaborator

For anyone still watching this issue. Solar is supported in Mio v1 using the poll(2) implementation (which is not the most efficient).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants