deps: update to uvwasi 0.0.5 #31432

cjihrig · 2020-01-20T23:32:47Z

This update improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber() (which was broken on FreeBSD). Missing uvwasi_fd_renumber() test coverage is also added via an freopen() test.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

bnoordhuis · 2020-01-21T09:08:42Z

deps/uvwasi/src/fd_table.c

+  uv_fs_req_cleanup(&req);
+  if (r != 0) {
+    uv_mutex_unlock(&dst_entry->mutex);
+    uv_mutex_unlock(&src_entry->mutex);


These unlocks and the ones at lines 383-384 look like a potential deadlock. You should normally release mutexes in reverse order of acquiring them.

Even when there's no deadlock it's probably still bad for performance because the acquirer can get rescheduled twice in case of contention.

bnoordhuis · 2020-01-21T09:12:38Z

deps/uvwasi/src/fd_table.c

+  /* Move the source entry to the destination slot in the table. */
+  table->fds[dst] = table->fds[src];
+  table->fds[dst]->id = dst;
+  uv_mutex_unlock(&table->fds[dst]->mutex);


I'm also not 100% sure it's safe for this mutex's critical section and &table->rwlock to overlap (but it might be - I just can't tell from the diff. :-))

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber().

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired.

Original commit message: prevent locking fd table while holding a mutex uvwasi_path_rename(), uvwasi_path_link(), uvwasi_path_open(), and uvwasi_fd_renumber() operate on multiple file descriptors. uvwasi_fd_renumber() has been updated prior to this commit, and is not relevant here. The other three functions would perform the following locking operations: - lock the file table - acquire a file descriptor mutex - unlock the file table - unlock the file table again - acquire another file descriptor mutex - unlock the file table - unlock the two mutexes Attempting to acquire the second mutex introduced the possibility of deadlock because another thread could attempt to acquire the first mutex while holding the file table lock. This commit ensures that multiple mutexes are either: - acquired in a single lock of the file table - or, only acquired after releasing previously held mutexes Fixes: nodejs/uvwasi#89

Original commit message: prevent race conditions with uvwasi_fd_close() uvwasi_fd_close() performed the following operations: - lock the file descriptor mutex - close the file - release the file descriptor mutex - call the file table's remove() function Once the fd's mutex is released, another thread could acquire it before the fd is removed from the file table. If this happens, remove() could destroy a held mutex. This commit updates uvwasi_fd_close() to perform the entire sequence while holding the file table's lock, preventing new acquisitions of the fd's mutex. Fixes: nodejs/uvwasi#88

This test provides missing coverage for __wasi_fd_renumber().

nodejs-github-bot · 2020-01-23T08:03:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/28588/ ✔️

Trott · 2020-01-23T13:23:12Z

Landed in c692568...96058f3

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber(). PR-URL: nodejs#31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired. PR-URL: nodejs#31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent locking fd table while holding a mutex uvwasi_path_rename(), uvwasi_path_link(), uvwasi_path_open(), and uvwasi_fd_renumber() operate on multiple file descriptors. uvwasi_fd_renumber() has been updated prior to this commit, and is not relevant here. The other three functions would perform the following locking operations: - lock the file table - acquire a file descriptor mutex - unlock the file table - unlock the file table again - acquire another file descriptor mutex - unlock the file table - unlock the two mutexes Attempting to acquire the second mutex introduced the possibility of deadlock because another thread could attempt to acquire the first mutex while holding the file table lock. This commit ensures that multiple mutexes are either: - acquired in a single lock of the file table - or, only acquired after releasing previously held mutexes Fixes: nodejs/uvwasi#89 PR-URL: nodejs#31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent race conditions with uvwasi_fd_close() uvwasi_fd_close() performed the following operations: - lock the file descriptor mutex - close the file - release the file descriptor mutex - call the file table's remove() function Once the fd's mutex is released, another thread could acquire it before the fd is removed from the file table. If this happens, remove() could destroy a held mutex. This commit updates uvwasi_fd_close() to perform the entire sequence while holding the file table's lock, preventing new acquisitions of the fd's mutex. Fixes: nodejs/uvwasi#88 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This test provides missing coverage for __wasi_fd_renumber(). PR-URL: nodejs#31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired. PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent locking fd table while holding a mutex uvwasi_path_rename(), uvwasi_path_link(), uvwasi_path_open(), and uvwasi_fd_renumber() operate on multiple file descriptors. uvwasi_fd_renumber() has been updated prior to this commit, and is not relevant here. The other three functions would perform the following locking operations: - lock the file table - acquire a file descriptor mutex - unlock the file table - unlock the file table again - acquire another file descriptor mutex - unlock the file table - unlock the two mutexes Attempting to acquire the second mutex introduced the possibility of deadlock because another thread could attempt to acquire the first mutex while holding the file table lock. This commit ensures that multiple mutexes are either: - acquired in a single lock of the file table - or, only acquired after releasing previously held mutexes Fixes: nodejs/uvwasi#89 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent race conditions with uvwasi_fd_close() uvwasi_fd_close() performed the following operations: - lock the file descriptor mutex - close the file - release the file descriptor mutex - call the file table's remove() function Once the fd's mutex is released, another thread could acquire it before the fd is removed from the file table. If this happens, remove() could destroy a held mutex. This commit updates uvwasi_fd_close() to perform the entire sequence while holding the file table's lock, preventing new acquisitions of the fd's mutex. Fixes: nodejs/uvwasi#88 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This test provides missing coverage for __wasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired. PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent locking fd table while holding a mutex uvwasi_path_rename(), uvwasi_path_link(), uvwasi_path_open(), and uvwasi_fd_renumber() operate on multiple file descriptors. uvwasi_fd_renumber() has been updated prior to this commit, and is not relevant here. The other three functions would perform the following locking operations: - lock the file table - acquire a file descriptor mutex - unlock the file table - unlock the file table again - acquire another file descriptor mutex - unlock the file table - unlock the two mutexes Attempting to acquire the second mutex introduced the possibility of deadlock because another thread could attempt to acquire the first mutex while holding the file table lock. This commit ensures that multiple mutexes are either: - acquired in a single lock of the file table - or, only acquired after releasing previously held mutexes Fixes: nodejs/uvwasi#89 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent race conditions with uvwasi_fd_close() uvwasi_fd_close() performed the following operations: - lock the file descriptor mutex - close the file - release the file descriptor mutex - call the file table's remove() function Once the fd's mutex is released, another thread could acquire it before the fd is removed from the file table. If this happens, remove() could destroy a held mutex. This commit updates uvwasi_fd_close() to perform the entire sequence while holding the file table's lock, preventing new acquisitions of the fd's mutex. Fixes: nodejs/uvwasi#88 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This test provides missing coverage for __wasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired. PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent locking fd table while holding a mutex uvwasi_path_rename(), uvwasi_path_link(), uvwasi_path_open(), and uvwasi_fd_renumber() operate on multiple file descriptors. uvwasi_fd_renumber() has been updated prior to this commit, and is not relevant here. The other three functions would perform the following locking operations: - lock the file table - acquire a file descriptor mutex - unlock the file table - unlock the file table again - acquire another file descriptor mutex - unlock the file table - unlock the two mutexes Attempting to acquire the second mutex introduced the possibility of deadlock because another thread could attempt to acquire the first mutex while holding the file table lock. This commit ensures that multiple mutexes are either: - acquired in a single lock of the file table - or, only acquired after releasing previously held mutexes Fixes: nodejs/uvwasi#89 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Original commit message: prevent race conditions with uvwasi_fd_close() uvwasi_fd_close() performed the following operations: - lock the file descriptor mutex - close the file - release the file descriptor mutex - call the file table's remove() function Once the fd's mutex is released, another thread could acquire it before the fd is removed from the file table. If this happens, remove() could destroy a held mutex. This commit updates uvwasi_fd_close() to perform the entire sequence while holding the file table's lock, preventing new acquisitions of the fd's mutex. Fixes: nodejs/uvwasi#88 PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

This test provides missing coverage for __wasi_fd_renumber(). PR-URL: #31432 Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: David Carlier <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

nodejs-github-bot added the wasi Issues and PRs related to the WebAssembly System Interface. label Jan 20, 2020

tniessen approved these changes Jan 21, 2020

View reviewed changes

bnoordhuis approved these changes Jan 21, 2020

View reviewed changes

This was referenced Jan 21, 2020

unlock all fd mutexes in reverse order nodejs/uvwasi#87

Merged

Potential uvwasi_fd_table_renumber() deadlock nodejs/uvwasi#89

Closed

devnexen approved these changes Jan 21, 2020

View reviewed changes

cjihrig mentioned this pull request Jan 22, 2020

prevent locking fd table while holding a mutex nodejs/uvwasi#90

Merged

cjihrig added 5 commits January 22, 2020 20:09

deps: update to uvwasi 0.0.5

978f477

This version improves file descriptor renumbering, and as a result fixes uvwasi_fd_renumber().

deps: uvwasi: cherry-pick ea73af5

6ec9a92

Original commit message: unlock all fd mutexes in reverse order Some functions acquire mutexes for multiple file descriptors. This commit ensures that the mutexes are released in the reverse order that they are aquired.

test: add wasi test for freopen()

1dc6d2f

This test provides missing coverage for __wasi_fd_renumber().

cjihrig force-pushed the uvwasi branch from 63210b3 to 1dc6d2f Compare January 23, 2020 01:28

This comment has been minimized.

Sign in to view

Trott approved these changes Jan 23, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

cjihrig added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jan 23, 2020

gengjiawen approved these changes Jan 23, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

Trott closed this Jan 23, 2020

cjihrig deleted the uvwasi branch January 23, 2020 14:49

codebytere mentioned this pull request Feb 17, 2020

v13.9.0 proposal #31837

Merged

codebytere mentioned this pull request Mar 17, 2020

v12.16.2 proposal #32313

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deps: update to uvwasi 0.0.5 #31432

deps: update to uvwasi 0.0.5 #31432

cjihrig commented Jan 20, 2020

bnoordhuis Jan 21, 2020

bnoordhuis Jan 21, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Jan 23, 2020 •

edited by Trott

Loading

Trott commented Jan 23, 2020

deps: update to uvwasi 0.0.5 #31432

deps: update to uvwasi 0.0.5 #31432

Conversation

cjihrig commented Jan 20, 2020

Checklist

bnoordhuis Jan 21, 2020

Choose a reason for hiding this comment

bnoordhuis Jan 21, 2020

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Jan 23, 2020 • edited by Trott Loading

Trott commented Jan 23, 2020

nodejs-github-bot commented Jan 23, 2020 •

edited by Trott

Loading