-
Notifications
You must be signed in to change notification settings - Fork 239
Add WASI reactor build and re-entrant event loop APIs #1308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Rework of #1307 - this exposes quickjs-ng as a library, with all of the C symbols properly exported. |
53c1239 to
f9b3d57
Compare
| target_link_options(qjs_wasi PRIVATE | ||
| -mexec-model=reactor | ||
| # Memory management | ||
| -Wl,--export=malloc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe a better way to do this is to use __attribute__((export_name(...))) although that doesn't quite mesh with how we currently export symbols.
Right now, we do:
JS_EXTERN void JS_Foo(JSContext *ctx);Where JS_EXTERN is __attribute__((visibility("default"))). For export_name we'd need something additional, like e.g.:
#ifdef __wasm__
#define JS_EXPORT(name) __attribute__((export_name(#name))) name
#else
#define JS_EXPORT(name) name
#endifAnd then it's used like this:
JS_EXTERN void JS_EXPORT(JS_Foo)(JSContext *ctx);Which honestly is pretty ugly but is arguably better than trying to keep exports lists in sync manually.
@saghul wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My thinking after #1307 was that we could maybe cop out :-) That is, we expose the 2 extra functions in libc and then someone could built a tiny wrapper library exposing whichever entrypoints deemed appropriate, think some qjs-reactor.wasm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was able to reduce the CMakeLists.txt LOC by using --export-dynamic which automatically exports all symbols w/ default visibility.
Add js_std_loop_once() and js_std_poll_io() to quickjs-libc for embedding QuickJS in host environments where the host controls the event loop (browsers, Node.js, Deno, Bun, Go with wazero, etc). The standard js_std_loop() blocks until all work is complete, which freezes the host's event loop and prevents host callbacks from running. The new APIs enable cooperative scheduling: js_std_loop_once() - Run one iteration of the event loop (non-blocking) - Executes all pending promise jobs (microtasks) - Runs at most one expired timer callback - Returns >0: next timer fires in N ms, use setTimeout - Returns 0: more microtasks pending, call again immediately - Returns -1: idle, no pending work - Returns -2: error occurred js_std_poll_io(timeout_ms) - Poll for I/O and invoke read/write handlers - Separate from loop_once so host can call it only when I/O is ready - Avoids unnecessary poll() syscalls when host knows data is available - Required because loop_once only handles timers/microtasks, not I/O - Returns 0: success, -1: error, -2: exception in handler Add QJS_WASI_REACTOR cmake option that builds QuickJS as a WASI reactor module. Unlike the command model (which has _start and blocks in js_std_loop), reactors export library functions that can be called repeatedly by the host. Build: cmake -B build -DCMAKE_TOOLCHAIN_FILE=.../wasi-sdk.cmake -DQJS_WASI_REACTOR=ON cmake --build build --target qjs_wasi Output: qjs.wasm (reactor module with exported quickjs.h / quickjs-libc.h APIs) The reactor wasm is included in releases as qjs-wasi-reactor.wasm. Signed-off-by: Christian Stewart <[email protected]>
dd6edb6 to
e63c573
Compare
Address review feedback to eliminate code duplication between js_std_poll_io() and js_os_poll(). Both functions now call a shared js_os_poll_internal() with configurable behavior via flags: - JS_OS_POLL_RUN_TIMERS: process timer callbacks - JS_OS_POLL_WORKERS: include worker message pipes in poll - JS_OS_POLL_SIGNALS: check and dispatch pending signal handlers js_os_poll() passes all flags for full event loop behavior. js_std_poll_io() passes no flags to poll only I/O handlers, ensuring it does not unexpectedly run timers, worker handlers, or signal handlers. This reduces code by ~40 lines and ensures bug fixes apply to both paths. Signed-off-by: Christian Stewart <[email protected]>
e63c573 to
cf50ed8
Compare
Replace the long list of -Wl,--export=<symbol> flags with -Wl,--export-dynamic
which automatically exports all symbols with default visibility. Since JS_EXTERN
is defined as __attribute__((visibility("default"))), all public API functions
are exported automatically.
Only libc memory functions (malloc, free, realloc, calloc) still need explicit
exports since they don't have default visibility in wasi-libc.
This addresses review feedback about keeping export lists in sync manually and
reduces the CMakeLists.txt WASI reactor section from ~80 lines to ~15 lines.
Signed-off-by: Christian Stewart <[email protected]>
|
I tried minimizing (removing) the init_argv routine but had to add it back eventually. Setting up the module loader requires calling The
This lets hosts do: Now the question is how to reduce duplicated code by re-using the arg parsing logic from qjs.c. I am looking into that. Ok, I put a commit that does that as well. This is much more of a change than I had intended originally in #1307 - but this should at least be a good starting point to discuss what the "right" solution is. |
6c305d7 to
4dd4be3
Compare
The WASI reactor build exports raw QuickJS C APIs, but setting up the module
loader requires calling JS_SetModuleLoaderFunc2() with js_module_loader - a C
function pointer that cannot be obtained or called from the host side (Go,
JavaScript, etc). Without the module loader, dynamic import() fails.
Add qjs_init_argv() to qjs.c which initializes the reactor with CLI argument
parsing (like main() but without blocking in the event loop). This:
- Creates runtime and context
- Sets up the module loader via JS_SetModuleLoaderFunc2()
- Sets up the promise rejection tracker
- Parses CLI flags: --std, -m/--module, -e/--eval, -I/--include
- Loads and evaluates the initial script file
The functions are added to qjs.c (guarded by #ifdef QJS_WASI_REACTOR) rather
than quickjs-libc.c because they depend on static functions in qjs.c like
eval_buf(), eval_file(), parse_limit(), and JS_NewCustomContext().
Exported functions:
- qjs_init() - Initialize with default args
- qjs_init_argv(argc, argv) - Initialize with CLI args
- qjs_get_context() - Get JSContext* for use with js_std_loop_once etc
- qjs_destroy() - Cleanup runtime
Example usage from host:
qjs_init_argv(3, ["qjs", "--std", "/boot/script.js"])
while running:
result = js_std_loop_once(qjs_get_context())
// handle result
qjs_destroy()
Signed-off-by: Christian Stewart <[email protected]>
e6127d5 to
6cc707e
Compare
Extract common initialization logic from qjs.c into reusable functions that can be shared with the WASI reactor build. This eliminates code duplication between main() and qjs_init_argv(), reducing the reactor file by 45%. Add qjs-common.h with public API: - QJSConfig struct for initialization options - qjs_config_init() / qjs_config_parse_args() for CLI parsing - qjs_setup_runtime() for module loader and promise tracker setup - qjs_apply_config() for loading std modules, includes, and eval - qjs_new_context() (formerly static JS_NewCustomContext) - qjs_eval_buf() / qjs_eval_file() / qjs_parse_limit() Changes to qjs.c: - Make eval_buf, eval_file, parse_limit, JS_NewCustomContext non-static - Add shared helper functions for runtime/context initialization - Guard main() with #ifndef QJS_NO_MAIN for reactor builds The reactor now uses these shared functions instead of duplicating the argument parsing and initialization code. This also removes the #include "qjs.c" hack that was used to access static functions. Signed-off-by: Christian Stewart <[email protected]>
6cc707e to
aa950c0
Compare
|
This is quite a large change so I understand if you reject it outright, but for the sake of discussion... This pulls out all of the common logic from main() like parsing args into a struct, etc, so we can re-use it. Overall cleaner result but the changeset appears large (Due to moving things around) |
bnoordhuis
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly LGTM but I don't know about that last commit. @saghul?
| .tv_sec = min_delay / 1000, | ||
| .tv_nsec = (min_delay % 1000) * 1000000L | ||
| }; | ||
| nanosleep(&ts_sleep, NULL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nanosleep can return early with EINTR. If you want to handle that, it should do something like this:
uint64_t mask = os_pending_signals;
while (nanosleep(&ts_sleep, &ts_sleep)
&& errno == EINTR
&& mask == os_pending_signals);
return 0;(os_pending_signals is racy but that's being tracked in #614.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, I will fix that!
Add js_std_loop_once() and js_std_poll_io() functions to quickjs-libc for re-entrant event loop control. These enable embedding QuickJS in host environments where the host controls scheduling (browsers, Node.js, Deno, C programs).
js_std_loop_once() runs one iteration of the event loop:
js_std_poll_io() polls for I/O and invokes read/write handlers:
Add QJS_WASI_REACTOR cmake option that builds QuickJS as a WASI reactor module, exporting the quickjs.h and quickjs-libc.h library functions. Unlike the command model (which has _start and blocks), reactors export functions that can be called repeatedly by the host.
Build with:
cmake -B build -DCMAKE_TOOLCHAIN_FILE=.../wasi-sdk.cmake -DQJS_WASI_REACTOR=ON
cmake --build build --target qjs_wasi
Output: qjs.wasm (reactor module)
The reactor wasm will be included in releases as qjs-wasi-reactor.wasm.