[wasm64] Fix WebSocket API under wasm64 #21514
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dschuff
approved these changes
Mar 11, 2024
kripken
reviewed
Mar 13, 2024
| @@ -349,3 +349,11 @@ def test_posix_proxy_sockets(self): | |||
| with PythonTcpEchoServerProcess('7777'): | |||
| # Build and run the TCP echo client program with Emscripten | |||
| self.btest_exit('websocket/tcp_echo_client.c', args=['-lwebsocket', '-sPROXY_POSIX_SOCKETS', '-pthread', '-sPROXY_TO_PTHREAD']) | |||
|
|
|||
|
|
|||
| class sockets64(sockets): | |||
There was a problem hiding this comment.
If you mean, is it run by CI anywhere, no, I just rand it locally.
lewing
pushed a commit
to dotnet/emscripten
that referenced
this pull request
Apr 12, 2024
* embind: Ensure embind works under c++11. (emscripten-core#21100) We no longer need to test c++17 since that is the default used, but we should ensure we maintain compatibility with c++11. * Fix C++ function signatures in mimalloc (emscripten-core#21127) The C++ functions that take `std::nothrow_t` all take a reference. Without this change I'm seeing function signature mismatches because the final argument is not pointer-width. I'm guessing this simply went unnoticed because most platforms don't do signature checking in the linker and this just happens to work. e.g. `_ZnwjRKSt9nothrow_t` which demangles to `operator new(unsigned int, std::nothrow_t const&)` See upstream bug at microsoft/mimalloc#840. * Bump github/codeql-action from 2.22.5 to 3.23.1 (emscripten-core#21134) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.22.5 to 3.23.1. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@74483a3...0b21cf2) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [test] Pass filename explicitly to `compile_btest` helper. NFC (emscripten-core#21103) This allows us to use `compiler_for` to determine whether to run `EMCC` or `EMXX` appropriately. * Remove some unnecessary complexity in `create_receiving`. NFC (emscripten-core#21131) * Fix makeBlendState assert for non-existent nextInChain (emscripten-core#21119) * Use single quotes in JS code where possible. NFC (emscripten-core#21133) For some reason this actually saves on code size too. * WebGPU: Fill out webgpu-externs.js (emscripten-core#21144) Fixes test browser.test_webgpu_basic_rendering_closure by fixing wgpuShaderModuleGetCompilationInfo (compilationInfo.messages.length). Fixes: emscripten-core#20415 * Remove second argument to `ErrnoError`. NFC (emscripten-core#21136) There was only one single place (out of hundreds) where this was set and one single place in `doStat` where it was read. I removed the `doStat` workaround by updating `mayLookup` instead. * [Docs] Mention information that may exist in unoptimized builds (emscripten-core#21147) Fixes emscripten-core#21084 * Convert test_stat_fail_alongtheway to a file based test (emscripten-core#21138) Also remove the special logging and just use assert which fails fast and gives a backtrace. * Minor webidl binder fixes (emscripten-core#21152) Split out from emscripten-core#21151. The only semantic change here is the fix the type of second argument to to the array member getter function (previously it was using `m.type` which is the array type itself, but that second argument to the setter function is actually the inner type `m.type.inner`. This enables the type of this argument to be checked correctly, which in turn required a minor fix to the test case. This only effects `IDL_CHECKS=all` builds. * [ci] Pin firefix to dev-edition until the threading issue is resolved. NFC (emscripten-core#21164) See emscripten-core#21163 * Bump expected LLVM version from 18 to 19. NFC (emscripten-core#21165) * Add LLVM version bump to ChangeLog. NFC (emscripten-core#21166) * Remove use of `demangleAll` helper function (emscripten-core#21156) This was used to demangle C++ symbols JS stacktraces. However for a long time now native symbols in JS backtraces are already demanged because names in the name section are in demangled form. So this function has not done anything for a very long time. As a followup I think we can probably completely remove the `DEMANGLE_SUPPORT` settings. * Rebaseline codesize expectations. NFC * Rebaseline codesize expectations. NFC * Use JavaScript class for FS.ErrnoError. NFC (emscripten-core#21149) * [wasm64] Update webidl to support wasm64 (emscripten-core#21151) * Rebaseline codesize expectations. NFC * Acorn optimizer: Allow top-level await expressions (emscripten-core#21117) `-pthread` + `-sEXPORT_ES6` emits a top-level await expression in the main worker file after commit 49ab23a. Ensure the Acorn optimizer is aware of this. * fix wgpu get dimension bug (emscripten-core#21173) Co-authored-by: Jerry.Zhuang <[email protected]> * Avoid unnecessary dependency on `SYSCALLS` object. NFC (emscripten-core#21170) I noticed that `proc_exit` was depending on `SYSCALLS` even though it wasn't actually using `SYSCALLS`. * embind: Fix method pointers for AOT JS generation. (emscripten-core#21168) Previously, AOT JS glue code for method pointers used the address of the method pointer as a unique ID to match JS glue with the binding. However, sometimes static initializers malloc different sizes when running the AOT generation code versus when the code is run normally. This caused the method pointer address to be different and the binding didn't match up. Instead of using the function pointer or the method pointer address we now use a generated signature for the invoker to match up the glue code. This works for both types of pointers and also has the advantage that many of the glue code functions are nearly identical and can be reusued (closure arguments make them behave differently). Fixes emscripten-core#20994 * [wasm64] Error on `-fsanitize=address` + wasm64 (emscripten-core#21180) This chance also includes a couple of minor fixes to asan + memory64 that helped me to confirm that it really doesn't work yet. See emscripten-core#21177 * [wasm64] Enable freetype test. NFC (emscripten-core#21179) The poppler test also seesm to work fine under wasm64, just not the >4Gb mode. * [ci] Split browser64_4gb into its own runner (emscripten-core#21182) Also add test_TextDecoder to the list of test. * [wasm64] Fix glSharedSource in >4gb mode and enable test under wasm64 (emscripten-core#21184) * Add profiling to `read_ports`. NFC (emscripten-core#21194) See emscripten-core#21191 * Remove unused arguments from test_emmalloc_trim. NFC (emscripten-core#21181) This test was originally added with these unused args. Perhaps while it was being developed in was parameterized, but is not anymore. * Fix GL.getSource bug introduced in emscripten-core#21184 (emscripten-core#21197) This only seems to show up in `browser.test_sdl_ogl_regal`. * Move emscripten_webgl_init_context_attributes to native code. NFC (emscripten-core#21189) * Rebaseline codesize expectations. NFC * Allow-list 7 more WebGL extensions (emscripten-core#21185) These 6 extensions don't add any entry points, so shouldn't require any Emscripten support (aside from having any enums present in gl2ext.h, which they do): - EXT_conservative_depth - NV_shader_noperspective_interpolation - EXT_depth_clamp - WEBGL_blend_func_extended - WEBGL_compressed_texture_etc - WEBGL_compressed_texture_etc1 Finally: - WEBGL_compressed_texture_astc does add an entry point getSupportedProfiles() in WebGL. This is a WEBGL_ extension, so it is still safe to allow-list without implementing a C version of that function. There is no corresponding GLES entry point for this; instead, profile support is indicated by the presence of KHR_texture_compression_astc_{ldr,hdr}. Emscripten should emulate those extensions by checking getSupportedProfiles(), but I haven't implemented this. Applications can still use the WEBGL_ version directly. WEBGL_compressed_texture_astc guarantees support for the LDR profile (since HDR is a superset of LDR, and there are only two profiles), so applications that only need LDR don't need to call this function. It can also be easily called using EM_ASM in applications that do use the HDR profile. Fixes emscripten-core#21183 * Run unsafe_optimizations.js again after closure (emscripten-core#21196) This allows `unsafe_optimizations.js` to cleanup some thing that closure compiler was able to remove usages of. Split out from emscripten-core#21195 * Mark test_itimer_proxy_to_pthread as flaky. NFC (emscripten-core#21175) This test recently flaked on window too. * Cleanup formating in audio_worklet JS code. NFC (emscripten-core#21203) * Mark 3.1.53 as released (emscripten-core#21211) * Don't force the use of imported memory under WASM2JS (emscripten-core#21195) Setting `IMPORTED_MEMORY` can have other implications and don't want t by default just because we chose WASM2JS. The there is a very minor code size regression here that only effects WASM2JS, but when combined with emscripten-core#21196 the overall effect is still a win. See emscripten-core#21071 * Use arrow function in library_webgl.js. NFC (emscripten-core#21186) * Handle `-Wl,--stack-first` being passed explicitly (emscripten-core#21216) Fixes: emscripten-core#21215 * [debug] Use two digits for the temporary file counter. NFC (emscripten-core#21207) This means that filenames will sort correctly under normal sorting rules. * [wasm64] Fix fetch api under wasm64 (emscripten-core#21219) See emscripten-core#21177 * Rename/cleanup fetch tests. NFC (emscripten-core#21218) * Make use of `emscripten_supports_offscreencanvas`. NFC (emscripten-core#21204) * Use JS naming convention for JS-only symbol. NFC (emscripten-core#21188) The `emscripten_` naming convention is something we use for native function and this symbol is not visible to native code. * Run more of out tests with `-sNO_DEFAULT_TO_CXX` (emscripten-core#21096) We were previously only applying this flag to tests that use `self.build` (i.e. most test). Now we apply it to and even larger set. The code size changes here only occur in debug builds where we enumerate all known JS symbols, and there are come JS symbols are the C++ only. * Some fixes for library_webgl.js in CAN_ADDRESS_GB mode (emscripten-core#21220) This is an overall size win too after compression. * When parsing settings as JSON, check the types we get back (emscripten-core#21206) * Rebaseline codesize expectations. NFC * More use of expectToReceiveOnModule in library_browser.js. NFC (emscripten-core#21224) * Fix emscripten_webgl_make_context_current under wasm64/CAN_ADDRESS_2GB (emscripten-core#21228) The first problem here was that emscripten_webgl_make_context_current_calling_thread had the wrong signature. The second issue was use of `>> 2` on a pointer in `emscripten_webgl_do_create_context` which doesn't work under wasm64 or CAN_ADDRESS_2GB. * [embuilder] Add support for '*' wildcard for matching library names (emscripten-core#21236) This is very useful for doing things like `./embuild build libGL*` to `./embuilder clean libc*`. * [test] Remove also_wasm2js btest argument. NFC (emscripten-core#21232) This only have one user, and we have a cleaner mechanism for doing this these days in terms of the decorator. * Add --use-port option (emscripten-core#21214) * [wasm64] Fix glUniformMatrix4fv (emscripten-core#21235) * embind - Fix reading various bindings types from val on wasm64. (emscripten-core#21225) - The function simpleReadValueFromPointer was always reading pointers as i32 numbers which was incorrect on wasm64. - Decoding of std::wstring was using a right shift to read pointers. Right shift was truncating numbers larger than i32 and causing misreads. - Added tests to execute readValueFromPointer on all the binding types to ensure these don't break on wasm64. * Move internal webgl signatures to a separate file. NFC (emscripten-core#21239) This is in preparation for enabling these functions under wasm64. * Avoid redundant function declarations in gl.c. NFC (emscripten-core#21237) * Use @parmeterize in more browser tests. NFC (emscripten-core#21230) * embind: Use optional return type for vector and maps. (emscripten-core#21102) This helps create better TypeScript definitions for what is actually returned. * [test] Make more use of `also_with_wasm2js`. NFC (emscripten-core#21242) * Support WGPUPrimitiveDepthClipControl / unclippedDepth (emscripten-core#21231) Co-authored-by: Kai Ninomiya <[email protected]> * [wasm64] Fix library_glemu.js under wasm64 (emscripten-core#21241) * Fix test_cubegeom_pre_relocatable (emscripten-core#21240) This test was broken by emscripten-core#21103 but when unnoticed because we run our browser tests with swiftshader enabled, which skips this test. * Consistent use of ValueError when parsing settings. NFC (emscripten-core#21245) This is in preparation for a change to avoid using json parsing for settings by default. It also adds more context to the errors since when the ValueError is caught higher up we print nice error messages. * [wasm64] Fix wasm64 memory read in Fetch.js (emscripten-core#21246) This bug only showed up under wasm64 when the address of the fetch object was between 2Gb and 4Gb. This causes the JS ">> 2" operation to generate a negative number becuase the high bit is set: ``` $ node > a = 2**31 + 10 2147483658 > a >> 2 -536870910 > ``` In `browser64_4gb` mode this bug resulted in a read from the first 4gb of memory somewhere, which results a in 0 whereas read from a negative address yields `undefined`. * Fix broken compilation of regal tests (emscripten-core#21248) This should have been part of emscripten-core#21096. These failures don't show up in CI because they are disabled under swiftshader. * Minor formatting cleanup in src/library_glemu.js. NFC (emscripten-core#21249) * Fix glDrawElements under CAN_ADDRESS_2GB (emscripten-core#21247) * Re-enable preprocessing of pre/post JS file. NFC (emscripten-core#21227) This change got reverted in emscripten-core#19006 so this time we make the preprocessing optional. This is useful as it allows things like `{{{ POINTER_SIZE }}}` and `{{{ makeGetValue(..) }}}` to be used in pre/post JS files, just like they can be in JS library files. This change allows threadprofiler.js to be fixed such that it works under wasm64. Fixes: emscripten-core#21226 * Fix glDrawElements under CAN_ADDRESS_2GB (emscripten-core#21251) This is partially a workround for emscripten-core#21250 * Fix capitalization of "GitHub". (emscripten-core#21257) * docs: Remove discussion of using Travis CI. (emscripten-core#21256) These are outdated and the docker images mentioned here haven't been updated in 4 years. * [test] Simplify post_manual_reftest. NFC (emscripten-core#21243) Rather than embedding `reftest.js` directly in the html, simply include it via script src attribute. * Add contrib ports mechanism (emscripten-core#21244) * Simplify get_subresource_location. NFC (emscripten-core#21260) This function only ever gets called with no arguments. I also confirmed that when we use a memory init file, that file always exists. * Improve manual_wasm_instantiate.html (emscripten-core#21258) Modernize this test a little by using fetch API and JS spread operator. * [Docs] Mention emscripten_exit_with_live_runtime wrt pthreads (emscripten-core#8167) * Cleanup test_async_compile. NFC (emscripten-core#21259) * Fix glUniformMatrix4fv in CAN_ADDRESS_2GB mode (emscripten-core#21264) * Fix "emscripten emit" typo in Optimizing Code - WebAssembly section (emscripten-core#21269) * Use JS string templates in src/library_glemu.js. NFC (emscripten-core#21261) * Use rest and spread operators in multi-threaded code. NFC (emscripten-core#21270) These operators are available everywhere that we can run mutli-threaded code. * [Docs] Added note to clarify new glfw3 port (emscripten-core#21272) * Support multiple argument in pthread err/dbg wrappers (emscripten-core#21266) * Cleanup generate_traditional_runtime_html. NFC (emscripten-core#21267) - Use fetch() over XHR. - No need for tryParseAsDataURI. This code is already in a `not settings.SINGLE_FILE` block which means URL for the wasm binary will never be data URI. See `get_subresource_location`. * WebGPU: Swapchain getCurrentTexture (emscripten-core#21271) * WebGPU: added GetCurrentTexture method to swapchain. * WebGPU: fix for GetCurrentTexture signature. * Revert "[ci] Pin firefix to dev-edition until the threading issue is resolved. NFC (emscripten-core#21164)" (emscripten-core#21282) This reverts commit 0eb6567. The upstream has been fixed: https://bugzilla.mozilla.org/show_bug.cgi?id=1876355#c3 * [test] Use modern list format for `-s` setting. NFC (emscripten-core#21280) * Fix `#preprocess` detection on windows (emscripten-core#21284) Followup to emscripten-core#21227 * Update EMSCRIPTEN_WEBGL_CONTEXT_HANDLE to handle full pointer range (emscripten-core#21281) In particular this changes the return type of `emscripten_webgl_create_context` such that negative values can no longer be returned. This allows pointers (specifically 32-bit pointers over 2gb) to be returned which is needed for pthread builds. The only valid failure return code for `emscripten_webgl_create_context` is now 0/NULL. As it happens this function never fails with anything except 0 so this should not be an impactful change. Split out from emscripten-core#21268 Fixes: emscripten-core#21278 * [Docs] Document EXPORT_ES6 implicit behavior (emscripten-core#21274) * It is implicitly enabled for .mjs output. * MODULARIZE is implicitly enabled when EXPORT_ES6 is set. * Remove redundant step in promise change. NFC (emscripten-core#21285) This stop in the chain is only need if the offset converter is needed. * Fix some typos [NFC] (emscripten-core#21275) * Make use of JS rest and spread operators throughout the codebase. NFC (emscripten-core#21283) Following up on emscripten-core#21270. These operators were introduced way before our current default set of browser versions. Anyone targeting an browser so old as to not support these will already be on the transpilation path. * [NFC] Assert we do not use --low-memory-unused when STACK_FIRST (emscripten-core#21291) Assuming low memory was unused when the stack was first - which is where low memory is - would be wrong. We never did that wrong thing as we only use that optimization when not optimizing. Add an assertion + tests to prevent regressions. * Cleanup some pthread tests. NFC (emscripten-core#21293) * Refactor Emval implementation to avoid JS heap allocations. (emscripten-core#21205) * Rebaseline codesize expectations. NFC * Fix deadlock in test_pthread_run_on_main_thread (emscripten-core#21296) Fixes: emscripten-core#18210 * [test] Remove some unneeded `-sINITIAL_MEMORY` settings. NFC (emscripten-core#21295) I think the reason a lot of these are no longer needed is because the default pthread stack size is less that it used to be. * Fix broken JS glue for AUDIO_WORKLETS with EXPORT_ES6 (emscripten-core#21192) WASM Audio Worklets with EXPORT_ES6 may break at runtime: test.js:989 Uncaught (in promise) TypeError: Cannot set property wasmTable of #<Object> which has only a getter at receiveInstance (test.js:989:25) at receiveInstantiationResult (test.js:1011:5) The read-only getter at issue is created in ASSERTIONS-enabled builds, and conflicts with the current way of exporting wasmTable on the Module object. Exporting wasmTable via EXPORTED_RUNTIME_METHODS prevents the getter from being created in normal builds. In MINIMAL_RUNTIME builds, we make sure to delete the getter before manually exporting as before. We also prevent an ES6 Audio Worklet from loading the .wasm binary via `new URL()`, as `URL` is unavailable in AudioWorkletGlobalScope. * Disable tests that rely on changing output from size and llvm-objdump (emscripten-core#21298) * Add options support to contrib ports (emscripten-core#21276) * Fix MAIN_THREAD_EM_ASM_INT + CAN_ADDRESS_2GB (emscripten-core#21292) We were using the sign bit of an integer to distinguish between data pointers and fixed JS function indexes, but that doesn't work once that data address can be larger than 2^31. Technically this is very unlikely in practice since in order to get an EM_ASM address over 2^31 you would either need 2Gb of static data to be using `-sGLOBAL_BASE=2gb` like we do in the tests. An alternative approach here would be assume we have fewer than `GLOBAL_BASE` (1024 is most cases) proxied JS library functions and then we could assume that small integers we JS library functions and larger ones were data pointers (EM_ASM functions). However that seems fragile too. Passing an extra argument around seems like a small cost here. * WebGPU: Remove unused signalValue argument (emscripten-core#21299) This was never implemented and was removed from Dawn and upstream webgpu.h a long time ago (and the JS API even longer ago). (This will be a tiny breaking change for a few apps but we're not stable and still need to do many more breaking changes.) * Rebaseline codesize expectations. NFC * Move __glGenObject to GL.genObject. NFC (emscripten-core#21302) This function logically belongs alongside getNewId. Split out from emscripten-core#18874 * Rebaseline codesize expectations. NFC * [test] Fix test_pthread_run_on_main_thread under asan. NFC (emscripten-core#21301) The output of this test is racey without this change and has been failing on the emscripten-releases waterfall in the asan configuration. We only just started caring about the precise output in emscripten-core#21296. * [test] Remove some unnecessary usage of TOTAL_MEMORY and INITIAL_MEMORY. NFC (emscripten-core#21303) * Fix remaining webgl + CAN_ADDRESS_2GB issues (emscripten-core#21306) * Add --quiet option to file_packager (emscripten-core#21307) This suppressed the `Remember to build the main file with ..` message that is otherwise printed to stderr. This also allows the tests to assert that no stderr is produced in successful cases. * [wasm64] Fix emscripten_get_preloaded_image_data (emscripten-core#21308) Here we extract an inner JS function that take a JS string so that the externally facing once can assume its being passed a pointer. * Link to bug in test_webgl_multi_draw test. NFC (emscripten-core#21309) * Add option to emit TypeScript definitions for Wasm module exports. (emscripten-core#21279) The new flag `--emit-tsd <filename>` will generate a TypeScript defintion file for any Wasm module exports. If embind is also used the definitions for those types will also be included in the same file. This still doesn't give the full picture of Wasm module e.g. missing HEAP<N> and the various helper functions defined in JS. * Fix wasm worker tests in high memory modes (emscripten-core#21319) * Don't define JS versions of abort or __assert_fail in standalone mode (emscripten-core#21317) In STANDALONE_WASM mode these functions are defined in libstandalonewasm and not in JS. The JS definitions meant they were not able to be added to EXPORTED_FUNCTIONS. Fixes: emscripten-core#21312 * embind- Fix tsgen failure from merge. (emscripten-core#21320) Forgot to pull in the upstream branch before updating this test. * Don't include `main.c` in libSDL_mixer (emscripten-core#21322) This was broken when sdl2_mixer was updated recently in in emscripten-core#21013. Fixes: emscripten-core#21321 * Run all browser tests in 2gb mode (emscripten-core#21268) * Update emsymbolizer and emsize tests (emscripten-core#21305) LLVM rev 8b0f47bfa updated section addresses to use binary offsets in linked wasm files, which is reflected in the llvm-size (and thus emsize) tool. It also made llvm-objdump display file offsets rather than section offsets when disassembling binaries. Update the emsize and emsymbolizer tests accordingly (expect section addresses in emsize, and use file rather than section offsets in the emsymbolizer test, since the offsets are read from objdump output). * [wasm64] Fix makeHEAPView for addresses over 4gb (emscripten-core#21324) This enables all the cubegeom tests to pass * Rebaseline codesize expectations. NFC * Fix typo in dynamic linking debug prints (emscripten-core#21329) * ChangeLog.md typo: INCOMING_MODULE_API should be INCOMING_MODULE_JS_API (emscripten-core#21314) * embind - Disable wasm2js when emitting types. (emscripten-core#21333) Fixes emscripten-core#21315 * Refactor `makeHEAPView` to use `getHeapOffset`. NFC (emscripten-core#21332) The code for calculating the offsets was being duplicated here. Also, optimize getHeapOffset to avoid any shifting when size is 1. * Add suppport for external ports (emscripten-core#21316) * [test] Improve test_sdl2_key and test_sdl2_mouse. NFC (emscripten-core#21336) The TEST_EMSCRIPTEN_SDL_SETEVENTHANDLER macro is copied over from `test_sdl_key.c` (the SDL1 version) but its not applicable with SDL2. Remove the 3 second delay from test_sdl_mouse/test_sdl2_mouse. I ran these tests many times without any issues. We can perhaps revive with something less long but I can't really see how a delay here could be useful in practice. * library_idbfs.js: Handle `transaction.onabort` in `reconcile()` (emscripten-core#21326) If the transaction aborts (which can happen due to, for example, a QuotaExceededError), `onabort` must be handled to ensure that the `callback` actually gets called. * [wasm64] Run all browser tests in 4gb mode (emscripten-core#21334) * Allow comments in response files. (emscripten-core#21330) It can be useful to have comments in a response file for exported symbols to explain what some stuff might be in there for. * Site: Remove old, unused copy of theme layout (emscripten-core#21338) This was added in 2014, not used. * Fix asan.test_main_thread_em_asm_pthread (emscripten-core#21339) Without this change this tests fails to compile with `em++: error: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly`. This test was recently added in emscripten-core#21292 and this wasn't caught immediately because we don't run the `asan` suite during CI. * Site: Fix typos (emscripten-core#21344) * Add external ports + options support to embuilder (emscripten-core#21345) * Update SDL2 version from 2.24.2 to 2.26.0 (emscripten-core#21337) * Move `demangle` helper to library_legacy.js (emscripten-core#21346) Followup to emscripten-core#21156 We no longer use this function anywhere in emscripten. * [embind] Minor refactor to use more modern JS features (emscripten-core#21331) * Unify handling of deprecated settings (emscripten-core#21355) Unlike `LEGACY_SETTINGS`, deprecated settings can continue to be used and the values passed in will continue to be honored. Once folks have stopped using them we can move them to `LEGACY_SETTINGS`. * Warn of usage of legacy library symbols (emscripten-core#21357) * Rebaseline codesize expectations. NFC * Remove unused src/library_cyberdwarf.js (emscripten-core#21356) The rest of the cyberdwarf support was removed back in emscripten-core#12195 * Mark 3.1.54 as released (emscripten-core#21358) * [test] Simplify new KeyboardEvent calls in test_browser.py. NFC (emscripten-core#21351) Closure knows about these property bags so there is no need to quote the keys. * Consolidate port loading code + embuilder documentation (emscripten-core#21353) * Always free allocaed memory in mmunmap. (emscripten-core#21363) Without this change we were returning early from `__syscall_munmap` if `_munmap_js` failed. This could happen if `fd` was no longer a valid open file descriptor. However, even in this case we still want to go ahead and free any allocated memory. Fixes: emscripten-core#21360 * Use `isfile` rather than `exists` when checking for config files (emscripten-core#21361) This avoids trying parse non-files. Also, explicitly log when we are not using a config file at all. * Rebaseline codesize expectations. NFC * Avoid malloc dependency in tzset (emscripten-core#21375) * Update glfw port to 1.0.5 (emscripten-core#21369) This update includes a fix for a memory corruption bug. * [test] Add EMTEST_SKIP_NETWORK_TESTS (emscripten-core#21380) I was doing some work on an airplane over the weekend and I found this useful. * Feedback from emscripten-core#21380 I accidentally hit submit before applying this change. * Clarify MIN_CHROME_VERSION also applies to modern Edge [NFC] (emscripten-core#18222) * Fix python error when building with audio worklets enabled (emscripten-core#21340) MIN_EDGE_VERSION was removed (emscripten-core#20881), so we do not need it in the feature matrix. * Cleanup cpuprofiler.js. NFC (emscripten-core#21387) - Use mathod syntax - Cleanup long line * Fix typo in embind.js (emscripten-core#21384) nonexistant ==> nonexistent Fixes: emscripten-core#21372 * Avoid calling addCxaCatch when not linking as C++ (emscripten-core#21386) For the test I was unable to figure out a way to generate a call to `__cxa_find_matching_catch` withing hand coding it like this. This doesn't fix emscripten-core#21381 but it makes the error message way less confusing. * Add changelog entry for SDL2 update. NFC (emscripten-core#21390) Followup to emscripten-core#21337 * Update sdl2-mixer from 2.6.0 to 2.8.0 (emscripten-core#21391) * [test] Remove redundante/broken call to glTexImage2D. NFC (emscripten-core#21394) This line was fixed/patched in 2736a26 but I think the old line was mistakenly left in. * Simplify test_gl_subdata and test_float_tex. NFC (emscripten-core#21393) Also, remove test_std_cout_new.cpp completely. This test seems to be based on the code from these other two tests but doesn't seem to server any useful purpose anymore. Its was first added in ad285f6. * [Docs] Fix typos [NFC] (emscripten-core#21395) * Rebaseline codesize expectations. NFC * Run test_subdata in webgl2 mode. (emscripten-core#21398) This involved improving the emulation in glTexSubImage2D. Sadly I needed to disable this new tests in 4gb+ mode since it runs into a chromium bug. * file_packager.py: Handle failure to get cached package (emscripten-core#21328) In certain scenarios (such as when quota is exceeded), it's possible for a metadata entry to exist, but the package data to be missing - and `event.target.result` will be undefined. Check for and properly handle this by calling `errback`, so that the `preloadFallback` gets triggered. * [test] Simplify test_openjpeg. NFC (emscripten-core#21401) This change simplifies the test in 2 ways: 1. Remove the "extra testing" part of the test that tries to run the test again if `ALLOW_MEMORY_GROWTH` is set. The only test modes that have `ALLOW_MEMORY_GROWTH` are the santizier modes and it seems odd for an individual test to decide to disable this. 2. Remove the alternate `is_sanitizing` method of running the test. I believe this was only needed becuase of (1), and so can be safely removed. By removing this complexity this test now has less nesting and is more directly readable and more similar to other tests. * Parameterize test_webgpu_compiletest. NFC (emscripten-core#21402) * Fix webgl tracing issues (emscripten-core#21389) The webgl tracing mechanism was assuming a fixed number of arguments for a given function whereas some webgl function can take variable number of arguments. Fix this by using the rest operator. This is not as slow as using `arguments`. Also, performance is not really an issue here since we are about to serialize all the arguments to string and send to the console which will vastly out weight the cost of using spread here. I believe the comments here about hot and cold functions as well as the comment at about the cost of using `arguments` were copied from the cpuprofiler.js but they don't apply here. Also, avoid serializing the entire heap, which can cause chrome to hang. * [test] Add `reftest` helper method for browser tests. NFC (emscripten-core#21392) This makes it easy distinguish reference tests from other btests. * Fix EM_JS macro in the face of recent llvm change (emscripten-core#21403) A recent change to llvm (llvm/llvm-project#81539) means that the `used` attribute on data forces the segment to be present in the output file, even if it's not used. The usage here in `em_js.h` was assuming that the data was not incldued in the final output. This change makes the symbol non-static instead of used, which is enough to force clang to keep the data around but not the linker (which is exactly the effect we want). * Fix dup in nodefs by refcounting nodefs file descriptors (emscripten-core#21399) * Mark test_itimer_pthread as flaky. NFC (emscripten-core#21404) This should have been marked as such when the other itimer tests were. * Rebaseline codesize expectations. NFC * Use JS class for FSStream, FSNode and LazyUint8Array. NFC (emscripten-core#21406) Converting each of these to classes led to code size savings. * Added a way to get more information a port (emscripten-core#21376) * Fix test_openjpeg under wasm64. NFC (emscripten-core#21408) openjpeg was assuming a different signature for getopt_long which was not problem under wasm32. * [test] Move `requires_wasm2js` to `test/common.js`. NFC (emscripten-core#21412) * Fix some closure compile warnings in JS library code (emscripten-core#21385) * Fix for resize events when building with ASSERTIONS (emscripten-core#21362) Building with ASSERTIONS is failing the safe heap checks when e.detail is undefined. * [emval] Reduce C++ -> JS calls in emscripten::val lifetime management (emscripten-core#21366) Related to emscripten-core#21300, but some obviously good incremental wins to reduce the cost of ref counting: 1. Have C++ consistently avoid inc/dec for the special reserved values that don't need to be counted, this saves decref calls every time such a `val` object goes out of scope. 2. Add an rvalue reference version of toWireType for `emscripten::val` that can transfer ownership to JS. This saves one call to incref and one call to decref for the case of a c++ function with return type `emscripten::val` The cost seems to be single-digit bytes to the WASM. * [test] Make use of `is_wasm2js` helper (emscripten-core#21413) Also, don't check for `WASM2JS` settings. That is a private setting that cannot be set on the command line. * Feedback from emscripten-core#21406. NFC (emscripten-core#21415) * Fix interactive SDL1 tests (emscripten-core#21420) * [ports] upgrade SDL to 2.28.4 (emscripten-core#20417) * embind - Remove node exception handling flag for node 17+. (emscripten-core#21411) Fixes: emscripten-core#21405 * Remove DEMANGLE_SUPPORT from docs (NFC) (emscripten-core#21430) * Revert "Proxy webgpu calls back to the main thread. NFC (emscripten-core#20124)" (emscripten-core#21388) This is a partial revert of 992d1e9 * [emrun] Use GNU command line flag names (emscripten-core#21424) For example `--no-browser` vs `--no_browser`. Continue to support the old style but don't advertise these. * Remove IE workaround in jsStackTrace (emscripten-core#21431) * Extract getIncludeFile function. NFC (emscripten-core#21425) * Round down any canvas-relative mouse event coordinates (fixes SAFE_HEAP asserts) (emscripten-core#21429) When using a custom target the coordinates may be doubles, which causes SAFE_HEAP to assert. These are truncated to avoid this (which is what happens anyway, only now it's explicit). * Cleanup src/library_sdl.js. NFC (emscripten-core#21416) - Wrap long lines - Format comments - Use method syntax - Remove unnecessary `SDL_audio`/`SDL_unicode`/`SDL_ttfContext` accessor functions. * Remove remaining uses of `eval` in the JS compiler (emscripten-core#21423) Use `vm.runInCurrentContext` instead. We already use this in the preprocessor, but not we also use it for macro expansion and loading of the compiler code itself. This change should facilitate future changes to the JS compiler, such as changing the compiler itself to use JS modules. * Run test_metadce_minimal in STRICT mode. NFC (emscripten-core#21438) * Rebaseline codesize expectations. NFC * [Docs] Document how to build and view the website locally (emscripten-core#21434) * Allow HEAPX symbols in EXPORT_RUNTIME_METHODS (emscripten-core#21439) - Factor out updateMemoryViews. - Enable HEAPXX exports to be specified in EXPORT_RUNTIME_METHODS. - Add all HEAP symbosl to EXPORT_RUNTIME_METHODS unless in STRICT mode. Replaces emscripten-core#21407 * Add name section and object symbol table support to emsymbolizer (emscripten-core#21367) With llvm/llvm-project#82083, llvm-symbolizer works correctly with name sections, so emsymbolizer can use it. Also do the same for object files with symbol tables. Object files do not currently work correctly in llvm-symbolizer where code addresses overlap with data addresses; when that is fixed, emsymbolizer will start working, but we should add a test for it. * Add some comments regarding __c_longjmp. NFC (emscripten-core#21442) * [wasm2js] Remove all handling for external memory file (emscripten-core#21217) We have a lot of support code and complexity in emscripten for handling of external memory files. However, its only ever used in wasm2js mode which is a legacy mode. The only reason we continue to support it here IIUC is because its slightly more space efficient than embedding the data as base64. For small programs like hello_world this is an over codesize win. For larger programs there is a regression in overall size in proportion to the amount of static data in the program. * Limit `isCurrentContextWebGL2` usage. NFC (emscripten-core#21444) I observed that this helper macro is only needed when webgl2 is supported. In fact, it would be a bug if it was used when webgl2 was not supported. * Update expected binaryen version to 116 (emscripten-core#21446) After WebAssembly/binaryen#6358, we need to update this to pass the tests. * Update val.h.rst (emscripten-core#21449) Update typo in EM_ASYNC_JS code sample * Update Binaryen version to 117 (emscripten-core#21451) After WebAssembly/binaryen#6358, I tried to update this to 117, but it looks due to some unfinished builds I ended up updating it to 116 (emscripten-core#21446 (comment)). This brings it up to 117. * [circleci] Create `run-tests-firefox` task. NFC (emscripten-core#21452) This allows use to create different firefox test runs, just like we already can for chrome. * Update test_sdl_canvas (emscripten-core#21453) When I run this test locally in firefox I was getting 2785 as the sum value. I believe this is because the test includes SDL1 font rendering which can depend on the system fonts you have installed. * Minor library_webgl.js cleanup (emscripten-core#21457) Split out from emscripten-core#21445 * Another library_webgl.js cleanup. NFC (emscripten-core#21458) Similar to emscripten-core#21457 but this one actually as a code size saving * Avoid garbage-free WebGL APIs when memory size is over 2gb. (emscripten-core#21445) Both chrome and firefox see have some issues with passing 2gb+ and 4gb+ offsets to these APIs. Once the browser issues are addressed we can lift these restrictions over time. Fixes: emscripten-core#20533 * Update GLFW contrib port to v1.1.0 (GLFW 3.4.0) (emscripten-core#21455) On Feb 23rd 2024, GLFW released a new version 3.4.0. This is the changes in contrib.glfw3 (https://github.com/pongasoft/emscripten-glfw) to implement the new apis/features added to 3.4 Release notes for the port: https://github.com/pongasoft/emscripten-glfw/releases/tag/v1.1.0 * Initial new `INITIAL_HEAP` setting (emscripten-core#21071) Changes in default behavior: 1) INITIAL_HEAP is the new default for most builds. This means that there is an increase in the effective initial memory used by "sizeof(stack) + sizeof(static data)". In typical small applications this should be on the order of half a megabyte. 2) Because we cannot precisely calculate the amount of initial memory now, ASAN support will use the conservative upper estimate of MAXIMUM_MEMORY. This only affects ALLOW_MEMORY_GROWTH=0 builds. This change does not yet enable INITIAL_HEAP for builds that instantiate the memory in JS, e. g. with threading. * Fix file:// protocol not working in Ports.fetch_project (emscripten-core#21456) * Mark 3.1.55 as released (emscripten-core#21461) * Improve ES6 support in TypeScript definitions generation (emscripten-core#21460) Users building with --emit-tsd and the EXPORT_ES6 setting should get TypeScript definitions accounting for the fact that ES6 modules include a default export that is a functor to instantiate and access the wasm module in question When generating TS definitions, query the EXPORT_ES6 setting and, if present, add the appropriate exports to the output Include options param in default functor export for `--emit-tsd`. This serves as a starting point for incluing all options such as `locateFile` (and the slew of others), without requiring manually iterating through them Update tests to include factory function * Fix Getting WebGPU Context in pthread (emscripten-core#21467) Failed to run compiled WGPU code clips: wgpu::SurfaceDescriptor surfDesc{}; surfDesc.nextInChain = &canvasDesc; wgpu::Surface surface = instance.CreateSurface(&surfDesc); The reason is that it cannot call getContext from correct canvas object. This PR fixed the problem and add correct build flags for webgpu_basic_rendering test in pthread. * Honor WEBGL_USE_GARBAGE_FREE_APIS (emscripten-core#21462) Followup to emscripten-core#21445. I overlooked some of the webgl2-only paths. * Rebaseline codesize expectations. NFC * [test] Disable asan.test_externref_emjs_dynlink. NFC (emscripten-core#21490) This test is failing due to a crash in LLVM that was introduced in llvm/llvm-project#83196. * [JSPI] - Assert that JSPI is supported by the environment. (emscripten-core#21483) * Fix test_extended_const after recent binaryen change. NFC (emscripten-core#21492) See WebAssembly/binaryen#6379 * Fix for longjmp(.., 0) under wasm EH (emscripten-core#21493) Fixes: emscripten-core#21486 * [wasm64] Enable SDL2 tests under wasm64 (emscripten-core#21489) I think the recent SDL2 version bump must have fixed the issues we were previously seeing. See emscripten-core#21337 and emscripten-core#20417. * Avoid string property access in library_webgpu.js. NFC (emscripten-core#21454) * Multiple fixes to EmscriptenOrientationChangeEvent (emscripten-core#21428) * Add support for "-Ofast" (emscripten-core#21494) Based on emscripten-core#11884 * Fix broken regex from emscripten-core#21492 (emscripten-core#21498) * [wasm64] Enable test_sdl_audio_beep_sleep (emscripten-core#21495) * Warn when setting used on the command line more than once (emscripten-core#21464) See emscripten-core#19938 * [test] Remove EMTEST_LACKS_THREAD_SUPPORT (emscripten-core#21496) All the browsers we test against these days support threads * Convert unsafe_optimizations.js to mjs. NFC (emscripten-core#21504) * [test] Disable asan.test_externref_emjs. NFC (emscripten-core#21508) Should have been part of emscripten-core#21490. * Re-enable test_dlfcn_em_asm. NFC (emscripten-core#21500) This test was disabled due to EM_ASM not working in side modules but that was fixed in emscripten-core#18228. * Remove unused genArgSequence and needsQuoting. NFC (emscripten-core#21505) The usage of genArgSequence removed in emscripten-core#17420. The usage of needsQuoting was removed in emscripten-core#17462 * Avoid generating empty statement in receiveI64ParamAsI53 (emscripten-core#21487) Fixes a few Closure lint warnings. (Though of course there are very many other Closure warnings.) * Convert lz4-compress.js to mjs. NFC (emscripten-core#21507) * Fix acorn-optimizer error when destructuring array with empty elements (emscripten-core#21501) This fixes cases like this: function ([, a]) { } In such arrays it is possible to have empty elements, which must be skipped. * Run JS linter on src/*.mjs files. NFC (emscripten-core#21510) As of today all `.mjs` files are part of the compiler itself and don't contains preproccess macros, so that are safe to be run through linter tools. * [test] Add `skipExecIf` which just skips the running of the test (emscripten-core#21499) This means that even in CI when we might skip certain tests for graphics or sounds we still get some assurance that the test code compiles. * Avoid use of port 8088 in socket tests. NFC (emscripten-core#21512) This port is used by the goma compiler webserver which causes these tests to fail on machines that are running the goma compiler. * Remove check for file type of `.a` files. NFC (emscripten-core#21513) Instead just pass them through the linker like other inputs. See emscripten-core#21128 * [wasm64] Fix WebSocket API under wasm64 (emscripten-core#21514) Fixes: emscripten-core#21503 * Only check for WebAssembly support in debug builds. NFC (emscripten-core#21511) Also, avoid the use of `abort` in this case since abort itself depends on `WebAssembly.RuntimeError`. Fixes: emscripten-core#21484 * Fix unconditional usage of `requests` import (emscripten-core#21523) On systems that don't have the requests python we cannot catch this exception. * Use curl to download files on macOS (emscripten-core#21525) * Use DEPRECATED_SETTINGS for RUNTIME_LINKED_LIBS (emscripten-core#21522) And also for CLOSURE_WARNINGS --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Brendan Dahl <[email protected]> Co-authored-by: Sam Clegg <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Filip Lundgren <[email protected]> Co-authored-by: Kai Ninomiya <[email protected]> Co-authored-by: Alon Zakai <[email protected]> Co-authored-by: Kleis Auke Wolthuizen <[email protected]> Co-authored-by: Xdestiny <[email protected]> Co-authored-by: Jerry.Zhuang <[email protected]> Co-authored-by: Yan Pujante <[email protected]> Co-authored-by: Bruce Mitchener <[email protected]> Co-authored-by: MichalGrzymek <[email protected]> Co-authored-by: Simon Cooper <[email protected]> Co-authored-by: mrolig5267319 <[email protected]> Co-authored-by: kangtastic <[email protected]> Co-authored-by: Derek Schuff <[email protected]> Co-authored-by: Piotr Wierciński <[email protected]> Co-authored-by: Dannii Willis <[email protected]> Co-authored-by: pastdue <[email protected]> Co-authored-by: François Hautier <[email protected]> Co-authored-by: Marcin Wojdyr <[email protected]> Co-authored-by: Hood Chatham <[email protected]> Co-authored-by: Carl Woffenden <[email protected]> Co-authored-by: Charlie Birks <[email protected]> Co-authored-by: Érico Porto <[email protected]> Co-authored-by: Heejin Ahn <[email protected]> Co-authored-by: patrick kettner <[email protected]> Co-authored-by: SingleAccretion <[email protected]> Co-authored-by: J Nicholas Giampietro <[email protected]> Co-authored-by: Yan, Shaobo <[email protected]> Co-authored-by: nsfisis <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #21503