Don't set shared.Settings from test runners

[jgravelle-google]

Instead of shared.Settings.FOO = x, now we say self.set_settting('FOO', x)
This avoids needing to reload the settings for each test, because we can store the modifications in self.settings_mods, and just clear that dict between runs.

Instead of if shared.Settings.BAR > 2, now self.get_setting('BAR') > 2
This is needed because we need to respect the default settings, as well as any modifications made for test purposes.

Instead of shared.Settings.serialize(), now self.serialize_settings()
This is the real win of this patch: we only need to serialize the setting changes that the test actually cares about. This cleans up command-line errors tremendously.
Before:

CalledProcessError: Command '['/usr/bin/python2', '/emscripten/emcc.py', '-Wno-dynamic-class-memaccess', '-Wno-format', '-Wno-format-extra-args', '-Wno-format-security', '-Wno-pointer-bool-conversion', '-Wno-unused-volatile-lvalue', '-Wno-c++11-compat-deprecated-writable-strings', '-Wno-invalid-pp-token', '-O2', '-s', 'GL_DEBUG=0', '-s', 'EXCEPTION_CATCHING_WHITELIST=[]', '-s', 'INCLUDE_FULL_LIBRARY=0', '-s', 'ASMFS=0', '-s', 'TEXTDECODER=1', '-s', 'FINALIZE_ASM_JS=1', '-s', 'BINARYEN_ROOT=""', '-s', 'SPLIT_MEMORY=0', '-s', 'WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=0', '-s', 'LIBRARY_DEBUG=0', '-s', 'USE_CLOSURE_COMPILER=0', '-s', 'USE_SDL_IMAGE=1', '-s', 'SDL2_IMAGE_FORMATS=[]', '-s', 'INLINING_LIMIT=0', '-s', 'MODULARIZE=0', '-s', 'NO_FILESYSTEM=0', '-s', 'SYSTEM_JS_LIBRARIES=[]', '-s', 'FETCH_DEBUG=0', '-s', 'BINARYEN_TRAP_MODE="allow"', '-s', 'ERROR_ON_MISSING_LIBRARIES=0', '-s', 'SOCKET_WEBRTC=0', '-s', 'EVAL_CTORS=0', '-s', 'ELIMINATE_DUPLICATE_FUNCTIONS_DUMP_EQUIVALENT_FUNCTIONS=0', '-s', 'USE_PTHREADS=0', '-s', 'ENVIRONMENT_MAY_BE_WEB_OR_WORKER=1', '-s', 'USE_OGG=0', '-s', 'LEGACY_VM_SUPPORT=0', '-s', 'EXPORT_NAME="Module"', '-s', 'AGGRESSIVE_VARIABLE_ELIMINATION=0', '-s', 'SIMPLIFY_IFS=1', '-s', 'ALIASING_FUNCTION_POINTERS=1', '-s', 'EMULATED_FUNCTION_POINTERS=0', '-s', 'EMSCRIPTEN_VERSION=""', '-s', 'OPENAL_DEBUG=0', '-s', 'NO_EXIT_RUNTIME=1', '-s', 'RELOCATABLE=0', '-s', 'ENVIRONMENT=""', '-s', 'PRECISE_F32=0', '-s', 'DEBUG_TAGS_SHOWING=[]', '-s', 'USE_VORBIS=0', '-s', 'SAFE_HEAP_LOG=0', '-s', 'PROXY_TO_WORKER_FILENAME=""', '-s', 'LIBRARY_DEPS_TO_AUTOEXPORT=["memcpy"]', '-s', 'ASSERTIONS=0', '-s', 'QUANTUM_SIZE=4', '-s', 'PRECISE_I64_MATH=1', '-s', 'SEPARATE_ASM=0', '-s', 'GLOBAL_BASE=-1', '-s', 'STACK_OVERFLOW_CHECK=0', '-s', 'FORCE_FILESYSTEM=0', '-s', 'LZ4=0', '-s', 'ASM_JS=1', '-s', 'WARN_UNALIGNED=0', '-s', 'WASM_BINARY_FILE=""', '-s', 'ASMJS_CODE_FILE=""', '-s', 'EMULATE_FUNCTION_POINTER_CASTS=0', '-s', 'BINARYEN_PASSES=""', '-s', 'BINARYEN_SCRIPTS=""', '-s', 'TOTAL_STACK=5242880', '-s', 'TRACE_WEBGL_CALLS=0', '-s', 'BINARYEN_ASYNC_COMPILATION=1', '-s', 'EMTERPRETIFY_SYNCLIST=[]', '-s', 'RESERVED_FUNCTION_POINTERS=0', '-s', 'EXCEPTION_DEBUG=0', '-s', 'GL_PREINITIALIZED_CONTEXT=0', '-s', 'SWAPPABLE_ASM_MODULE=0', '-s', 'SHELL_FILE=0', '-s', 'GL_ASSERTIONS=0', '-s', 'BOOTSTRAPPING_STRUCT_INFO=0', '-s', 'MEM_INIT_METHOD=0', '-s', 'GL_TESTING=0', '-s', 'DETERMINISTIC=0', '-s', 'SAFE_HEAP=0', '-s', 'DEAD_FUNCTIONS=[]', '-s', 'USE_LIBPNG=0', '-s', 'ENVIRONMENT_MAY_BE_SHELL=1', '-s', 'GL_FFP_ONLY=0', '-s', 'EMSCRIPTEN_TRACING=0', '-s', 'OPT_LEVEL=0', '-s', 'MEMFS_APPEND_TO_TYPED_ARRAYS=0', '-s', 'NO_DYNAMIC_EXECUTION=0', '-s', 'BUILD_AS_WORKER=0', '-s', 'EXPORT_ALL=0', '-s', 'EMTERPRETIFY_ADVISE=0', '-s', 'VERBOSE=0', '-s', 'NODERAWFS=0', '-s', 'DISABLE_EXCEPTION_CATCHING=1', '-s', 'PTHREADS_PROFILING=0', '-s', 'EMTERPRETIFY_WHITELIST=[]', '-s', 'SKIP_STACK_IN_SMALL=1', '-s', 'STRICT=0', '-s', 'PROFILING_FUNCS=0', '-s', 'SOURCE_MAP_BASE=""', '-s', 'EMTERPRETIFY_ASYNC=0', '-s', 'FULL_ES2=0', '-s', 'FULL_ES3=0', '-s', 'OFFSCREENCANVAS_SUPPORT=0', '-s', 'USE_WEBGL2=0', '-s', 'PTHREADS_DEBUG=0', '-s', 'SIMD=0', '-s', 'PROXY_TO_WORKER=0', '-s', 'UNALIGNED_MEMORY=0', '-s', 'PROXY_TO_PTHREAD=0', '-s', 'FORCE_ALIGNED_MEMORY=0', '-s', 'EXTRA_EXPORTED_RUNTIME_METHODS=[]', '-s', 'FETCH=0', '-s', 'ASYNCIFY=0', '-s', 'TOTAL_MEMORY=16777216', '-s', 'USE_COCOS2D=0', '-s', 'USE_SDL_NET=1', '-s', 'SOCKET_DEBUG=0', '-s', 'ELIMINATE_DUPLICATE_FUNCTIONS_PASSES=5', '-s', 'EXPLICIT_ZEXT=0', '-s', 'STACK_START=-1', '-s', 'HEADLESS=0', '-s', 'USE_SDL=1', '-s', 'CASE_INSENSITIVE_FS=0', '-s', 'MAIN_MODULE=0', '-s', 'ASYNCIFY_WHITELIST=["qsort", "trinkle", "__toread", "__uflow", "__fwritex", "MUSL_vfprintf"]', '-s', 'MALLOC="dlmalloc"', '-s', 'SAFE_SPLIT_MEMORY=0', '-s', 'EXPORT_ES6=0', '-s', 'WEBSOCKET_URL="ws:#"', '-s', 'EMTERPRETIFY=0', '-s', 'DEBUG_LEVEL=0', '-s', 'SMALL_XHR_CHUNKS=0', '-s', 'USE_GLFW=2', '-s', 'WARN_ON_UNDEFINED_SYMBOLS=1', '-s', 'CYBERDWARF=0', '-s', 'ORIGINAL_EXPORTED_FUNCTIONS=[]', '-s', 'ONLY_MY_CODE=0', '-s', 'DEMANGLE_SUPPORT=0', '-s', 'RUNNING_JS_OPTS=0', '-s', 'WEBSOCKET_SUBPROTOCOL="binary"', '-s', 'INVOKE_RUN=1', '-s', 'PTHREAD_HINT_NUM_CORES=4', '-s', 'ENVIRONMENT_MAY_BE_WORKER=1', '-s', 'MODULARIZE_INSTANCE=0', '-s', 'PTHREAD_POOL_SIZE=0', '-s', 'SUPPORT_BASE64_EMBEDDING=0', '-s', 'DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=["memcpy", "memset", "malloc", "free"]', '-s', 'WASM=1', '-s', 'ENVIRONMENT_MAY_BE_WEB=1', '-s', 'EXPORT_FUNCTION_TABLES=0', '-s', 'SIDE_MODULE=0', '-s', 'USE_SDL_TTF=1', '-s', 'SINGLE_FILE=0', '-s', 'GL_MAX_TEMP_BUFFER_SIZE=2097152', '-s', 'EXPORT_BINDINGS=0', '-s', 'FS_LOG=0', '-s', 'STB_IMAGE=0', '-s', 'IGNORE_CLOSURE_COMPILER_ERRORS=0', '-s', 'RUNTIME_LOGGING=0', '-s', 'USE_HARFBUZZ=0', '-s', 'ELIMINATE_DUPLICATE_FUNCTIONS=0', '-s', 'ABORTING_MALLOC=1', '-s', 'SYSCALL_DEBUG=0', '-s', 'DEFAULT_PTHREAD_STACK_SIZE=2097152', '-s', 'PGO=0', '-s', 'IN_TEST_HARNESS=0', '-s', 'BUILD_AS_SHARED_LIB=0', '-s', 'EMTERPRETIFY_BLACKLIST=[]', '-s', 'LEGALIZE_JS_FFI=1', '-s', 'USE_FREETYPE=0', '-s', 'DOUBLE_MODE=1', '-s', 'BUNDLED_CD_DEBUG_FILE=""', '-s', 'MAX_GLOBAL_ALIGN=-1', '-s', 'EMTERPRETIFY_FILE=""', '-s', 'IMPLEMENTED_FUNCTIONS=[]', '-s', 'LEGACY_GL_EMULATION=0', '-s', 'STRUCT_INFO=""', '-s', 'RETAIN_COMPILER_SETTINGS=0', '-s', 'ENVIRONMENT_MAY_BE_NODE=1', '-s', 'EXPORTED_FUNCTIONS=["_main"]', '-s', 'RUNTIME_LINKED_LIBS=[]', '-s', 'EXPORTED_RUNTIME_METHODS=[]', '-s', 'MEM_INIT_IN_WASM=0', '-s', 'ALLOW_MEMORY_GROWTH=0', '-s', 'USE_BULLET=0', '-s', 'NODEJS_CATCH_EXIT=1', '-s', 'USE_ZLIB=0', '-s', 'WASM_TEXT_FILE=""', '-s', 'GL_UNSAFE_OPTS=1', '-s', 'BENCHMARK=0', '-s', 'WASM_MEM_MAX=-1', '-s', 'BINARYEN_METHOD="native-wasm"', '-s', 'LINKABLE=0', '-s', 'WASM_BACKEND=1', '-s', 'ASYNCIFY_FUNCTIONS=["emscripten_sleep", "emscripten_wget", "emscripten_yield"]', '-s', 'ERROR_ON_UNDEFINED_SYMBOLS=0', '-s', 'BINARYEN_IGNORE_IMPLICIT_TRAPS=0', '-s', 'OUTLINING_LIMIT=0', '-O2', '-s', 'EXPORTED_FUNCTIONS=@exps', '-I', '/tmp/emscripten_test_binaryen2_kih1hf', '-I', '/tmp/emscripten_test_binaryen2_kih1hf/include', '/tmp/emscripten_test_binaryen2_kih1hf/src.cpp', '-o', '/tmp/emscripten_test_binaryen2_kih1hf/src.cpp.o.js']' returned non-zero exit status 1

After:

CalledProcessError: Command '['/usr/bin/python2', '/usr/local/google/home/jgravelle/code/wasm/waterfall/src/work/emscripten/emcc.py', '-Wno-dynamic-class-memaccess', '-Wno-format', '-Wno-format-extra-args', '-Wno-format-security', '-Wno-pointer-bool-conversion', '-Wno-unused-volatile-lvalue', '-Wno-c++11-compat-deprecated-writable-strings', '-Wno-invalid-pp-token', '-O2', '-O2', '-s', 'EXPORTED_FUNCTIONS=@exps', '-I', '/tmp/emscripten_test_binaryen2_NtIDO1', '-I', '/tmp/emscripten_test_binaryen2_NtIDO1/include', '/tmp/emscripten_test_binaryen2_NtIDO1/src.cpp', '-o', '/tmp/emscripten_test_binaryen2_NtIDO1/src.cpp.o.js']' returned non-zero exit status 1

[sbc100]

I ❤️ this change so much.

It also makes me wonder why some cases use Building.emcc(..) and others just run_process([PYTHON, EMCC...). Maybe in a followup we can remove the latter to at least make it a method on the test baseclass rather than in shared.

[kripken]

I don't feel strongly either way, but another option here might be to just do the diff when serializing - that is, don't emit -s X=Y if the value of X is Y anyhow in the default settings. That should achieve the same results here, with less work. But since the work here is already done, up to you.

[jgravelle-google]

That was my first approach that I tried a few weeks back. I kept running into snags for reasons I don't fully remember (at least one of which probably stopped being true in #6793 now that I think about it). This solution I feel leaves the code simpler (despite being a larger diff), and also has some other nice properties.
The work here was mostly verifying that the regex worked, and moving comments out of the way of the find+replace.

[jgravelle-google]

Fixed CI, going to merge in a hour or so if there's no comments

[kripken]

@jgravelle-google this appears to have broken asm2f.test_simd15 (this commit and all after it fail on CI).

[kripken]

Confirmed locally as well.

[jgravelle-google]

Also confirmed locally.
Poking at this for a while, forcibly passing -s INLINING_LIMIT=0 makes this work again.

[jgravelle-google]

Fixed in #6848