Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make run-test262 multi-threaded #547

Closed
bnoordhuis opened this issue Sep 24, 2024 · 2 comments · Fixed by #564
Closed

Make run-test262 multi-threaded #547

bnoordhuis opened this issue Sep 24, 2024 · 2 comments · Fixed by #564
Labels
enhancement New feature or request

Comments

@bnoordhuis
Copy link
Contributor

We run an ever larger slice of test262 and running time grows concomitantly. Using threads should give roughly an N-fold speedup.

Caveat: failing tests should be written in a reproducible manner to test262_errors.txt; for instance, by sorting them alphabetically.

@bnoordhuis bnoordhuis added the enhancement New feature or request label Sep 24, 2024
@chqrlie
Copy link
Collaborator

chqrlie commented Sep 24, 2024

Regarding the alphabetical order, the list of test files should already be sorted in a deterministic order:

/* find js files from the directory tree and sort the list */
static void enumerate_tests(const char *path)
{
    namelist_t *lp = &test_list;
    int start = lp->count;
    ftw(path, add_test_file, 100);
    qsort(lp->array + start, lp->count - start, sizeof(*lp->array),
              namelist_cmp_indirect);
}

@chqrlie
Copy link
Collaborator

chqrlie commented Sep 24, 2024

If we run multiple threads, the errors will be written to test262_errors.txt in a non deterministic order (I guess this is what you meant @bnoordhuis ) so the file should be re-sorted after all threads complete, which is straightforward if all errors are written as single lines. Storing these errors in one of more lists (eg: in the test list) and writing them at the end is a palatable approach. The file test262_report.txt does not need sorting and can be used in case run-test262 crashes.

bnoordhuis added a commit to bnoordhuis/quickjs that referenced this issue Sep 28, 2024
This commit introduces a couple of changes in order to make run-test262
go brr and execute tests in parallel:

- Remove CONFIG_AGENT build option. The disabled version of the build
  was already broken and no one noticed, Remove the define altogether.

- Remove the -C switch. Hard to support in multi-threaded mode.
  I may bring it back some day because it _is_ useful.

- Remove the -r switch. Also hard to support and I never look at
  test262_report.txt anyway so on the chopping block it goes.

- Judicious use of thread-local storage so I don't have to thread
  through state everywhere and embiggen the diff even more.

This is what Real Programmers(TM) do: stay up coding way past midnight
just so the test suite finishes in one minute instead of four.

Fixes: quickjs-ng#547
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants