Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor plumbing for test goal to support batched tests. #17044

Closed

Commits on Sep 28, 2022

  1. Refactor plumbing for test goal to support batched tests.

    The new structure is heavily inspired by recent work to refactor the way
    `lint` and `fmt` batch inputs. I don't expect any behavior change from
    this diff - all plugins providing a `test` implementation are coded to
    return single-element partitions. Once the plumbing has merged to `main`
    I plan to follow-up with an implementation of `pytest` batching.
    
    Known TODOs:
      - Restore the warning/error raised if no applicable targets are found
        for the input specs
      - Restore use of environments in this flow
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    bb641b8 View commit details
    Browse the repository at this point in the history
  2. Restore some tests.

    I deleted a handful of tests when I was getting started because I
    thought they would no longer be relevant. Turns out they continue to
    work with a few tweaks, so we can restore them to avoid losing coverage.
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    779089a View commit details
    Browse the repository at this point in the history
  3. Fix some more existing tests.

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    d86caa5 View commit details
    Browse the repository at this point in the history
  4. Remove tester_name field from TestResult.

    Tracking the name of the test tool didn't really give us much - delete
    it, and simplify the code that used to use it.
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    3f9e68d View commit details
    Browse the repository at this point in the history
  5. Restore sorting of results.

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    b8344d0 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2022

  1. Delete batch_size option from [test] subsystem.

    Hard-code a value of 1 for now where it was being used.
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    9332ec4 View commit details
    Browse the repository at this point in the history
  2. Restore environment-awareness to test goal.

    Look up the environment name for each element in a test partition, and
    assert that everything in a partition matches.
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    8e08a21 View commit details
    Browse the repository at this point in the history
  3. Restore logic to warn/error if test is run on a non-applicable target.

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    6562137 View commit details
    Browse the repository at this point in the history
  4. Add field for prefix of extra test output.

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    68eb776 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2022

  1. Configuration menu
    Copy the full SHA
    ef75935 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2032440 View commit details
    Browse the repository at this point in the history
  3. raise AssertionErrors instead of asserting.

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    85afb9b View commit details
    Browse the repository at this point in the history
  4. Clean up comments and unused property

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    a4d1d77 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'main' into danxmoran/batched-test-plumbing

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    7d7833a View commit details
    Browse the repository at this point in the history
  6. Rename description -> partition_description

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    479fe6a View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2022

  1. Merge branch 'main' into danxmoran/batched-test-plumbing

    [ci skip-build-wheels]
    danxmoran committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    2718f49 View commit details
    Browse the repository at this point in the history
  2. Rename elements to field_sets in TestRequest.SubPartition

    [ci skip-build-wheels]
    danxmoran committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    98b5711 View commit details
    Browse the repository at this point in the history
  3. Use Any-typed partition keys.

    Add a separate field for `extra_output_prefix` for clarity
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    91b8b06 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'main' into danxmoran/batched-test-plumbing

    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    fb1fc15 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2022

  1. Stop passing around extra_output_prefix.

    This is currently only used by `pytest`, and we have a couple other
    fields with the same problem (de-duplicating between test runs) that we
    currently expect plugin implementers to handle. Incidentally, deleting
    the field flushed out some invalid `SubPartition`s built in tests that
    snuck through the previous cleanup commit.
    
    [ci skip-rust]
    
    [ci skip-build-wheels]
    danxmoran committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    7c6da21 View commit details
    Browse the repository at this point in the history