Bug 1609358 [wpt PR 21185] - [WPT/common/security-features] Allow `ex…

…cluded_tests` based on `expectation` value, a=testonly

Automatic update from web-platform-tests
[WPT/common/security-features] Allow `excluded_tests` based on `expectation` value

Previously: `excluded_tests` excluded selections
that have the same values for the fields included in `selection_pattern`.

However, as `selection_pattern` doesn't contain `expectation`,
`expectation` can't be used in `excluded_tests`, blocking [1].

After this CL: `excluded_tests` excluded selections
that have the same values for all the fields (except for `name`).

For this purpose, this CL:

- Sets `delivery_key` for `excluded_selection`, so that
  selections and excluded selections have the same set of fields.
- Uses `dump_test_parameters()` rather than `selection_pattern`
  for exclusion matching (`exclusion_dict`).
- Applies exclusions after all overrides are processed.
  This is a fix for cases like:
  - "default" selection A
  - "override" selection B, overriding A
  - exclusion pattern E, matching B but not A
  Previously: E excludes B but not A, so A is generated.
  After this CL: B overrides A, then E excludes B.
    As A is already overridden, no selections are generated.

This CL doesn't change generated results, because before [1]
there are no cases hitting these issues.

[1] https://chromium-review.googlesource.com/c/chromium/src/+/1991066

Bug: 906850
Change-Id: Ifa36167df710edd89d4e572346a3f1b928710119
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1991074
Commit-Queue: Hiroshige Hayashizaki <[email protected]>
Reviewed-by: Mike West <[email protected]>
Cr-Commit-Position: refs/heads/master@{#731904}

--

wpt-commits: 61616b5bfd5e7c0e86e68b6e4fbb0192ba78d170
wpt-pr: 21185

testing/web-platform/tests/common/security-features/tools/generate.py

@@ -261,9 +261,8 @@ def generate_test_source_files(spec_directory, spec_json, target):
             expand_pattern(excluded_pattern, test_expansion_schema)
         for excluded_selection in permute_expansion(excluded_expansion,
                                                     artifact_order):
-            excluded_selection_path = spec_json[
-                'selection_pattern'] % excluded_selection
-            exclusion_dict[excluded_selection_path] = True
+            excluded_selection['delivery_key'] = spec_json['delivery_key']
+            exclusion_dict[dump_test_parameters(excluded_selection)] = True
 
     for spec in specification:
         # Used to make entries with expansion="override" override preceding
@@ -276,20 +275,20 @@ def generate_test_source_files(spec_directory, spec_json, target):
             for selection in permute_expansion(expansion, artifact_order):
                 selection['delivery_key'] = spec_json['delivery_key']
                 selection_path = spec_json['selection_pattern'] % selection
-                if not selection_path in exclusion_dict:
-                    if selection_path in output_dict:
-                        if expansion_pattern['expansion'] != 'override':
-                            print(
-                                "Error: %s's expansion is default but overrides %s"
-                                % (selection['name'],
-                                   output_dict[selection_path]['name']))
-                            sys.exit(1)
-                    output_dict[selection_path] = copy.deepcopy(selection)
-                else:
-                    print('Excluding selection:', selection_path)
+                if selection_path in output_dict:
+                    if expansion_pattern['expansion'] != 'override':
+                        print(
+                            "Error: %s's expansion is default but overrides %s"
+                            % (selection['name'],
+                               output_dict[selection_path]['name']))
+                        sys.exit(1)
+                output_dict[selection_path] = copy.deepcopy(selection)
 
         for selection_path in output_dict:
             selection = output_dict[selection_path]
+            if dump_test_parameters(selection) in exclusion_dict:
+                print('Excluding selection:', selection_path)
+                continue
             try:
                 generate_selection(spec_directory, spec_json, selection, spec,
                                    html_template)