fix(#7623): Resize ConductorAxis properly (#7624)

* fix: resize conductor properly * refactor: more computed properties, unregister listener * fix: beforeUnmounted hook * test(visual): add time conductor visual test for fixed mode * fix: initialize to `null` * feat: extend the base `screenshot` function to mask elements which will always create variance in an Open MCT screenshot * docs: add types for fixtures * fix: remove unneeded await * chore: add sinon timers types package back * docs: remove unused docs * doc: remove unused docs * test: add visual realtime url, update imports * feat: provide wrapped page.screenshot fixture that applies defaults * test: add basic timeConductor snapshot tests * chore: update eslint config * lint: remove unused disable directives * test: remove redundant navigation * fix: remove listeners * fix: maybe stabilize unit tests * docs: remove * fix: provide sourcemaps in unit tests * test: add regression snapshot test for time conductor axis * lint: remove unused imports * feat(e2e): add fixture to manually tick the clock and use it * test: reactivate test now that we don't use deploysentinel :( * test: update snapshots * test: add test for clockOptions and tick fixtures * test: add afterEach stub and fixme * test: try and stabilize fault management flake * lint: defy the word gods * chore: ignore `*-darwin.png` screenshots * chore: remove darwin screenshot binaries * docs: markdownlint * docs: remove MacOS specific instructions from snapshot testing * fix: remove a11y
nasa · Mar 26, 2024 · 5b4ee19 · 5b4ee19
1 parent 7e926cc
commit 5b4ee19
Show file tree

Hide file tree

Showing 54 changed files with 671 additions and 393 deletions.
diff --git a/.eslintrc.cjs b/.eslintrc.cjs
@@ -1,10 +1,13 @@
 const LEGACY_FILES = ['example/**'];
-module.exports = {
+/** @type {import('eslint').Linter.Config} */
+const config = {
   env: {
     browser: true,
-    es6: true,
+    es2024: true,
     jasmine: true,
-    amd: true
+    node: true,
+    worker: true,
+    serviceworker: true
   },
   globals: {
     _: 'readonly'
@@ -23,10 +26,11 @@ module.exports = {
     parser: '@babel/eslint-parser',
     requireConfigFile: false,
     allowImportExportEverywhere: true,
-    ecmaVersion: 2015,
+    ecmaVersion: 'latest',
     ecmaFeatures: {
       impliedStrict: true
-    }
+    },
+    sourceType: 'module'
   },
   rules: {
     'simple-import-sort/imports': 'warn',
@@ -152,7 +156,7 @@ module.exports = {
         cases: {
           pascalCase: true
         },
-        ignore: ['^.*\\.js$']
+        ignore: ['^.*\\.(js|cjs|mjs)$']
       }
     ],
     'vue/first-attribute-linebreak': 'error',
@@ -179,3 +183,5 @@ module.exports = {
     }
   ]
 };
+
+module.exports = config;
diff --git a/.gitignore b/.gitignore
@@ -47,3 +47,6 @@ index.html.bak
 .nyc_output
 coverage
 codecov
+
+# Don't commit MacOS screenshots
+*-darwin.png
diff --git a/e2e/README.md b/e2e/README.md
@@ -76,28 +76,30 @@ To read about how to write a good visual test, please see [How to write a great
 
 `npm run test:e2e:visual` commands will run all of the visual tests against a local instance of Open MCT. If no `PERCY_TOKEN` API key is found in the terminal or command line environment variables, no visual comparisons will be made.
 
- - `npm run test:e2e:visual:ci` will run against every commit and PR.
- - `npm run test:e2e:visual:full` will run every night with additional comparisons made for Larger Displays and with the `snow` theme.
+- `npm run test:e2e:visual:ci` will run against every commit and PR.
+- `npm run test:e2e:visual:full` will run every night with additional comparisons made for Larger Displays and with the `snow` theme.
+
 #### Percy.io
 
 To make this possible, we're leveraging a 3rd party service, [Percy](https://percy.io/). This service maintains a copy of all changes, users, scm-metadata, and baselines to verify that the application looks and feels the same _unless approved by a Open MCT developer_. To request a Percy API token, please reach out to the Open MCT Dev team on GitHub. For more information, please see the official [Percy documentation](https://docs.percy.io/docs/visual-testing-basics).
 
-At present, we are using percy with two configuration files: `./e2e/.percy.nightly.yml` and `./e2e/.percy.ci.yml`. This is mainly to reduce the number of snapshots. 
+At present, we are using percy with two configuration files: `./e2e/.percy.nightly.yml` and `./e2e/.percy.ci.yml`. This is mainly to reduce the number of snapshots.
 
 ### Advanced: Snapshot Testing (Not Recommended)
 
 While snapshot testing offers a precise way to detect changes in your application without relying on third-party services like Percy.io, we've found that it doesn't offer any advantages over visual testing in our use-cases. Therefore, snapshot testing is **not recommended** for further implementation.
 
 #### CI vs Manual Checks
+
 Snapshot tests can be reliably executed in Continuous Integration (CI) environments but lack the manual oversight provided by visual testing platforms like Percy.io. This means they may miss issues that a human reviewer could catch during manual checks.
 
 #### Example
-A single visual test assertion in Percy.io can be executed across 10 different browser and resolution combinations without additional setup, providing comprehensive testing with minimal configuration. In contrast, a snapshot test is restricted to a single OS and browser resolution, requiring more effort to achieve the same level of coverage.
 
+A single visual test assertion in Percy.io can be executed across 10 different browser and resolution combinations without additional setup, providing comprehensive testing with minimal configuration. In contrast, a snapshot test is restricted to a single OS and browser resolution, requiring more effort to achieve the same level of coverage.
 
 #### Further Reading
-For those interested in the mechanics of snapshot testing with Playwright, you can refer to the [Playwright Snapshots Documentation](https://playwright.dev/docs/test-snapshots). However, keep in mind that we do not recommend using this approach.
 
+For those interested in the mechanics of snapshot testing with Playwright, you can refer to the [Playwright Snapshots Documentation](https://playwright.dev/docs/test-snapshots). However, keep in mind that we do not recommend using this approach.
 
 #### Open MCT's implementation
 
@@ -118,14 +120,6 @@ When the `@snapshot` tests fail, they will need to be evaluated to determine if
 
 To compare a snapshot, run a test and open the html report with the 'Expected' vs 'Actual' screenshot. If the actual screenshot is preferred, then the source-controlled 'Expected' snapshots will need to be updated with the following scripts.
 
-MacOS
-
-```
-npm run test:e2e:updatesnapshots
-```
-
-Linux/CI
-
 ```sh
 // Replace {X.X.X} with the current Playwright version 
 // from our package.json or circleCI configuration file
@@ -335,9 +329,11 @@ We have a Mission-need to support iPad and mobile devices. To run our test suite
 In general, our test suite is not designed to run against mobile devices as the mobile experience is a focused version of the application. Core functionality is missing (chiefly the 'Create' button). To bypass the object creation, we leverage the `storageState` properties for starting the mobile tests with localstorage.
 
 For now, the mobile tests will exist in the /tests/mobile/ suites and be executed with the
+
 ```sh
 npm run test:e2e:mobile
 ```
+
 command.
 
 #### **Skipping or executing tests based on browser, os, and/os browser version:**
@@ -377,13 +373,15 @@ In general, strive to test only through the UI as a user would. As stated in the
 By adhering to this principle, we can create tests that are both robust and reflective of actual user experiences.
 
 #### How to make tests robust to function in other contexts (VISTA, COUCHDB, YAMCS, VIPER, etc.)
+
   1. Leverage the use of `appActions.js` methods such as `createDomainObjectWithDefaults()`. This ensures that your tests will create unique instances of objects for your test to interact with.
   1. Do not assert on the order or structure of objects available unless you created them yourself. These tests may be used against a persistent datastore like couchdb with many objects in the tree.
   1. Do not search for your created objects. Open MCT does not performance uniqueness checks so it's possible that your tests will break when run twice.
   1. Avoid creating locator aliases. This likely means that you're compensating for a bad locator. Improve the application instead.
   1. Leverage `await page.goto('./', { waitUntil: 'domcontentloaded' });` instead of `{ waitUntil: 'networkidle' }`. Tests run against deployments with websockets often have issues with the networkidle detection.
   
 #### How to make tests faster and more resilient
+
   1. Avoid app interaction when possible. The best way of doing this is to navigate directly by URL:
 
   ```js
@@ -396,10 +394,11 @@ By adhering to this principle, we can create tests that are both robust and refl
 
   1. Leverage `await page.goto('./', { waitUntil: 'domcontentloaded' });`
     - Initial navigation should _almost_ always use the `{ waitUntil: 'domcontentloaded' }` option.
-  1.  Avoid repeated setup to test a single assertion. Write longer tests with multiple soft assertions.
+  1. Avoid repeated setup to test a single assertion. Write longer tests with multiple soft assertions.
   This ensures that your changes will be picked up with large refactors.
 
 ##### Utilizing LocalStorage
+
   1. In order to save test runtime in the case of tests that require a decent amount of initial setup (such as in the case of testing complex displays), you may use [Playwright's `storageState` feature](https://playwright.dev/docs/api/class-browsercontext#browser-context-storage-state) to generate and load localStorage states.
   1. To generate a localStorage state to be used in a test:
     - Add an e2e test to our generateLocalStorageData suite which sets the initial state (creating/configuring objects, etc.), saving it in the `test-data` folder:
@@ -420,7 +419,6 @@ By adhering to this principle, we can create tests that are both robust and refl
       }); 
     ```
 
-
 ### How to write a great test
 
 - Avoid using css locators to find elements to the page. Use modern web accessible locators like `getByRole`
@@ -436,7 +434,7 @@ By adhering to this principle, we can create tests that are both robust and refl
   await notesInput.fill(testNotes);
   ```
 
- #### How to Write a Great Visual Test
+#### How to Write a Great Visual Test
 
 1. **Look for the Unknown Unknowns**: Avoid asserting on specific differences in the visual diff. Visual tests are most effective for identifying unknown unknowns.
 
@@ -445,23 +443,27 @@ By adhering to this principle, we can create tests that are both robust and refl
 3. **Expect the Unexpected**: Use functional expect statements only to verify assumptions about the state between steps. A great visual test doesn't fail during the test itself, but rather when changes are reviewed in Percy.io.
 
 4. **Control Variability**: Account for variations inherent in working with time-based telemetry and clocks.
-  - Utilize `percyCSS` to ignore time-based elements. For more details, consult our [percyCSS file](./.percy.ci.yml).
-  - Use Open MCT's fixed-time mode unless explicitly testing realtime clock
-  - Employ the `createExampleTelemetryObject` appAction to source telemetry and specify a `name` to avoid autogenerated names.
-  - Avoid creating objects with a time component like timers and clocks.
+
+- Utilize `percyCSS` to ignore time-based elements. For more details, consult our [percyCSS file](./.percy.ci.yml).
+- Use Open MCT's fixed-time mode unless explicitly testing realtime clock
+- Employ the `createExampleTelemetryObject` appAction to source telemetry and specify a `name` to avoid autogenerated names.
+- Avoid creating objects with a time component like timers and clocks.
 
 5. **Hide the Tree and Inspector**: Generally, your test will not require comparisons involving the tree and inspector. These aspects are covered in component-specific tests (explained below). To exclude them from the comparison by default, navigate to the root of the main view with the tree and inspector hidden:
     - `await page.goto('./#/browse/mine?hideTree=true&hideInspector=true')`
 
 6. **Component-Specific Tests**: If you wish to focus on a particular component, use the `/visual-a11y/component/` folder and limit the scope of the comparison to that component. For instance:
+
     ```js
     await percySnapshot(page, `Tree Pane w/ single level expanded (theme: ${theme})`, {
         scope: treePane
     });
     ```
+
     - Note: The `scope` variable can be any valid CSS selector.
 
 7. **Write many `percySnapshot` commands in a single test**: In line with our approach to longer functional tests, we recommend that many test percySnapshots are taken in a single test. For instance:
+
     ```js
     //<Some interesting state>
     await percySnapshot(page, `Before object expanded (theme: ${theme})`);
@@ -511,6 +513,7 @@ test.describe('foo test suite', () => {
   });
 });
   ```
+
   More info and options for `overrideClock` can be found in [baseFixtures.js](baseFixtures.js)
 
 - Working with multiple pages
@@ -539,7 +542,6 @@ const key = getFirstKeyFromOpenMctJson(jsonData);
 expect(jsonData.openmct[key]).toHaveProperty('name', 'e2e folder');
 ```
 
-
 ### Reporting
 
 Test Reporting is done through official Playwright reporters and the CI Systems which execute them.
@@ -615,6 +617,7 @@ A single e2e test in Open MCT is extended to run:
 ### Writing Tests
 
 Playwright provides 3 supported methods of debugging and authoring tests:
+
 - A 'watch mode' for running tests locally and debugging on the fly
 - A 'debug mode' for debugging tests and writing assertions against tests
 - A 'VSCode plugin' for debugging tests within the VSCode IDE.

diff --git a/e2e/avpFixtures.js b/e2e/avpFixtures.js
@@ -36,27 +36,67 @@
 import AxeBuilder from '@axe-core/playwright';
 import fs from 'fs';
 import path from 'path';
+import { fileURLToPath } from 'url';
 
 import { expect, test } from './pluginFixtures.js';
-
 // Constants for repeated values
-const TEST_RESULTS_DIR = './test-results';
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const TEST_RESULTS_DIR = path.join(__dirname, './test-results');
+
+const extendedTest = test.extend({
+  /**
+   * Overrides the default screenshot function to apply default options that should apply to all
+   * screenshots taken in the AVP tests.
+   *
+   * @param {import('@playwright/test').PlaywrightTestArgs} args - The Playwright test arguments.
+   * @param {Function} use - The function to use the page object.
+   * Defaults:
+   * - Disables animations
+   * - Masks the clock indicator
+   * - Masks the time conductor last update time in realtime mode
+   * - Masks the time conductor start bounds in fixed mode
+   * - Masks the time conductor end bounds in fixed mode
+   */
+  page: async ({ page }, use) => {
+    const playwrightScreenshot = page.screenshot;
+
+    /**
+     * Override the screenshot function to always mask a given set of locators which will always
+     * show variance across screenshots. Defaults may be overridden by passing in options to the
+     * screenshot function.
+     * @param {import('@playwright/test').PageScreenshotOptions} options - The options for the screenshot.
+     * @returns {Promise<Buffer>} Returns the screenshot as a buffer.
+     */
+    page.screenshot = async function (options = {}) {
+      const mask = [
+        this.getByLabel('Clock Indicator'), // Mask the clock indicator
+        this.getByLabel('Last update'), // Mask the time conductor last update time in realtime mode
+        this.getByLabel('Start bounds'), // Mask the time conductor start bounds in fixed mode
+        this.getByLabel('End bounds') // Mask the time conductor end bounds in fixed mode
+      ];
+
+      const result = await playwrightScreenshot.call(this, {
+        animations: 'disabled',
+        mask,
+        ...options // Pass through or override any options
+      });
+      return result;
+    };
+
+    await use(page);
+  }
+});
 
 /**
  * Scans for accessibility violations on a page and writes a report to disk if violations are found.
  * Automatically asserts that no violations should be present.
  *
- * @typedef {Object} GenerateReportOptions
- * @property {string} [reportName] - The name for the report file.
- *
  * @param {import('playwright').Page} page - The page object from Playwright.
  * @param {string} testCaseName - The name of the test case.
- * @param {GenerateReportOptions} [options={}] - The options for the report generation.
- *
- * @returns {Promise<object|null>} Returns the accessibility scan results if violations are found,
- *                                  otherwise returns null.
+ * @param {{ reportName?: string }} [options={}] - The options for the report generation.
+ * @returns {Promise<Object|null>} Returns the accessibility scan results if violations are found, otherwise returns null.
  */
-/* eslint-disable no-undef */
+
 export async function scanForA11yViolations(page, testCaseName, options = {}) {
   const builder = new AxeBuilder({ page });
   builder.withTags(['wcag2aa']);
@@ -93,4 +133,4 @@ export async function scanForA11yViolations(page, testCaseName, options = {}) {
   }
 }
 
-export { expect, test };
+export { expect, extendedTest as test };
diff --git a/e2e/baseFixtures.js b/e2e/baseFixtures.js
@@ -1,4 +1,3 @@
-/* eslint-disable no-undef */
 /*****************************************************************************
  * Open MCT, Copyright (c) 2014-2024, United States Government
  * as represented by the Administrator of the National Aeronautics and Space
@@ -111,6 +110,40 @@ const extendedTest = test.extend({
       scope: 'test'
     }
   ],
+  /**
+   * Exposes a function to manually tick the clock. This is useful when overriding the clock to not
+   * tick (`shouldAdvanceTime: false`) for visual tests, as events such as re-renders and router params
+   * updates are clock-driven and must be manually ticked.
+   *
+   * Usage:
+   * ```js
+   * test.describe('Manual Clock Tick', () => {
+   *  test.use({
+   *   clockOptions: {
+   *     now: MISSION_TIME, // Set to the desired time
+   *     shouldAdvanceTime: false // Clock overridden to no longer tick
+   *   }
+   *  });
+   *  test('Visual - Manual Clock Tick', async ({ page, tick }) => {
+   *   // Tick the clock 2 seconds in the future
+   *   await tick(2000);
+   *  });
+   * });
+   * ```
+   *
+   * @param {Object} param0
+   * @param {import('@playwright/test').Page} param0.page
+   * @param {import('@playwright/test').Use} param0.use
+   */
+  tick: async ({ page }, use) => {
+    // eslint-disable-next-line func-style
+    const tick = async (milliseconds) => {
+      await page.evaluate((_milliseconds) => {
+        window.__clock.tick(_milliseconds);
+      }, milliseconds);
+    };
+    await use(tick);
+  },
   /**
    * Extends the base context class to add codecoverage shim.
    * @see {@link https://github.com/mxschmitt/playwright-test-coverage Github Project}
@@ -154,17 +187,13 @@ const extendedTest = test.extend({
     // function in the generatorWorker context. This is necessary
     // to ensure that example telemetry data is generated for the new clock time.
     if (clockOptions?.now !== undefined) {
-      page.on(
-        'worker',
-        (worker) => {
-          if (worker.url().includes('generatorWorker')) {
-            worker.evaluate((time) => {
-              self.Date.now = () => time;
-            });
-          }
-        },
-        clockOptions.now
-      );
+      page.on('worker', (worker) => {
+        if (worker.url().includes('generatorWorker')) {
+          worker.evaluate((time) => {
+            self.Date.now = () => time;
+          }, clockOptions.now);
+        }
+      });
     }
 
     // Capture any console errors during test execution