[Iris 2.0] Control Center Data

Title:

Control Center Data

Author(s):

Matt Wobensmith

Tracking issue:

TBD

Short description:

The files and format required to display data in the Control Center, and the logic within Iris to generate these files.

Use case:

This feature enables the Control Center.

Implementation:

• Overview

There will be several discrete JSON files required to display various parts of the Control Center page. This spec will break down the details of these files and how they correspond to the page designs. Also, the spec will give some insight into how this data is generated.

• Sample code

targets.json

{
  "targets": [1]
  [
    {
      "name": "TARGET_NAME", [2]
      "icon": "IMAGE_NAME.png", [3]
      "tests": [4]
      [ 
        {
          "name": "PACKAGE_NAME", [5]
          "children": [6]
          [
            {
             "description": "TEST_DESCRIPTION", [7]
             "enabled": true/false, [8]
             "file": "/PATH/TO/TEST.py", [9]    
             "name": "TEST_NAME", [11]
             "values": {} [13a]
            }
          ]
        }
      ],
      "settings": [14]
      [
        {
          "name": "" [15]
          "type": "checkbox", [16]
          "label": "LABEL", [17]
        },
        {
          "name": "",
          "type": "list", [19]
          "label": "LABEL", [20]
          "value": [], [21]
          "default": value [22]
        },
        {
          "type": "text", [23]
          "label": "LABEL", [24]
          "value": "TEXT", [25]
        }        
      ]
    }
  ]
}

• [1] The root property targets is an array with zero or more target objects.
• [2] A target object will always have a name, which will be displayed in the left nav.
• [3] A target object will always have an icon, which will be displayed to the left of its name in the left nav.
• [4] A target object will always have a tests array, with zero or more test objects.
• [5] A package object will always have a name, which is displayed in the accordion/list.
• [6] A package object will always have a children array, with zero or more objects (packages or tests).
• [7] A test object will always have a description, which is displayed below the test name in the accordion/list.
• [8] A test object will always have a boolean property enabled, which will affect the visual state of the test name in the accordion/list.
• [9] A test object will always have a property named file, equal to the path to the test.
• [10] (deleted: locale)
• [11] A test object will always have a property named name, which is displayed in the accordion/list.
• [12] (deleted: platform)
• [13] (deleted: tags)
• [13a] A test object may have a property named values, which is an object containing target-specific key/value pairs.
• [14] A target object will always have a property named settings, which is an array of zero or more ordered HTML controls to be displayed in the right pane.
• [15] An HTML element object uses the property name to indicate the variable name to be associated with the returned value of this control.
• [16] An HTML element object uses the property type to indicate which control to display, here as a checkbox.
• [17] An HTML element object uses the property label to indicate the desired string to display next to this checkbox.
• [18] (deleted: no default value for checkbox)
• [19] An HTML element object with property type equal to list specifies a drop-down list.
• [20] This label property is used to specify what label to display next to the drop-down list.
• [21] This HTML element object uses the value property to specify an array of values to display in the list.
• [22] The default value will be used to indicate the default selected value in this list.
• [23] An HTML element object with property type equal to text specifies an editable text field.
• [24] This label property is used to specify what label to display next to the text field.
• [25] This HTML element uses the value property to specify what - if any - default text should populate this text field.

Notes:

• A package may contain any number of children, and they can be packages or tests. The example here only has one level of packages.

• Universal test properties: description, enabled (bool), file, name

  • Proposed changes from current tests:
    • Rename meta to description
    • Rename module to file

• These are the properties that all tests will have. However, a test may have additional properties in the values array, and they should be listed along side the base properties mentioned here. All properties should be alphabetized before display.

• The settings should be rendered in the page in the order that they are specified in this array. This is to allow the target module developer to have some influence over the look of the settings panel.

---

runs.json

{
 "runs": [
  {
   "duration": 96, [1]
   "failed": 1, [2]
   "id": "20181121164616", [3]
   "target": "firefox", [5]
   "total": 2 [6]
  }
 ]
}

• [1] The duration of the run, in seconds
• [2] Number of failed tests
• [3] Unique ID of run, derived from a timestamp (YYYYMMDDHHMMSS)
• [4] (deleted: locale)
• [5] Target name, to be used to find image icon on disk in reserved location
• [6] Total tests run

---

run.json

{
 "meta": { [1]
  "args": "/Users/username/.local/share/virtualenvs/iris-qxLEFYfH/bin/iris -t TEST_NAME -o", 
  "config": "10.13, 64-bit, x86_64", 
  "end_time": 1542824441, 
  "errors": 0, 
  "failed": 1, 
  "iris_branch": "dev", 
  "iris_branch_head": "fb42579fb9e6363391c571a95c2e344e63ea8fd7", 
  "iris_repo": "/Users/username/PycharmProjects/iris2", 
  "iris_version": 2.0, 
  "locale": "en-US", 
  "log": "/Users/username/.iris/runs/20181121182014/iris_log.log", [2]
  "params": { [3]
   "a_value": true,
   "b_value": 1
  }, 
  "passed": 1, 
  "platform": "osx", 
  "run_id": "20181121182014", 
  "skipped": 0, 
  "start_time": 1542824431, 
  "total": 2, 
  "total_time": 9,
  "values": { [4]
    "channel": "beta",
    "fx_build_id": "20181119162153", 
    "fx_version": "64.0", 
    "locale": "en-US"
    }
  }, 
  "failed_tests": [ [5]
    {
      "name": "PACKAGE_NAME", [6]
      "children": [ [7]
        {
          "assert": { [8]
              "actual": false, 
              "call_stack": "LONG_MULTILINE_ERROR_MESSAGE_GOES_HERE", [9],
              "code": "LONG_MULTILINE_CODE_SNIPPET_GOES_HERE", [9a]
              "expected": true, 
              "message": "A sentence that describes the condition being tested", 
              "outcome": "FAILED"
          }, 
          "debug_image_directory": "/Users/username/.iris2/runs/RUN_ID/package_name/test_name/debug_images", [10]
          "debug_images": [], [11]
          "description": "This is an empty test case that does nothing", 
          "name": "TEST_NAME", 
          "result": "FAILED", 
          "time": 8,
          "values": { [12]
            "fx_args": "",
            "pref_iris.enabled": true, 
            "profile": "like_new" 
          }
        }
      ]
    }
  ],
  "all_tests": [ [13]
    {
      "name": "PACKAGE_NAME",
      "children": [
        {
          "assert": { [14]
              "actual": false, 
              "call_stack": "LONG_MULTILINE_ERROR_MESSAGE_GOES_HERE",
              "code": "LONG_MULTILINE_CODE_SNIPPET_GOES_HERE",
              "expected": true, 
              "message": "A sentence that describes the condition being tested", 
              "outcome": "FAILED"
          }, 
          "debug_image_directory": "/Users/username/.iris2/runs/RUN_ID/package_name/test_name/debug_images", 
          "debug_images": [], 
          "description": "This is an empty test case that does nothing", 
          "name": "TEST_NAME", 
          "result": "FAILED", 
          "time": 8,
          "values": {
            "fx_args": "",
            "pref_iris.enabled": true, 
            "profile": "like_new" 
          }        
        }
      ]
    }
  ]
}

• [1] This is the data that will be placed under "Test Details"
• [2] This is the disk location of the log, perhaps we just display a clickable link
• [3] This is a list of key/value pairs to be displayed in alphabetical order
• [4] This is an object with zero or more values specific to the target to be displayed below the params. These values here are for Firefox.
• [5] This is the data that will be placed under "Failed Tests"
• [6] This is a package name, equal to a directory name.
• [7] The presence of a children array indicates that this is a package containing zero or more sub items.
• [8] The assert object will be displayed in an area specific for asserts.
• [9] The call_stack value will be a long string that will require wrappable text and a fixed-width font.
• [9a] The code value will be a long string that will require wrappable text and a fixed-width font.
• [10] The debug_image_directory is not to be displayed, but instead, used to build links to the next item below in [11].
• [11] The debug_images array is a list of file names to be concatenated to the value in [10] in order to be displayed in the image carousel.
• [12] This serves the same purpose as item [4] above, except that it applies to test cases.
• [13] This is the data that will be placed under "All Tests". It is in the same format as "Failed Tests" but may contain far more items.
• [14] If a test has failed, it will contain an assert object. A passed test will not.

---

• New and/or changed files and directories

• Localization

• Documentation

• Bootstrap

• Setup

Risks:

• Dependencies on other people/teams/software.

• Other factors that could influence functionality or schedule.

Estimated schedule:

• xxxx-xx-xx Spec due date (this)
• xxxx-xx-xx Implementation
• xxxx-xx-xx Code review
• xxxx-xx-xx Testing
• xxxx-xx-xx Final check-in date