Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support V2 Test Format build #997

Merged
merged 68 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
a6a18e6
Add scripts to convert a test plan from v1 to v2 format
mcking65 Sep 3, 2023
4ee7e76
Add and revise support and commands JSON data
mcking65 Sep 4, 2023
de428a2
WIP: Building test index page
howard-e Sep 21, 2023
e3d3398
Adding additional TODOs to be covered; start to addressing review page
howard-e Sep 22, 2023
af9d4ec
Fix references after rebase
howard-e Sep 25, 2023
4613dc7
Add additional data validations
howard-e Sep 25, 2023
7873d9b
Use MUST/SHOULD/MAY on review page
howard-e Sep 25, 2023
c980a8a
Update validations
howard-e Sep 25, 2023
aa8ce7d
Now accounting for modeless ATs
howard-e Sep 26, 2023
1882ab8
Additional validations
howard-e Sep 27, 2023
dbc8e97
Accounting for assertionExceptions
howard-e Sep 27, 2023
24989f7
Additional validations and accounting for assertionExceptions; review…
howard-e Oct 2, 2023
877d8e6
Keys handling in test-io-format.mjs
howard-e Oct 2, 2023
c48c07f
Update support.json
howard-e Oct 3, 2023
2e0adb6
Remove 'v2-made' files
howard-e Oct 3, 2023
57a431d
Change 'NA' for modeless to 'defaultMode'
howard-e Oct 3, 2023
8c2eeea
Format .mjs files
howard-e Oct 3, 2023
b05acc7
Fix lines in keys.mjs
howard-e Oct 3, 2023
48c7d2f
Remove create-example-tests2
howard-e Oct 3, 2023
97dfcec
Revert keys.mjs quotations
howard-e Oct 3, 2023
0d4c363
assertionTokens replacement
howard-e Oct 3, 2023
a6f20f8
show replaced assertion tokens in reviewed tests
howard-e Oct 3, 2023
4b65f78
Conditionally show modeInstructions
howard-e Oct 3, 2023
8ebddc3
testing
howard-e Oct 3, 2023
16091a0
Set mode for commandInfo
howard-e Oct 4, 2023
a700100
Better support for modeless in aria-at-test-io-format.mjs and aria-at…
howard-e Oct 7, 2023
8e5be78
Better support for modeless in test-reviewer.mjs
howard-e Oct 8, 2023
654340c
Formatting
howard-e Oct 8, 2023
9da42f2
Update v2maker to create a new row for commands previously defined as…
howard-e Oct 8, 2023
2006c50
Avoid setupScript thrown error when looking for script file
howard-e Oct 8, 2023
e517266
Formatting test-reviewer.mjs
howard-e Oct 8, 2023
c9ab183
Handle unexpected cases with v2 maker related scripts
howard-e Oct 8, 2023
2c21785
Handle thrown error csv properties to pass validations (for now)
howard-e Oct 8, 2023
9027cab
Additional cleanup
howard-e Oct 10, 2023
87ffecd
Fix pattern name identifier on index pagae
howard-e Oct 10, 2023
8e813f9
Formatting
howard-e Oct 10, 2023
c20d7c9
Handle all 'at'-commands.csv files without looking specifically for k…
howard-e Oct 10, 2023
84400a4
Additional error handling for missing at-commands.csv files
howard-e Oct 10, 2023
bcf6453
Update jsdocs
howard-e Oct 10, 2023
6530433
Formatting
howard-e Oct 10, 2023
761a1b7
Conditionally build v2 over v1 test format
howard-e Oct 10, 2023
8f40d54
Refactor name for clarity
howard-e Oct 10, 2023
23b23be
Edge case handling for test with multiple settings
howard-e Oct 11, 2023
a41bf24
Optimize edge case handling
howard-e Oct 11, 2023
809bd84
Misc
howard-e Oct 11, 2023
54e5eb2
Fix showing mode instructions on review page
howard-e Oct 12, 2023
cd9b78a
Update validations
howard-e Oct 12, 2023
f54c30e
Update error message
howard-e Oct 12, 2023
0793b63
Revert changes in package.json
howard-e Oct 12, 2023
7963d7b
Fix issues with Delete command validation
stalgiag Oct 25, 2023
cd4aef2
Remove unused function
howard-e Nov 1, 2023
b38756c
Ensure rows in *-commands.csv are maintained by the presentationNumber
howard-e Nov 1, 2023
13d4597
Fix case where titles and instructions were still unexpectedly refere…
howard-e Nov 2, 2023
9a7bf5d
Fix review page to stop using generated commandPresentationNumbers
howard-e Nov 3, 2023
ce83a4d
Update aria-at-test-run.mjs to drop support for generated command seq…
howard-e Nov 3, 2023
0bcb378
Better support of 0-level assertion exceptions on review page
howard-e Nov 6, 2023
d7de561
Fix linter
howard-e Nov 6, 2023
423a901
Remove 'modes' from v2 test file names
howard-e Nov 6, 2023
cc149f3
Remove comment
howard-e Nov 6, 2023
b294d4d
Add mode to individual *.collected.(json|html) files
howard-e Nov 6, 2023
c16ed8a
Fix '1 test's'' editorial error on review page
howard-e Nov 7, 2023
6fe5722
Fix for presenting .collected.html when needed
howard-e Nov 7, 2023
6b28a9b
Add support for multiple settings with .collected
howard-e Nov 13, 2023
ef5bcb9
Don't show assertionPhrase column on review page for v1 build
howard-e Nov 14, 2023
3563516
Fix when commandListSettingsPreface gets appended
howard-e Nov 15, 2023
52fa984
Change from {at}-{settings} .collected files to a single {at} .collec…
howard-e Nov 15, 2023
0eead51
Fix combined .collected.(json|logic)
howard-e Nov 16, 2023
e617ea4
Update results collection form to support pass/fail assertion verdict…
stalgiag Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion lib/data/command-tuples-at-mode-task-lookup.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,28 @@ function createCommandTuplesATModeTaskLookup(commands) {
return data;
}

exports.createCommandTuplesATModeTaskLookup = createCommandTuplesATModeTaskLookup;
function createAtCommandTuplesATSettingsTestIdLookupByPresentationNumber(commands) {
return commands.reduce((carry, command) => {
const commandTask = carry[command.testId] || {};
const commandTaskMode = commandTask[command.target.at.settings || 'defaultMode'] || {};
const commandTaskModeAT = commandTaskMode[command.target.at.key] || [];
const commandTuples = command.commands.map(({ id, presentationNumber }) => [
`${id}|${presentationNumber}`,
]);
return {
...carry,
[command.testId]: {
...commandTask,
[command.target.at.settings || 'defaultMode']: {
...commandTaskMode,
[command.target.at.key]: [...commandTaskModeAT, ...commandTuples],
},
},
};
}, {});
}

module.exports = {
createCommandTuplesATModeTaskLookup,
createAtCommandTuplesATSettingsTestIdLookupByPresentationNumber,
};
192 changes: 191 additions & 1 deletion lib/data/parse-command-csv-row.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,194 @@ function parseCommandCSVRow(commandRow) {
};
}

exports.parseCommandCSVRow = parseCommandCSVRow;
function flattenObject(obj, parentKey = '') {
const flattened = {};

for (const key in obj) {
if (typeof obj[key] === 'object') {
const subObject = flattenObject(obj[key], parentKey + key + '.');
Object.assign(flattened, subObject);
} else {
flattened[parentKey + key] = obj[key];
}
}

return flattened;
}

function sanitizeWhitespace(value) {
return value.replace(/\s+/g, ' ').trim();
}

function sanitizeCommand(command) {
return {
...command,
command: sanitizeWhitespace(command.command),
};
}

function findValueByKey(keyMappings, keyToFindText) {
const keyToFind = keyToFindText.replace(/\s+/g, ' ').trim();
const keyMap = Object.keys(keyMappings);

// Need to specially handle VO modifier key combination
if (keyToFind === 'vo')
return findValuesByKeys(keyMappings, [keyMappings['modifierAliases.vo']])[0];

if (keyToFind.includes('modifiers.') || keyToFind.includes('keys.')) {
const parts = keyToFind.split('.');
const keyToCheck = parts[parts.length - 1]; // value after the '.'

if (keyMappings[keyToFind])
return {
value: keyMappings[keyToFind],
key: keyToCheck,
};

return null;
}

for (const key of keyMap) {
const parts = key.split('.');
const parentKey = parts[0];
const keyToCheck = parts[parts.length - 1]; // value after the '.'

if (keyToCheck === keyToFind) {
if (parentKey === 'modifierAliases') {
return findValueByKey(keyMappings, `modifiers.${keyMappings[key]}`);
} else if (parentKey === 'keyAliases') {
return findValueByKey(keyMappings, `keys.${keyMappings[key]}`);
}

return {
value: keyMappings[key],
key: keyToCheck,
};
}
}

// Return null if the key is not found
return null;
}

function findValuesByKeys(commandsMapping, keysToFind = []) {
const result = [];

const patternSepWithReplacement = (keyToFind, pattern, replacement) => {
if (keyToFind.includes(pattern)) {
let value = '';
let validKeys = true;
const keys = keyToFind.split(pattern);

for (const key of keys) {
const keyResult = findValueByKey(commandsMapping, key);
if (keyResult) value = value ? `${value}${replacement}${keyResult.value}` : keyResult.value;
else validKeys = false;
}
if (validKeys) return { value, key: keyToFind };
}

return null;
};

const patternSepHandler = keyToFind => {
let value = '';

if (keyToFind.includes(' ') && keyToFind.includes('+')) {
const keys = keyToFind.split(' ');
for (let [index, key] of keys.entries()) {
const keyToFindResult = findValueByKey(commandsMapping, key);
if (keyToFindResult) keys[index] = keyToFindResult.value;
if (key.includes('+')) keys[index] = patternSepWithReplacement(key, '+', '+').value;
}
value = keys.join(' then ');

return { value, key: keyToFind };
} else if (keyToFind.includes(' ')) return patternSepWithReplacement(keyToFind, ' ', ' then ');
else if (keyToFind.includes('+')) return patternSepWithReplacement(keyToFind, '+', '+');
};

for (const keyToFind of keysToFind) {
if (keyToFind.includes(' ') || keyToFind.includes('+')) {
result.push(patternSepHandler(keyToFind));
} else {
const keyToFindResult = findValueByKey(commandsMapping, keyToFind);
if (keyToFindResult) result.push(keyToFindResult);
}
}

return result;
}

function parseCommandCSVRowV2({ commands }, commandsJson) {
const commandsParsed = [];
const flattenedCommandsJson = flattenObject(commandsJson);

function createParsedCommand(command, atTargetInfo) {
const commandIds = command.command.split(' ');
const commandKVs = findValuesByKeys(flattenedCommandsJson, [command.command]);

const assertionExceptions = command.assertionExceptions.length
? sanitizeWhitespace(command.assertionExceptions).split(' ')
: [];

const commands = commandKVs.map(e => ({
id: e.key,
keystroke: e.value,
keypresses: e.value.split(' then ').map((e, index) => ({
id: commandIds[index],
keystroke: e,
})),
presentationNumber: Number(command.presentationNumber),
assertionExceptions,
}));

// Also account for 'modeless' AT configurations with 'defaultMode'
const settingsValue = command.settings || 'defaultMode';

const foundCommandIndex = commandsParsed.findIndex(
e =>
e.testId === command.testId &&
e.target.at.key === atTargetInfo.key &&
e.target.at.settings === settingsValue &&
(e.commands.length
? e.commands.every(
({ presentationNumber }) =>
parseInt(presentationNumber) === parseInt(command.presentationNumber)
)
: true)
);

// Add new parsed command since none exist
if (foundCommandIndex < 0) {
commandsParsed.push({
testId: command.testId,
target: {
at: {
...atTargetInfo,
settings: settingsValue,
},
},
commands,
});
} else {
commandsParsed[foundCommandIndex].commands.push(...commands);
}
}

const generateCommandsParsed = (commands, key) => {
for (const command of commands) createParsedCommand(command, { key });
};

commands.forEach(command =>
generateCommandsParsed(command.commands.map(sanitizeCommand), command.atKey)
);

return commandsParsed;
}

module.exports = {
parseCommandCSVRow,
parseCommandCSVRowV2,
flattenObject,
};
2 changes: 2 additions & 0 deletions lib/data/parse-support.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ function parseSupport(supportRaw) {
ats: [at],
})),
],
testPlanStrings: supportRaw.testPlanStrings,
references: supportRaw.references,
};
}

Expand Down
Loading
Loading