Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Minimum / Exact AT Version to Test Plan Reports #997

Closed
wants to merge 6 commits into from

Conversation

alflennik
Copy link
Contributor

See issue #791 - this PR addresses the server part of this task.

  • Adds minimumAtVersionId and exactAtVersionId to the TestPlanReports table and allows those fields to be passed in through the whole server from GraphQL. Note there is not currently any UI to provide these fields.
  • Removes limitation on duplicate TestPlanReports for a single AT / Browser combination. Since the frontend can't accommodate duplicates yet I made sure there is still no way for the frontend to generate any duplicates.
  • Removes some old migrations and endpoints that are no longer needed that would be incompatible with this change.

@alflennik alflennik requested a review from howard-e April 3, 2024 18:02
Copy link
Contributor

@howard-e howard-e left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and a promising start to an upcoming big process change here! Exciting! It's also great to see a lot of unnecessary code being removed.

I've left a few inline comments.

Comment on lines 181 to 183
support. The W3C ARIA-AT Community Group is currently
focusing on a stable and mature test suite for five
screen readers. In the future, we plan to test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great eye for detail to propose this change! This should probably go in it's own PR to get feedback on the copy change.

Copy link
Contributor Author

@alflennik alflennik Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I created #999 and removed the changes from here. Do you want to either assign yourself or whoever you think should review it?

@@ -1,634 +0,0 @@
'use strict';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

felt good!

browserId,
exactAtVersionId,
minimumAtVersionId,
vendorReviewStatus
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow why there would be a need for vendorReviewStatus to now be set when creating the test plan report. Could you explain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you're right, I removed it. Previously I got confused by code in the updateTestPlanReportTestPlanVersion file and that was why I abortively changed this function.

Comment on lines +157 to +160
// The proceed button is disabled because it will now create
// duplicate testPlanReports, and support has not yet been
// implemented across the full frontend to account for
// duplicates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably want to defend against this on the backend as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I added temporary code to enforce that.

Comment on lines +152 to +153
'browser and test plan version. Work is currently ' +
'underway to remove this limitation.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me wonder if we don't actually want to include this change in main until the rest of #791 and #792 is also ready. So we can instead treat this as a current dev "base"? Do you have any thoughts on that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it might become necessary at some point to have a branch for trend report changes. However personally I feel like this code is still mergeable to main, and we haven't arrived at the "ok now we need a separate branch" point.

Other than removing the "proceed" button, I don't think there's much impact on the user experience. In the event the admins do want to move a report from the reports page back to the test queue, we can accommodate that through the API.

That's my perspective but I'm happy to defer to whatever you think is better.

Do you think removing the "work is underway" sentence would mitigate your concern?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I'm okay with not yet considering this the "breaking point", so it could go to main but not just yet for 2 reasons:

  1. This particular feature has been used in recent times to make use of the current results copy featureset, in that it was unexpectedly only working if the results to be copied was in a report currently on the Test Queue, so this allowed a report to be pulled back. This won't be an issue once the enhancements identified in Improving the test results copy functionality #935 are also a part of main (PRs Support results being copied even when commands have been added or removed between versions #967, Copy results from advanced phase #976 and Copy results when adding to Test Queue #978) though.
  2. There is a planned release early next week and this new restriction hasn't yet been discussed. So this allows us the chance to notify them that this restriction is coming and so it also doesn't have to be forcibly excluded during the next push.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howard-e do you want to merge it into a branch with a nice name like "trends"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

@alflennik
Copy link
Contributor Author

@howard-e should be ready for a second pass.

Copy link
Contributor

@howard-e howard-e left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alflennik thanks for addressing the feedback! This is great and all very exciting for the next phase or process change!

Changes LGTM and I'm approving but also note that I don't think we should push to main just yet for reasons described in #997 (comment)

@howard-e howard-e changed the base branch from main to development April 4, 2024 22:27
@alflennik
Copy link
Contributor Author

This has been superseded by #1055!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants