Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the end-to-end tests by adding summary tests #926

Merged
merged 3 commits into from
Feb 15, 2023

Conversation

tsaglam
Copy link
Member

@tsaglam tsaglam commented Feb 13, 2023

This PR adds summary tests to the e2e test suite, thus allowing one to quickly glance at the test result and understand the impact of the changes on all tests in total. The information is presented like this:

org.opentest4j.AssertionFailedError: positive deviation over all AVG similarity values:
	count=42, average=0.0260, min=0.0008, max=0.0833

org.opentest4j.AssertionFailedError: negative deviation over all AVG similarity values:
	count=284, average=0.0906, min=0.0012, max=0.2681

This PR also cleans up some code that is touched and improves JavaDoc comments.

…eck the deviation of average similarities. Also improve code quality.
@tsaglam tsaglam added enhancement Issue/PR that involves features, improvements and other changes major Major issue/feature/contribution/change labels Feb 13, 2023
@tsaglam tsaglam requested a review from a team February 13, 2023 11:48
@tsaglam tsaglam marked this pull request as ready for review February 13, 2023 11:48
@tsaglam tsaglam added minor Minor issue/feature/contribution/change and removed major Major issue/feature/contribution/change labels Feb 13, 2023
@sebinside sebinside requested review from sebinside and removed request for a team February 13, 2023 13:38
* @param second is the second double value.
*/
public void accept(double first, double second) {
double delta = first - second;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether this could lead to false positives due to the floating point nature

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test case that uses this only fails if the average of all deltas is above an epsilon value:

if (Math.abs(statistics.getAverage()) > EPSILON) {
fail(textualSign + " deviation over all AVG similarity values:" + System.lineSeparator() + statistics.toString());
}

This test case is mainly for feedback. For precision, the individual test cases (which produce the deltas) will fail if they deviate by the expected value (again, with a certain epsilon).

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

66.7% 66.7% Coverage
0.0% 0.0% Duplication

@tsaglam tsaglam merged commit 7b6662e into develop Feb 15, 2023
@tsaglam tsaglam deleted the feature/better-e2e-testing branch February 15, 2023 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Issue/PR that involves features, improvements and other changes minor Minor issue/feature/contribution/change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants