Add periodic job stats

[anandrkskd]

Signed-off-by: anandrkskd [email protected]

This PR will add periodic job stats to the dashboard and will improve analysis of flakes on prow-CI.

Fixes #3678

[anandrkskd]

NOTE

This is the new dashboard which will be usefull to analyis falkes.
This dashboard still require some tweaks to be done, but will be similar to this. 

odo test statistics

Last update: 2020-12-08 10:31:42 (UTC)

Generated with https://github.com/jgwest/odo-tools/ and https://github.com/kadel/odo-tools

FLAKY TESTS: Failed test scenarios in past 14 days

Failure Score*	Failures on PR	Failures on PrediocJob	Test Name	Last Seen	PR List and Logs	PeriodicJob failure List
1200	20	9	[Fail] odo service command tests for OperatorHub When linking devfile component with Operator backed service [It] should successfully connect and disconnect a component with an existing service	1 days ago	10: #42741, 2, 3, 4 #42681 #42671 #42651, 2 #42611, 2, 3 #42601 #42541, 2 #42471, 2, 3 #41761 #41191	4.2 4.4 4.4 4.5 4.5 4.6 4.6 4.6 4.6
3	2	9	[Fail] odo devfile push command tests Verify devfile push works [It] should create pvc and reuse if it shares the same devfile volume name	13 days ago	2: #42541 #42501
-	-	3	[Fail] odo devfile push command tests Pushing devfile without an .odo folder [It] should be able to push based on metadata.name in devfile WITH a dash in the name	-	-	4.2 4.3 4.5
-	-	1	[Fail] odo push command tests when push command is executed [It] should build when a file and a folder is renamed in the directory	-	-	4.2
-	-	9	[Fail] odo service command tests for OperatorHub When linking devfile component with Operator backed service [It] should successfully connect and disconnect a component with an existing service	-	-	4.2 4.4 4.4 4.5 4.5 4.6 4.6 4.6 4.6
-	-	1	[Fail] odo supported images e2e tests odo supported images deployment [It] Should be able to verify the openjdk-11-rhel7 image	-	-	4.3
-	-	1	[Fail] odo storage command tests when using storage command with -o json [It] should create and list output in json format	-	-	4.3
-	-	1	[Fail] odo sub component command tests Creating component using symlink [It] Should be able to deploy a wildfly war file using symlinks in some odo commands	-	-	4.3
-	-	2	[Fail] odo component command tests Creating component using symlink [It] Should be able to deploy a wildfly war file using symlinks in some odo commands	-	-	4.3 4.5
-	-	1	[Fail] odo devfile delete command tests when devfile delete command is executed [It] should execute preStop events if present	-	-	4.3
-	-	1	[Fail] odo devfile url command tests Testing URLs for OpenShift specific scenarios [It] should create a url for a unsupported devfile component	-	-	4.3
-	-	1	[Fail] odo service command tests for OperatorHub When creating and deleting an operator backed service [JustBeforeEach] should be able to create service with name passed on CLI	-	-	4.3
-	-	1	[Fail] odo service command tests for OperatorHub When linking devfile component with Operator backed service [JustBeforeEach] should fail if the provided service doesn't exist in the namespace	-	-	4.3
-	-	1	[Fail] odo service command tests for OperatorHub When using from-file option [JustBeforeEach] should fail to create service if metadata doesn't exist or is invalid	-	-	4.3
-	-	1	[Fail] odo service command tests for OperatorHub When deleting an invalid operator backed service [It] should correctly detect invalid service names	-	-	4.4
-	-	1	[Fail] odo sub component command tests Creating component [It] Without an application should create one	-	-	4.4
-	-	1	[Fail] odo service command tests for OperatorHub When using from-file option [It] should be able to create a service with name passed on CLI	-	-	4.4
-	-	1	[Fail] odo service command tests for OperatorHub When operator backed services are created [It] should list the services if they exist	-	-	4.4
-	-	1	[Fail] odo service command tests for OperatorHub When linking devfile component with Operator backed service [It] should fail if the provided service doesn't exist in the namespace	-	-	4.4
-	-	1	[Fail] odo service command tests for OperatorHub When using from-file option [It] should be able to create a service	-	-	4.4
-	-	1	[Fail] odo service command tests for OperatorHub When creating and deleting an operator backed service [It] should be able to create, list and then delete EtcdCluster from its alm example	-	-	4.4
-	-	2	[Fail] odo service command tests for OperatorHub When creating and deleting an operator backed service [It] should be able to create service with name passed on CLI	-	-	4.4 4.6
-	-	1	[Fail] odo supported images e2e tests odo supported private registry images deployment [It] Should be able to verify the openjdk-11-rhel8 image	-	-	4.4
-	-	1	[Fail] odo core beta flow when --context flag is used [It] create local java component and push code	-	-	4.4
-	-	1	[Fail] odo devfile push command tests [BeforeEach] Verify devfile push works should be able to handle a missing build command group	-	-	4.5
-	-	1	[Fail] odo generic [AfterEach] Check the help usage for odo Makes sure that we have the long-description when running odo and we dont error	-	-	4.5
-	-	1	[Fail] odo push command tests Check for label propagation after pushing [It] Check for labels	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component delete, list and describe should pass outside a odo directory with component name as parameter	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] odo component delete should clean owned resources should delete the component and the owned resources	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Test odo push with --source and --config flags when --context is used create local nodejs component and push source and code at once	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when push command is executed should build when a file and a folder is renamed in the directory	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should list out pushed components of different projects in json format along with path flag	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component delete should clean owned resources should delete the component and the owned resources with wait flag	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should list the component	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check memory and cpu config before odo push should fail if maxmemory is set but minmemory is not set..	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] creating component with an application and url should create the component in default application	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should list the component	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component updating should be able to create a git component and update it from local to git	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when push command is executed should build when no changes are detected in the directory and force flag is enabled	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] check that help works should display help info	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should list out component in json format along with path flag	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] When deleting two project one after the other should be able to delete sequentially	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should create but not list component even in new project with --project and --context at the same time	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component create component twice fails from same directory	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local java component and check unsupported warning hasn't occurred	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component updating should be able to update a component from git to local	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] odo component updating should be able to create a git component and update it from local to git	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] prevent the user from creating invalid URLs should not allow creating a URL with long name	-	-	4.5
-	-	1	[Fail] odo debug command tests [BeforeEach] odo debug info should work on a odo component should start a debug session and run debug info on a closed debug session	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component binary component should fail when --binary is not in --context folder	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check memory and cpu config before odo push Should fail if mincpu is set but maxcpu is not set..	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Test odo push with --source and --config flags Using project flag(--project) and current directory create local nodejs component and push source and code at once	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used create local nodejs component twice and fail	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component using symlink Should be able to deploy a spring boot uberjar file using symlinks in all odo commands	-	-	4.5
-	-	1	[Fail] odo storage command tests [BeforeEach] when running storage list command to check state should list storage with correct state	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Test odo push with --source and --config flags when --context is used create local nodejs component and push source and code at once	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Test odo push with --source and --config flags Using project flag(--project) and current directory create local nodejs component and push source and code separately	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should succeed listing catalog components	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should list out pushed components of different projects in json format along with path flag	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when using --now with config command should successfully set and unset variables	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should describe not pushed component when it is created with json output	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local python component, pushes it and then deletes it using --all flag	-	-	4.5
-	-	1	[Fail] odo url command tests [BeforeEach] Listing urls should create a secure URL	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Test odo push with --source and --config flags Using project flag(--project) and current directory create local nodejs component and push source and code at once	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when component is in the current directory and --project flag is used create local nodejs component twice and fail	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should list the component in the same app when one is pushed and the other one is not pushed	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should show an error when ref flag is provided with sources except git	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] when running help for app command should display the help	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when running odo push multiple times, check for existence of environment variables should should retain the same environment variable on multiple push	-	-	4.5
-	-	1	[Fail] odo storage command tests [BeforeEach] when using storage command with default flag values should add a storage, list and delete it	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component delete should clean owned resources should delete the component and the owned resources	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] odo component delete, list and describe should pass inside a odo directory without component name as parameter	-	-	4.5
-	-	1	[Fail] odo link and unlink command tests [BeforeEach] when running help for link and unlink command should display the help	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component binary component should not fail when --context is not set	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] when component's deployment config is deleted with oc should delete all OpenShift objects except the component's imagestream	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component using symlink Should be able to deploy a wildfly war file using symlinks in some odo commands	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local python component, pushes it and then deletes it using --all flag in local directory	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] convert s2i to devfile should convert s2i component to devfile component successfully	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should list out component in json format along with path flag	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check pod timeout Check that pod timeout works and we time out immediately..	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] when running app command app parameter in directory that doesn't contain .odo config directory should successfully execute list, describe and delete along with machine readable output	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when .odoignore file exists should create and push the contents of a named component excluding the contents and changes detected in .odoignore file	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when viewing local config without logging into the OpenShift cluster should list config successfully	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should list the state as unknown for disconnected cluster	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local python component, pushes it and then deletes it using --all flag	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should create default named component when passed same context differently	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when push command is executed should be able to create a file, push, delete, then push again propagating the deletions and build	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check memory and cpu config before odo push Should work when memory is set..	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] Check the help usage for odo Fail when entering an incorrect name for a component	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component with numeric named context should create default named component in a directory with numeric name	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local python component and check for unsupported warning	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should list the component when it is not pushed	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component using symlink Should be able to deploy a spring boot uberjar file using symlinks in all odo commands	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component binary component is valid if path is relative and includes ../	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Generic machine readable output tests Command should fail if json is non-existent for a command	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] Check the help usage for odo Fail with showing help only once for incorrect command	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when creating odo local config with env variables should set and unset env variables	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] when creating project -o json should be able to create project and show output in json format	-	-	4.5
-	-	1	[Fail] odo watch command tests [BeforeEach] when running help for watch command should display the help	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] odo component delete should clean owned resources should delete the component and the owned resources with wait flag	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] When running app describe with storage added in component in directory that doesn't contain .odo config directory should successfully execute describe	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should succeed listing catalog components	-	-	4.5
-	-	1	[Fail] odo debug command tests [BeforeEach] odo debug on a nodejs:latest component should expect a ws connection when tried to connect on default debug port locally	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Test odo push with --now flag during creation should successfully create config and push code in one create command with --now	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] check catalog component search functionality check that a component does not exist	-	-	4.5
-	-	1	[Fail] odo url command tests [BeforeEach] when listing urls using -o json flag should be able to list url in machine readable json format	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used creates and pushes local nodejs component and then deletes --all	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] deploying a component with a specific image name should deploy the component	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local nodejs component and check unsupported warning hasn't occurred	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should describe pushed component when it is created with json output	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should describe pushed component when it is created with json output	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] when running app command without app parameter in directory that doesn't contain .odo config directory should fail without app parameter (except the list command)	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should create but not list component even in new project with --project and --context at the same time	-	-	4.5
-	-	1	[Fail] odo debug command tests [BeforeEach] odo debug on a nodejs:latest component check that machine output debug information works	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] when executing odo version command should show the version of odo major components including server login URL	-	-	4.5
-	-	1	[Fail] odo url command tests [BeforeEach] Listing urls should list appropriate URLs and push message	-	-	4.5
-	-	1	[Fail] odo watch command tests [BeforeEach] when executing watch on a git source type component should fail	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Generic machine readable output tests Help for odo version should not contain machine output	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] when running app delete, describe and list command on fresh cluster should error out display the help	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] When viewing global config should get the default global config keys	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] odo component updating should be able to update a component from git to local	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] When deleting three project one after the other in opposite order should be able to delete	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] Delete the project with flag -o json should be able to delete project and show output in json format	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when viewing local config without logging into the OpenShift cluster should set config variable without logging in	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] convert s2i to devfile should convert s2i component to devfile component successfully	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] odo component delete, list and describe should pass inside a odo directory without component name as parameter	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when creating odo local config with context flag should allow setting and unsetting a config locally with context	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component using symlink Should be able to deploy a wildfly war file using symlinks in some odo commands	-	-	4.5
-	-	1	[Fail] odo watch command tests [BeforeEach] when executing odo watch against an app that doesn't exist should fail with proper error	-	-	4.5
-	-	1	[Fail] odo debug command tests [BeforeEach] odo debug info should work on a odo component should start a debug session and run debug info on a running debug session	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Generic machine readable output tests Command should fail if json is non-existent for a command	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Test odo push with --source and --config flags when --context is used create local nodejs component and push source and code separately	-	-	4.5
-	-	1	[Fail] odo storage command tests [BeforeEach] when running help for storage command should display the help	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check memory and cpu config before odo push should fail if maxcpu is set but mincpu is not set..	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should show an error when ref flag is provided with sources except git	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Creating component should describe not pushed component when it is created with json output	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when push command is executed should push only the modified files	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] when creating odo local config in the same config dir should set, unset local config successfully	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] Overwriting build timeout for git component should fail to build component if the given build timeout is pretty less(2s)	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component should create default named component when passed same context differently	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when running odo push multiple times, check for existence of environment variables should should retain the same environment variable on multiple push	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] when .gitignore file exists should create and push the contents of a named component and include odo-file-index.json path to .gitignore file to exclude the contents	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] Test odo push with --source and --config flags when --context is used create local nodejs component and push source and code separately	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] When configuring global config values should successfully updated	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] when component is in the current directory and --project flag is used creates and pushes local nodejs component and then deletes --all	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] Creating same project twice with flag -o json should fail along with proper machine readable output	-	-	4.5
-	-	1	[Fail] odo storage command tests [BeforeEach] when using storage command with -o json should create and list output in json format	-	-	4.5
-	-	1	[Fail] odo preference and config command tests [BeforeEach] When configuring global config values should show json output	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component binary component should fail when --binary is not in --context folder	-	-	4.5
-	-	1	[Fail] odo sub component command tests [BeforeEach] Creating component with numeric named context should create default named component in a directory with numeric name	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] When using cpu or memory flag with odo create should not allow using any memory or cpu flag	-	-	4.5
-	-	1	[Fail] odo push command tests [BeforeEach] Check memory and cpu config before odo push Should work when cpu is set	-	-	4.5
-	-	1	[Fail] odo component command tests [BeforeEach] when component is in the current directory and --project flag is used creates a local nodejs component and check unsupported warning hasn't occurred	-	-	4.5
-	-	1	[Fail] odo app command tests [BeforeEach] when running app command without app parameter in directory that contains .odo config directory should successfully execute list, describe and delete along with machine readable output	-	-	4.5
-	-	1	[Fail] odo generic [BeforeEach] should list applications in other project should be able to create a php component with application created	-	-	4.5
-	-	1	[Fail] odo push command tests Test push outside of the current working direcory [It] Push, modify a file and then push outside of the working directory	-	-	4.6
-	-	1	[Fail] odo devfile url command tests Creating urls [It] should create URLs under different container names	-	-	4.6

^* - Failure score is an arbitrary severity estimate, and is approximately (# of PRs the test failure was seen in * # of test failures) / (days since failure). See code for full algorithm -- PRs welcome for algorithm improvements.

[prietyc123]

commented code found