Improve GitHub Actions and CDash integration (#154)

[bartlettroscoe]

See #154 and commits.

[bartlettroscoe]

The final version of this is shown at:

• https://testing.sandia.gov/cdash/index.php?project=TriBITS&filtercount=2&showfilters=1&filtercombine=and&field1=buildstarttime&compare1=83&value1=2021-08-10T17%3A44%3A50%20UTC&field2=buildstarttime&compare2=84&value2=2021-08-10T17%3A45%3A09%20UTC

which shows:

There we see the icon which links back to this PR. This shows setting the CTEST_CHANGE_ID is working.

Note the "Revision" column showing the SHA1 9fb34d. This is showing that setting CTEST_UPDATE_VERSION_ONLY=ON is working. What is interesting is that SHA1 is for a local merge commit as shown here which is:

TriBITS repos versions:
--------------------------------------------------------------------------------
*** Base Git Repo: TriBITS
9fb34dd [Tue Aug 10 17:43:53 2021 +0000] <[email protected]>
Merge 60bf144236a21873ccc9145c8c8b5ba39374f16b into 9c210c534f2211cd864ca2ef7608
 --------------------------------------------------------------------------------

What is interesting about this local merge commit 9fb34dd (that merges the tip of this PR topic branch at 60bf144 into the tip of master at 9c210c5) is that GitHub actually provides a link for this (as you can see by the hyperlink). (NOTE: The shortened SHA1 9fb34d is not long enough for GitHub to show the link. This shows that CTest or CDash is not checking with git to see how many chars of the SHA1 is needed for the repo, so you have to click on the link to get it the full SHA1.) From the CDash page clicked from 9fb34d to:

• https://testing.sandia.gov/cdash/viewUpdate.php?buildid=9707976

if you click on 9fb34ddf7dcc64b28901d12afbbdde24653b28bc it takes you to:

• 0544139...9fb34dd

which is the full set of commits against 'master'. But note that CTest or CDash does not do this for every CDash build (it seems to show the new commits from the previous matching build which in this case showed them all because this was a full rebase on top of 'master').

The matching GithHub Actions jobs for this last set of PR builds is at:

• https://github.com/TriBITSPub/TriBITS/actions/runs/1117473935

You can see the links to the individual builds on CDash, for example build (ubuntu-latest, 3.17.5, makefiles, 2.7, gcc-8, g++-8, gfortran-8), at:

• https://github.com/TriBITSPub/TriBITS/runs/3293339241?check_suite_focus=true

which under "URL to results on CDash"

provides the hyperlink:

• https://testing.sandia.gov/cdash/index.php?project=TriBITS&filtercount=3&showfilters=1&filtercombine=and&field1=site&compare1=61&value1=ubuntu-latest&field2=buildname&compare2=61&value2=pr-401_tribits_cmake-3.17.5_makefiles_python-2.7_g%2B%2B-8&field3=buildstamp&compare3=61&value3=20210810-1744-Pull%20Request

which shows:

And one can see all of the builds for this PR at:

• https://testing.sandia.gov/cdash/index.php?project=TriBITS&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=pr-401_&field2=buildstarttime&compare2=84&value2=now

which shows:

What this demonstrates is:

• Links are provided from GitHub Actions Jobs to results on CDash (through CDash URL implemented manually in tribits_ctest_drdiver())
• Links on CDash builds are provided back to the GitHub PRs (through setting CTEST_CHANGE_ID=<pr-id>)
• The builds on CDash are prefixed by the PR ID with pr_<pr-id>_ so CDash queries can be easily constructed that show all of these.
• The Git repo SHA1 is shown on CDash (which shows merge commits for PRs but shows commits on 'master' for 'Continuous' and 'Nightly' builds).

I guess what is still missing is the ability to link from a CDash build back to the specific GitHub Action Job that ran it. For example, it might be possible to provide a URL link in a build note file back to the GitHub Action job like from the build https://testing.sandia.gov/cdash/index.php?project=TriBITS&parentid=9707974 back to https://github.com/TriBITSPub/TriBITS/runs/3293339241. That is the only link that I see that is missing in this setup that would be useful to have. But one can see all of the GitHub Actions runs for a given PR by running the query like:

• https://github.com/TriBITSPub/TriBITS/actions/workflows/tribits_testing.yml?query=event%3Apull_request+branch%3A154-improve-cdash-github-interaction

showing 8 GitHub Actions iterations showing:

and then one can compare that to the builds on CDash at:

• https://testing.sandia.gov/cdash/index.php?project=TriBITS&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=pr-401_&field2=buildstarttime&compare2=84&value2=now

(with output shown above showing 32 builds) and it is not too hard to match those up.

But if we wanted to provide links from CDash builds to the exact GitHub Actions job that run it, it seems that info might be provided by one of the env vars listed at:

• https://docs.github.com/en/actions/reference/environment-variables

That might be future work to cap that off if I decide to write a report on this or something. (Being able to link from CDash to the exact job that ran it is missing on most systems that I have seen.)

[bartlettroscoe]

With this review I think this is done and ready to merge (finally).

[bartlettroscoe]

Note, however, when there is an error in a PR build, the GitHub Actions job never executes the "URL to results on CDash" step and therefore the link to result on CDash is shown at the bottom in the "Run configure, build, test and submit to CDash" step as shown below.

NOTE: The above PR iteration was before I added https:// to the beginning of the URL to provide the hyperlink to the CDash address.

[bartlettroscoe]

FYI, you can see an example of the hyperlink when a build fails at:

• https://github.com/TriBITSPub/TriBITS/runs/3316657367?check_suite_focus=true

showing: