Change level of enabled-by-default CodeQL scans when merging to default branch?

[consideRatio]

We have CodeQL scans we can't opt out of

Apparently merging to master will trigger a CodeQL scan to run, which for us fails for binderhub with warnings of High severity or higher. Currently, binderhub fail, see https://github.com/jupyterhub/binderhub/runs/3774579963 as an example. We can't opt out of these, and these running on merge is entirely decoupled from the fact that we in binderhub have opted to run a CodeQL on a scheduled basis via a dedicated workflow job.

/cc: @betatim who observed this!

Observed failures

We currently fail by logging what CodeQL considers sensitive information, while I would disagree and consider it safe. As an example, see https://github.com/jupyterhub/binderhub/runs/3774579963.

Failure level choice

In https://github.com/jupyterhub/jupyterhub/settings/security_analysis we can change the failure level of the warnings by CodeQL, it was set at High by default, as it is for other repo's like JupyterHub, but I now made it Critical instead as we got false positives and I don't want us to fix those just because this tool is forced on us by GitHub.

Removal of CodeQL scheduled runs

The scheduled runs of CodeQL have never failed, so perhaps they are misconfigured or configured to fail on critical? See the latest runs here: https://github.com/jupyterhub/binderhub/actions/workflows/codeql-analysis.yml

I suggest these are removed entirely.

Suggested action points to close this issue

• Lower failure level from High to Critical for the CodeQL we are required to have, as configurable via https://github.com/jupyterhub/jupyterhub/settings/security_analysis
• Remove the scheduled GitHub Actions workflow running CodeQL as a cronjob

[betatim]

I'm in favour of dropping CodeQL. I don't think anyone looks at/reacts to what it tells us :-/