🌱 cron: generalize and expose worker (6/n)

[spencerschrock]

What kind of change does this PR introduce?

Part of a series of refactors to generalize the cron infrastructure to enable re-use by the ossf criticality score project

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

The worker logic is scorecard specific and can't be re-used.

What is the new behavior (if this is a feature change)?**

The general logic has been separated from the scorecard specific logic.
Criticality should be able to import the required modules and create their own worker which implements the interface.

The following packages are (re) exposed in cron:

• config
• data
• worker

• Tests for the changes have been added (for bug fixes/features)

more of a refactor of existing code. the worker release test should catch any major errors.

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

NONE

[spencerschrock]

Not sure if Friday is the best time to merge something like this, but wanted to get it up for discussion for any changes.
@oliverchang @calebbrown fyi

[codecov]

Codecov Report

Merging #2317 (b1b9601) into main (900701c) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2317   +/-   ##
=======================================
  Coverage   40.61%   40.61%           
=======================================
  Files         112      112           
  Lines        8831     8831           
=======================================
  Hits         3587     3587           
  Misses       4984     4984           
  Partials      260      260           

[github-actions]

Integration tests success for
[22d12ad]
(https://github.com/ossf/scorecard/actions/runs/3160647323)

[spencerschrock]

876acb0 will cause metadata to be populated to the BigQuery results. Might be worth spinning this off in it's own PR

[github-actions]

Integration tests success for
[9389f67]
(https://github.com/ossf/scorecard/actions/runs/3184068892)

[github-actions]

Stale pull request message

[github-actions]

Integration tests success for
[19ec3d7]
(https://github.com/ossf/scorecard/actions/runs/3283512444)

[azeemshaikh38]

876acb0 will cause metadata to be populated to the BigQuery results. Might be worth spinning this off in it's own PR

Oh, so this was the bug all along. Yeah doing this in a separate PR SGTM.

[github-actions]

Integration tests success for
[580d634]
(https://github.com/ossf/scorecard/actions/runs/3283903143)

[github-actions]

Integration tests success for
[b1b9601]
(https://github.com/ossf/scorecard/actions/runs/3284833174)