Populate joblist table for gateway jobs

[jsperhac]

Description

Provide back-end configuration needed for job performance drilldown and data export for the Gateways realm. The schema and ETL changes enabled by this configuration are:

• Break out gateways jobs aggregation by day from that by month, quarter, year
• Populate day aggregation job_id_list, last_modified, and id columns for gateways jobs
• Create and populate jobfact_by_day_joblist table for gateways
• Incorporate Gateways in rawstatistics configuration

Query classes have been adapted from those that govern the Jobs realm to enable display of Gateway jobs data in the job viewer. (These should be subclassed from existing Query classes later, once the Jobs Query classes are appropriately generalized/refactored, see also PR #1473)

Note that bootstrapping and automatic testing is not included in the final version of this code, so that this schema and realm are not created by default in an installation. Therefore, component and unit test artifacts that pertain to multiple realms, including this one, have been backed out here. Instructions for the installation of this realm will be provided separately (see PR #1509).

Motivation and Context

This PR extends and enhances the back-end Gateways realm support. The configuration provided here makes schema changes that enable direct drilldown to performance records from gateways jobs; and enable batch export of raw data from the Gateways realm (where this access is permitted by ACLs).

This PR also fixes an aggregation bug (unreported, thus no link here) that was present in the original gateways back-end configuration (PR #1262).

Note that the configuration changes as provided here must be run manually via the ETL v2 pipeline to provide the schema changes (bootstrap; ingestion; aggregation).

Tests performed

• Developed/deployed in a local Docker; ran all etl v2 governing gateways schema bootstrapping, ingestion, aggregation using small reference dataset.
• Ran etl v2 on staging instance to verify/identify aggregation bug.
• Ran all tests (adapted test artifacts to accommodate these new configurations).

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project as found in the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[jtpalmer]

I merged in those variable substitution changes, here's what should change.

[jsperhac]

@jtpalmer, I have a question about the component tests for data export. I see test failures that seem to be expecting array elements with specific numeric keys.

Updates to the component tests for the new realm don't help, since there are gaps in the keys; see my most recent commit. Am I missing something, or do we need an accommodation similar to the RawStatisticsConfiguration getRawDataRealms() method?

[jtpalmer]

I think the test failures are due to more of my array_filter bugs. I created #1481 to fix that.

[jsperhac]

I merged in those variable substitution changes, here's what should change.

These have been addressed.