Adds Github Action for Airflow compute test

[saurav-c]

Describe your changes and why you are making these changes

This PR adds a github action for running the Airflow compute tests once a week.

See https://github.com/aqueducthq/aqueduct/actions/runs/5043163833/jobs/9044608060 for an example run. Ignore the failures for now. These will all be filed as linear tasks that are beyond the scope of this task.

Related issue number (if any)

ENG 2599

Loom demo (if any)

Checklist before requesting a review

• I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
• I have performed a self-review of my code.
• I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
• If this is a new feature, I have added unit tests and integration tests.
• I have run the integration tests locally and they are passing.
• I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
• All features on the UI continue to work correctly.
• Added one of the following CI labels:
  • run_integration_test: Runs integration tests
  • skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

[saurav-c]

@reviewers Ignore these changes. This is just a mirror copy of the other file because github actions needs the file to already exist on main in order to test out an action.

[saurav-c]

I'll uncomment before merging

[saurav-c]

Actually will leave this commented out until I finish triaging all of the test case failures. Otherwise there will be a lot of noise generated. See https://linear.app/aqueducthq/issue/ENG-3009/investigate-airflow-integration-test-failures-and-triage-accordingly

[likawind]

Mostly looks good! A few high / low level questions:

• confirming you are adding back other tests in periodic-integration-tests.yml?
• should we publish pypi and start the test like how we do it in k8s / spark? What's the cadence of this test? If the same as k8s (and other periodic ones), we could add it as another job in periodic-integration-tests with publish-pypi as a dependency.
• I see a step copying over DAG template. I'm wondering if we should do this step in install_local.py -e rather than just in testing env?

[saurav-c]

@likawind

1. Yes, this is just for testing.
2. Sure, we can.
3. Sure, we can.

[saurav-c]

@likawind Addressed everything, see reason why I am leaving slack notif commented out for now.
The Airflow DAG is already copied as part of install_local so no changes needed there.

[likawind]

nice work!