Generate a Data Export Report and send Email - DAG failure

[ahafele]

When an export DAG (select OR send) has failed (and will not be retried) email should go out to [email protected] and [email protected] with:

Subject/body:

"Vendor Name: A data export has failed ..."

• Vendor name
• on Date
• Link to DAG failure
• Link to UI for downloading mrc or reports

For example, orafin dags generate emails.

No report needed for FULL DUMP

Acceptance Criteria

Generated by Zenhub AI.

• Scenario: Generate Data Export Report and send Email - DAG failure
• Given a DAG has failed (and will not be retried)
• When an email is sent to [email protected] and [email protected]
• Then the email should have the following content:
  1. Subject: "A data export has failed ..."
  2. Vendor name
  3. Date of failure
  4. Link to DAG failure
  5. Link to UI for downloading mrc or reports

[jgreben]

We should maybe have a story time about this...

As I see it, currently if the selection dag fails, it will not be retried for the same day, so we should always send out an email for a failed selection dag.

For the transmission dag failures there is already an email that is sent to sul-unicorn-devs each time there are failures in the file list. The failures do not get archived and will keep getting re-tried every day until someone clears them out, so there is no current state kept about the re-tries, and they will always be re-tried.

So maybe for this ticket we just need to send a notification about a failed selection dag, and maybe reformat the failed transmission email?

[ahafele]

Can you point me to an example of the transmission dag failure email?

[jgreben]

@ahafele

libsys-airflow/libsys_airflow/plugins/data_exports/email.py

Line 78 in e59f59c

def _failed_transmission_email_body(

[jgreben]

It will basically look something like this example:

Failed to Transmit Files for send_pod_records scheduled__2024-06-26T07:00:00+00:00

scheduled__2024-06-26T07%3A00%3A00%2B00%3A00

These files failed to transmit
data-export-files/some-vendor/marc-files/new/2024062801.mrc
data-export-files/some-vendor/marc-files/updates/2024062802.mrc
data-export-files/some-vendor/marc-files/updates/2024062803.mrc

[ahafele]

This looks good to me.