fix(glue): restore notifyDelayAfter across different job types

[humanzz]

Issue # (if applicable)

Related to #33356
Closes #33839

Reason for this change

address the regression of "notifyDelayAfter got removed from PySparkEtlJob props"

Description of changes

• move notifyDelayAfter prop to JobProps
• refactor to reduce code duplication
  • move jobRunQueuingEnabled prop to JobProps
  • introduce Job.setupJobResource utility static method to handle setting up the CfnJob for all job types
  • refactor all job types to use Job.setupJobResource while passing the overrides/customizations relevant to each job type
  • move extraJars, extraJarsFirst, and extraFiles props from the different spark job props into SparkJobProps

Describe any new or updated permissions being added

N/A

Description of how you validated changes

unit and integ tests

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

(This review is outdated)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.38%. Comparing base (e5f71db) to head (6dd66f7).
⚠️ Report is 668 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #33842   +/-   ##
=======================================
  Coverage   82.38%   82.38%           
=======================================
  Files         120      120           
  Lines        6955     6955           
  Branches     1173     1173           
=======================================
  Hits         5730     5730           
  Misses       1120     1120           
  Partials      105      105           

Flag	Coverage Δ
suite.unit	82.38% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
packages/aws-cdk	∅ <ø> (∅)
packages/aws-cdk-lib/core	82.38% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 6dd66f7
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[ozelalisen]

I notice this PR makes significant changes across multiple job types and includes high volume code refactoring. While the changes are well-organized, would it be possible to split this into smaller, more focused PRs? This would make the review process more manageable.

[humanzz]

Hi @ozelalisen

I had missed your comment, as it took quite sometime between submitting the PR your interaction.

Due to current commitments, I'm not sure I'd be able to manage breaking it down into smaller more focused PRs (and apologies for the large PR - but this was part of following up to a large breaking refactor already in glue alpha).

I also see that with time passing, there's now conflicts to resolve. I'll try to do that - but again with current commitments am not sure how much time I can dedicate to this.

[aws-cdk-automation]

This PR has been in the MERGE CONFLICTS state for 3 weeks, and looks abandoned. Note that PRs with failing linting check or builds are not reviewed, please ensure your build is passing

To prevent automatic closure:

• Resume work on the PR
• OR request an exemption by adding a comment containing 'Exemption Request' with justification e.x "Exemption Request: "
• OR request clarification by adding a comment containing 'Clarification Request' with a question e.x "Clarification Request: "

This PR will automatically close in 14 days if no action is taken.

[aws-cdk-automation]

This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.