feat(batch-predictor): remove CPU limits from SparkApplication resource

[mbruner]

Description

Similar to kserve models, this PR removes CPU limits for batch prediction jobs. If limits are set, executors will experience significant throttling because batch jobs tend to consume all available CPU by its nature.

Modifications

• SparkApplication resource no longer has coreLimit set in driver and executor specs. This field is optional according to the specs and it should be safe to simply remove it.

Tests

Checklist

• Added PR label
• Added unit test, integration, and/or e2e tests
• Tested locally
• Updated documentation
• Update Swagger spec if the PR introduce API changes
• Regenerated Golang and Python client if the PR introduces API changes

Release Notes

Remove CPU Limits for batch prediction jobs

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.67%. Comparing base (0877a4f) to head (be46b79).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #615      +/-   ##
==========================================
- Coverage   60.68%   59.67%   -1.01%     
==========================================
  Files         277      277              
  Lines       22245    25999    +3754     
==========================================
+ Hits        13500    15516    +2016     
- Misses       7877     9616    +1739     
+ Partials      868      867       -1     

Flag	Coverage Δ
api-test	57.86% <ø> (-0.80%)	⬇️
sdk-test-3.10	75.51% <ø> (ø)
sdk-test-3.8	75.49% <ø> (ø)
sdk-test-3.9	75.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[deadlycoconuts]

LGTM! 🚀 Thanks for the changes!