fix(glue): s3 path specified in --spark-event-logs-path needs to end with slash

[moomindani]

Issue # (if applicable)

Closes #29356.

Reason for this change

Currently S3 path specified in --spark-event-logs-path does not end with slash in case only bucket is provided but prefix is not provided. This parameter causes errors when viewing the event log through Spark UI / Spark history server.

Description of changes

Add trailing slash when it does not end with slash.

Description of how you validated changes

Completed unit test and integ test.

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

[scanlonp]

Hey @moomindani, this looks great! Have you confirmed that this works in Spark UI manually?

One thought: adding the trailing slash to the url seems like it could be used in more places than this. If so, we could add this functionality to the s3UrlForObject() function itself. Something along the lines of
public s3UrlForObject(key?: string, trailingSlash?: boolean): string {
in bucket.ts.

Then this line would be bucket.s3UrlForObject(props.prefix, true).

Lastly, this is changing current templates. Are we sure that

1. This is not breaking
2. if it is, that old templates were not working in the first place

We have some leeway since this is an alpha package, but we want to be clear on the PR if this is breaking.

[moomindani]

Hi @scanlonp,
Thanks for quick feedback and comment

Have you confirmed that this works in Spark UI manually?

Yes.

Lastly, this is changing current templates. Are we sure that

1. This is not breaking
2. if it is, that old templates were not working in the first place

Yes, actually it introduces change to the template but as you noticed the old template without trailing slash did not work at all in all the places.

Let me try to make changes on bucket.s3UrlForObject then update this PR.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Pull request has been modified.

[aws-cdk-automation]

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

[scanlonp]

Hey @moomindani, I fear I may have led you down the wrong path. I talked with some other team members, and my suggestion to change the function in the s3 package was the wrong call. We should keep this change scoped to glue. I feel pretty good approving your original PR if you would not mind going back to your first commit. Maybe adding a small comment on the purpose of the regex, if you would like.

Apologies again, but hope we can get this in smoothly from here!

[aws-cdk-automation]

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

[moomindani]

@scanlonp Thanks for your comment, I understand it.
I fixed the PR with reverting S3 side change. Can you take another look?

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Pull request has been modified.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 5cb1ef2
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).