Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Glue streaming job update #23275

Merged
merged 11 commits into from
Mar 3, 2022
Merged

Glue streaming job update #23275

merged 11 commits into from
Mar 3, 2022
+121 −7

Conversation

gerrardcowburn
Copy link

@gerrardcowburn gerrardcowburn commented Feb 18, 2022 •
edited
Loading

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #16797

Output from acceptance testing:

$make testacc TESTS=TestAccGlueJob PKG=glue
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/glue/... -v -count 1 -parallel 20 -run='TestAccGlueJob'  -timeout 180m
=== RUN   TestAccGlueJob_basic
=== PAUSE TestAccGlueJob_basic
=== RUN   TestAccGlueJob_basicStreaming
=== PAUSE TestAccGlueJob_basicStreaming
=== RUN   TestAccGlueJob_command
=== PAUSE TestAccGlueJob_command
=== RUN   TestAccGlueJob_defaultArguments
=== PAUSE TestAccGlueJob_defaultArguments
=== RUN   TestAccGlueJob_nonOverridableArguments
=== PAUSE TestAccGlueJob_nonOverridableArguments
=== RUN   TestAccGlueJob_description
=== PAUSE TestAccGlueJob_description
=== RUN   TestAccGlueJob_glueVersion
=== PAUSE TestAccGlueJob_glueVersion
=== RUN   TestAccGlueJob_executionProperty
=== PAUSE TestAccGlueJob_executionProperty
=== RUN   TestAccGlueJob_maxRetries
=== PAUSE TestAccGlueJob_maxRetries
=== RUN   TestAccGlueJob_notificationProperty
=== PAUSE TestAccGlueJob_notificationProperty
=== RUN   TestAccGlueJob_tags
=== PAUSE TestAccGlueJob_tags
=== RUN   TestAccGlueJob_streamingTimeout
=== PAUSE TestAccGlueJob_streamingTimeout
=== RUN   TestAccGlueJob_timeout
=== PAUSE TestAccGlueJob_timeout
=== RUN   TestAccGlueJob_security
=== PAUSE TestAccGlueJob_security
=== RUN   TestAccGlueJob_workerType
=== PAUSE TestAccGlueJob_workerType
=== RUN   TestAccGlueJob_pythonShell
=== PAUSE TestAccGlueJob_pythonShell
=== RUN   TestAccGlueJob_maxCapacity
=== PAUSE TestAccGlueJob_maxCapacity
=== RUN   TestAccGlueJob_disappears
=== PAUSE TestAccGlueJob_disappears
=== CONT  TestAccGlueJob_basic
=== CONT  TestAccGlueJob_notificationProperty
=== CONT  TestAccGlueJob_executionProperty
=== CONT  TestAccGlueJob_timeout
=== CONT  TestAccGlueJob_glueVersion
=== CONT  TestAccGlueJob_streamingTimeout
=== CONT  TestAccGlueJob_description
=== CONT  TestAccGlueJob_security
=== CONT  TestAccGlueJob_tags
=== CONT  TestAccGlueJob_nonOverridableArguments
=== CONT  TestAccGlueJob_maxRetries
=== CONT  TestAccGlueJob_disappears
=== CONT  TestAccGlueJob_pythonShell
=== CONT  TestAccGlueJob_command
=== CONT  TestAccGlueJob_maxCapacity
=== CONT  TestAccGlueJob_workerType
=== CONT  TestAccGlueJob_basicStreaming
=== CONT  TestAccGlueJob_defaultArguments
--- PASS: TestAccGlueJob_basic (128.36s)
--- PASS: TestAccGlueJob_disappears (138.74s)
--- PASS: TestAccGlueJob_basicStreaming (149.65s)
--- PASS: TestAccGlueJob_description (235.26s)
--- PASS: TestAccGlueJob_defaultArguments (235.66s)
--- PASS: TestAccGlueJob_streamingTimeout (236.81s)
--- PASS: TestAccGlueJob_timeout (237.15s)
--- PASS: TestAccGlueJob_notificationProperty (244.28s)
--- PASS: TestAccGlueJob_maxRetries (245.41s)
--- PASS: TestAccGlueJob_maxCapacity (246.62s)
--- PASS: TestAccGlueJob_executionProperty (246.87s)
--- PASS: TestAccGlueJob_security (249.15s)
--- PASS: TestAccGlueJob_command (255.52s)
--- PASS: TestAccGlueJob_nonOverridableArguments (256.43s)
--- PASS: TestAccGlueJob_pythonShell (322.16s)
--- PASS: TestAccGlueJob_tags (329.92s)
--- PASS: TestAccGlueJob_workerType (334.76s)
--- PASS: TestAccGlueJob_glueVersion (341.94s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/glue       347.402s

CHANGELOG.md

* resource/aws_glue_job: Add support for `gluestreaming` Command Name by removing `Timeout` as a default parameter on Jobs, moving to optional, and adding acc tests ([#16797]https://github.com/hashicorp/terraform-provider-aws/issues/16797)

@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. documentation Introduces or discusses updates to documentation. provider Pertains to the provider itself, rather than any interaction with AWS. service/glue Issues and PRs that pertain to the glue service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. size/M Managed by automation to categorize the size of a PR. labels Feb 18, 2022
@github-actions
Copy link

Thank you for your contribution! 🚀

Please note that the CHANGELOG.md file contents are handled by the maintainers during merge. This is to prevent pull request merge conflicts, especially for contributions which may not be merged immediately. Please see the Contributing Guide for additional pull request review items.

Remove any changes to the CHANGELOG.md file and commit them in this pull request to prevent delays with reviewing and potentially merging this pull request.

@github-actions github-actions bot removed provider Pertains to the provider itself, rather than any interaction with AWS. documentation Introduces or discusses updates to documentation. labels Feb 18, 2022
github-actions[bot]
github-actions bot reviewed Feb 18, 2022
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @gerrardcowburn 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

drewmullen
drewmullen suggested changes Feb 21, 2022
View reviewed changes
Copy link
Collaborator

@drewmullen drewmullen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 small Q and please create your changelog entry. looks good!

internal/service/glue/job.go Show resolved Hide resolved
@gerrardcowburn gerrardcowburn changed the title [WIP] Glue streaming job update Glue streaming job update Feb 21, 2022
@gerrardcowburn gerrardcowburn marked this pull request as ready for review February 21, 2022 15:07
drewmullen
drewmullen suggested changes Feb 21, 2022
View reviewed changes
Copy link
Collaborator

@drewmullen drewmullen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update the documentation to include a reference to gluestreaming and a brief note about how timeout behaves differently:

@gerrardcowburn
Copy link
Author

gerrardcowburn commented Feb 21, 2022 via email

@github-actions github-actions bot added the documentation Introduces or discusses updates to documentation. label Feb 21, 2022
drewmullen
drewmullen approved these changes Feb 21, 2022
View reviewed changes
Copy link
Collaborator

@drewmullen drewmullen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@DrFaust92 DrFaust92 added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Feb 21, 2022
DrFaust92
DrFaust92 approved these changes Feb 21, 2022
View reviewed changes
Copy link
Collaborator

@DrFaust92 DrFaust92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ewbankkit
ewbankkit approved these changes Mar 3, 2022
View reviewed changes
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTS=TestAccGlueJob_ PKG=glue ACCTEST_PARALLELISM=5
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/glue/... -v -count 1 -parallel 5 -run='TestAccGlueJob_'  -timeout 180m
=== RUN   TestAccGlueJob_basic
=== PAUSE TestAccGlueJob_basic
=== RUN   TestAccGlueJob_basicStreaming
=== PAUSE TestAccGlueJob_basicStreaming
=== RUN   TestAccGlueJob_command
=== PAUSE TestAccGlueJob_command
=== RUN   TestAccGlueJob_defaultArguments
=== PAUSE TestAccGlueJob_defaultArguments
=== RUN   TestAccGlueJob_nonOverridableArguments
=== PAUSE TestAccGlueJob_nonOverridableArguments
=== RUN   TestAccGlueJob_description
=== PAUSE TestAccGlueJob_description
=== RUN   TestAccGlueJob_glueVersion
=== PAUSE TestAccGlueJob_glueVersion
=== RUN   TestAccGlueJob_executionProperty
=== PAUSE TestAccGlueJob_executionProperty
=== RUN   TestAccGlueJob_maxRetries
=== PAUSE TestAccGlueJob_maxRetries
=== RUN   TestAccGlueJob_notificationProperty
=== PAUSE TestAccGlueJob_notificationProperty
=== RUN   TestAccGlueJob_tags
=== PAUSE TestAccGlueJob_tags
=== RUN   TestAccGlueJob_streamingTimeout
=== PAUSE TestAccGlueJob_streamingTimeout
=== RUN   TestAccGlueJob_timeout
=== PAUSE TestAccGlueJob_timeout
=== RUN   TestAccGlueJob_security
=== PAUSE TestAccGlueJob_security
=== RUN   TestAccGlueJob_workerType
=== PAUSE TestAccGlueJob_workerType
=== RUN   TestAccGlueJob_pythonShell
=== PAUSE TestAccGlueJob_pythonShell
=== RUN   TestAccGlueJob_maxCapacity
=== PAUSE TestAccGlueJob_maxCapacity
=== RUN   TestAccGlueJob_disappears
=== PAUSE TestAccGlueJob_disappears
=== CONT  TestAccGlueJob_basic
=== CONT  TestAccGlueJob_notificationProperty
=== CONT  TestAccGlueJob_description
=== CONT  TestAccGlueJob_defaultArguments
=== CONT  TestAccGlueJob_executionProperty
--- PASS: TestAccGlueJob_basic (131.42s)
=== CONT  TestAccGlueJob_glueVersion
--- PASS: TestAccGlueJob_description (211.79s)
=== CONT  TestAccGlueJob_workerType
--- PASS: TestAccGlueJob_defaultArguments (298.67s)
=== CONT  TestAccGlueJob_disappears
--- PASS: TestAccGlueJob_executionProperty (303.57s)
=== CONT  TestAccGlueJob_maxCapacity
--- PASS: TestAccGlueJob_disappears (68.17s)
=== CONT  TestAccGlueJob_pythonShell
--- PASS: TestAccGlueJob_notificationProperty (419.51s)
=== CONT  TestAccGlueJob_command
--- PASS: TestAccGlueJob_glueVersion (303.85s)
=== CONT  TestAccGlueJob_basicStreaming
--- PASS: TestAccGlueJob_workerType (261.75s)
=== CONT  TestAccGlueJob_nonOverridableArguments
--- PASS: TestAccGlueJob_basicStreaming (73.06s)
=== CONT  TestAccGlueJob_maxRetries
--- PASS: TestAccGlueJob_command (141.38s)
=== CONT  TestAccGlueJob_timeout
--- PASS: TestAccGlueJob_pythonShell (212.33s)
=== CONT  TestAccGlueJob_security
--- PASS: TestAccGlueJob_maxCapacity (278.03s)
=== CONT  TestAccGlueJob_tags
--- PASS: TestAccGlueJob_nonOverridableArguments (140.00s)
=== CONT  TestAccGlueJob_streamingTimeout
--- PASS: TestAccGlueJob_maxRetries (145.56s)
--- PASS: TestAccGlueJob_timeout (164.41s)
--- PASS: TestAccGlueJob_security (191.38s)
--- PASS: TestAccGlueJob_streamingTimeout (254.13s)
--- PASS: TestAccGlueJob_tags (289.21s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/glue	874.488s

@ewbankkit
Copy link
Contributor

@gerrardcowburn Thanks for the contribution 🎉 👏.

@ewbankkit ewbankkit merged commit ae123f7 into hashicorp:main Mar 3, 2022
@github-actions github-actions bot added this to the v4.4.0 milestone Mar 3, 2022
@github-actions
Copy link

github-actions bot commented Mar 4, 2022

This functionality has been released in v4.4.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@DrFaust92 DrFaust92 DrFaust92 approved these changes

@drewmullen drewmullen drewmullen approved these changes

@ewbankkit ewbankkit ewbankkit approved these changes

Assignees
No one assigned
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/glue Issues and PRs that pertain to the glue service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v4.4.0
Development

Successfully merging this pull request may close these issues.

Glue Job resource does not fully support streaming job types
4 participants
@gerrardcowburn @ewbankkit @drewmullen @DrFaust92