Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up subscription #622

Merged
merged 7 commits into from
May 4, 2023
Merged

Conversation

jiangpengcheng
Copy link
Member

(If this PR fixes a github issue, please add Fixes #<xyz>.)

Fixes #594, #611

(or if this PR is one task of a github issue, please add Master Issue: #<xyz> to link to the master issue.)

Master Issue: #

Motivation

CleanupSubscription option doesn't work, and batch source doesn't cleanup intermediate topic either

Modifications

Use a k8s job to do cleanup works when required

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

Check the box below.

Need to update docs?

  • doc-required

    (If you need help on updating docs, create a doc issue)

  • no-need-doc

    (Please explain why)

  • doc

    (If this PR contains doc changes)

@jiangpengcheng jiangpengcheng requested review from nlu90, freeznet and a team as code owners April 11, 2023 07:11
@github-actions github-actions bot added the doc-required This pr needs a document label Apr 11, 2023
Copy link
Contributor

@nlu90 nlu90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the job is finished, can we delete the job from k8s?

@jiangpengcheng
Copy link
Member Author

After the job is finished, can we delete the job from k8s?

the job is deleted automatically after it is completed since TTLSecondsAfterFinished is set to 0

freeznet
freeznet previously approved these changes Apr 26, 2023
TTY: false,
}, scheme.ParameterCodec)

fmt.Printf("query is %v", res.URL().Query())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe change it to logger with the proper logging level?

Copy link
Member Author

@jiangpengcheng jiangpengcheng Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are for debugging purpose, removed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-added m/2023-05 type/feature Indicates new functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cleanupSubscription is not working
4 participants