MINOR: remove duplicate code from resetByDuration#3
Closed
bertber wants to merge 14 commits into
Closed
Conversation
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…apache#9669) Changes the `WorkerSourceTask` class to only call `SourceTask::stop` from the task thread when the task is actually stopped (via `Source:task::close` just before `WorkerTask::run` completes), and only if an attempt has been made to start the task (which will not be the case if it was created in the paused state and then shut down before being started). This prevents `SourceTask::stop` from being indirectly invoked on the herder's thread, which can have adverse effects if the task is unable to shut down promptly. Unit tests are tweaked where necessary to account for this new logic, which covers some edge cases mentioned in PR apache#5020 that were unaddressed up until now. The existing integration tests for blocking connectors are expanded to also include cases for blocking source and sink tasks. Full coverage of every source/sink task method is intentionally omitted from these expanded tests in order to avoid inflating test runtime (each one adds an extra 5 seconds at minimum) and because the tests that are added here were sufficient to reproduce the bug with source task shutdown. Author: Chris Egerton <chrise@confluent.io> Reviewers: Nigel Liang <nigel@nigelliang.com>, Tom Bentley <tbentley@redhat.com>, Randall Hauch <rhauch@gmail.com>
…ent (apache#9660) Implements KIP-680. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Matthias J. Sax <matthias@confluent.io>
Reviewer: Matthias J. Sax <matthias@confluent.io>
…9615) Part of KIP-663. Reviewers: Bruno Cadonna <bruno@confluent.io>, A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>
Reviewer: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
For MemberIdRequiredException, we would not print the exception at INFO with a full exception message since it may introduce more confusion that clearance. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Boyang Chen <boyang@confluent.io>
… mask the exit procedures (apache#9698) Normally the `EmbeddedConnectCluster` class masks the `Exit` procedures using within the Connect worker. This normally works great when a single instance of the embedded cluster is used. However, the `MirrorConnectorsIntegrationTest` uses two `EmbeddedConnectCluster` instances, and when the first one is stopped it would reset the (static) exit procedures, and any problems during shutdown of the second embedded Connect cluster would cause the worker to shut down the JVM running the tests. Instead, the `MirrorConnectorsIntegrationTest` class should mask the `Exit` procedures and instruct the `EmbeddedConnectClusters` instances (via the existing builder method) to not mask the procedures. Author: Randall Hauch <rhauch@gmail.com> Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
…n with connection close delay (apache#9678) Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
…he#9512) This PR adds support for generating snapshot for KIP-630. 1. Adds the interfaces `RawSnapshotWriter` and `RawSnapshotReader` and the implementations `FileRawSnapshotWriter` and `FileRawSnapshotReader` respectively. These interfaces and implementations are low level API for writing and reading snapshots. They are internal to the Raft implementation and are not exposed to the users of `RaftClient`. They operation at the `Record` level. These types are exposed to the `RaftClient` through the `ReplicatedLog` interface. 2. Adds a buffered snapshot writer: `SnapshotWriter<T>`. This type is a higher-level type and it is exposed through the `RaftClient` interface. A future PR will add the related `SnapshotReader<T>`, which will be used by the state machine to load a snapshot. Reviewers: Jason Gustafson <jason@confluent.io>
…equests/responses (apache#7409) Generated request/response classes have code to serialize/deserialize directly to `ByteBuffer` so the intermediate conversion to `Struct` can be skipped for them. We have recently completed the transition to generated request/response classes, so we can also remove the `Struct` based fallbacks. Additional noteworthy changes: * `AbstractRequest.parseRequest` has a more efficient computation of request size that relies on the received buffer instead of the parsed `Struct`. * Use `SendBuilder` for `AbstractRequest/Response` `toSend`, made the superclass implementation final and removed the overrides that are no longer necessary. * Removed request/response constructors that assume latest version as they are unsafe outside of tests. * Removed redundant version fields in requests/responses. * Removed unnecessary work in `OffsetFetchResponse`'s constructor when version >= 2. * Made `AbstractResponse.throttleTimeMs()` abstract. * Using `toSend` in `SaslClientAuthenticator` instead of `serialize`. * Various changes in Request/Response classes to make them more consistent and to rely on the Data classes as much as possible when it comes to their state. * Remove the version argument from `AbstractResponse.toString`. * Fix `getErrorResponse` for `ProduceRequest` and `DescribeClientQuotasRequest` to use `ApiError` which processes the error message sent back to the clients. This was uncovered by an accidental fix to a `RequestResponseTest` test (it was calling `AbstractResponse.toString` instead of `AbstractResponse.toString(short)`). Rely on existing protocol tests to ensure this refactoring does not change observed behavior (aside from improved performance). Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…t.open (apache#9710) Reviewers: Jun Rao <junrao@gmail.com>
…setByDuration Add a request that it will call resetToDatetime of description to resetByDuration of function.
bertber
force-pushed
the
bertber-patch-1
branch
from
December 8, 2020 16:10
37cad7a to
32adca3
Compare
bertber
changed the title
remove local variables from resetByDuration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There is a request for putting topicPartition and timestamp into topicPartitionsAndTimes's container from inputTopicPartitions.
And then take topicPartition and topicPartition's offset to call client's seek in the resetByDuration.
Beacuse I find some code in the resetToDatetime, so I would like to remove them from resetByDuration.
And then I would like to take client, inputTopicPartitions and timestamp to call resetToDatetime from resetByDuration.
where client is Consumer, inputTopicPartitions is container and timestamp is Duration's EpochMilli.
Finally, I found some error information in the Checks' Tab. I think these problems aren't relative to my pull request. As below information.
Log tail
…398 lines…
[2020-12-05T01ː06ː29.710Z] > Task ːstreamsːtest-utils:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːtest-utils:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0100:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0100:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0100:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0101:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0101:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0101:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0102:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0102:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0102:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0110:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0110:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-0110:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-10:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-10:testClasses
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-10:checkstyleTest
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-11:processTestResources NO-SOURCE
[2020-12-05T01ː06ː30.837Z] > Task ːstreamsːupgrade-system-tests-11:testClasses
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-11:checkstyleTest
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-20:processTestResources NO-SOURCE
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-20:testClasses
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-20:checkstyleTest
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-21:processTestResources NO-SOURCE
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-21:testClasses
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-21:checkstyleTest
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-22:processTestResources NO-SOURCE
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-22:testClasses
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-22:checkstyleTest
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-23:processTestResources NO-SOURCE
[2020-12-05T01ː06ː31.789Z] > Task ːstreamsːupgrade-system-tests-23:testClasses
[2020-12-05T01ː06ː31.790Z] > Task ːstreamsːupgrade-system-tests-23:checkstyleTest
[2020-12-05T01ː06ː31.790Z] > Task ːstreamsːupgrade-system-tests-24:processTestResources NO-SOURCE
[2020-12-05T01ː06ː31.790Z] > Task ːstreamsːupgrade-system-tests-24:testClasses
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-24:checkstyleTest
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-25:processTestResources NO-SOURCE
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-25:testClasses
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-25:checkstyleTest
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-26:processTestResources NO-SOURCE
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-26:testClasses
[2020-12-05T01ː06ː32.743Z] > Task ːstreamsːupgrade-system-tests-26:checkstyleTest
[2020-12-05T01ː06ː33.697Z] > Task ːconnectːspotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-0100:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-0101:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-0102:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-0110:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-10:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-11:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-20:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-21:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-22:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-23:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-24:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-25:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː34.655Z] > Task ːstreamsːupgrade-system-tests-26:spotbugsMain NO-SOURCE
[2020-12-05T01ː06ː40.856Z]
[2020-12-05T01ː06ː40.856Z] > Task :rat
[2020-12-05T01ː06ː40.856Z] Rat report: /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-9699/build/rat/rat-report.html
[2020-12-05T01ː07ː03.231Z]
[2020-12-05T01ː07ː03.231Z] > Task ːclientsːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːcoreːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːexamplesːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːgeneratorːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːjmh-benchmarksːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːlog4j-appenderːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːraftːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːstreamsːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːtoolsːspotbugsMain
[2020-12-05T01ː07ː03.231Z] > Task ːconnectːapi:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːbasic-auth-extension:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːfile:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːjson:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːmirror:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːmirror-client:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːruntime:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːconnectːtransforms:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːstreamsːexamples:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːstreamsːstreams-scala:spotbugsMain
[2020-12-05T01ː07ː04.183Z] > Task ːstreamsːtest-utils:spotbugsMain
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] FAILURE: Build failed with an exception.
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] * What went wrong:
[2020-12-05T01ː09ː09.077Z] Execution failed for task 'ːstreamsːcompileTestJava'.
[2020-12-05T01ː09ː09.077Z] > Compilation failed; see the compiler error output for details.
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] * Try:
[2020-12-05T01ː09ː09.077Z] Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] * Get more help at https://help.gradle.org
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
[2020-12-05T01ː09ː09.077Z] Use '--warning-mode all' to show the individual deprecation warnings.
[2020-12-05T01ː09ː09.077Z] See https://docs.gradle.org/6.7.1/userguide/command_line_interface.html#sec:command_line_warnings
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] BUILD FAILED in 9m 15s
[2020-12-05T01ː09ː09.077Z] 203 actionable tasks: 169 executed, 34 up-to-date
[2020-12-05T01ː09ː09.077Z]
[2020-12-05T01ː09ː09.077Z] See the profiling report at: file:///home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-9699/build/reports/profile/profile-2020-12-05-00-59-52.html
[2020-12-05T01ː09ː09.077Z] A fine-grained performance profile is available: use the --scan option.