From 8223eec8d5ed821857ea10c8eeddff9dbc747bfd Mon Sep 17 00:00:00 2001 From: Yash Mayya Date: Tue, 5 Sep 2023 22:45:19 +0530 Subject: [PATCH 1/2] KAFKA-14876: Add stopped state to Kafka Connect Administration docs section --- docs/connect.html | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/connect.html b/docs/connect.html index d324f4384b401..16e4a92034934 100644 --- a/docs/connect.html +++ b/docs/connect.html @@ -1080,6 +1080,10 @@

Kafka Connect

It's sometimes useful to temporarily stop the message processing of a connector. For example, if the remote system is undergoing maintenance, it would be preferable for source connectors to stop polling it for new data instead of filling logs with exception spam. For this use case, Connect offers a pause/resume API. While a source connector is paused, Connect will stop polling it for additional records. While a sink connector is paused, Connect will stop pushing new messages to it. The pause state is persistent, so even if you restart the cluster, the connector will not begin message processing again until the task has been resumed. Note that there may be a delay before all of a connector's tasks have transitioned to the PAUSED state since it may take time for them to finish whatever processing they were in the middle of when being paused. Additionally, failed tasks will not transition to the PAUSED state until they have been restarted.

+ +

+ In 3.5.0, Connect introduced a stop API that completely shuts down the tasks for a connector and deallocates any resources claimed by them. This is different from pausing a connector where tasks are left idling and any resources claimed by them are left allocated (which allows the connector to begin processing data quickly once it is resumed). Stopping a connector is more efficient from a resource usage standpoint than pausing it, but can cause it to take longer to begin processing data once resumed. Note that the offsets for a connector can be only modified via the offsets management endpoints if it is in the stopped state. +

From d642342fc56927bdee18d1c8f93a924acbb2c601 Mon Sep 17 00:00:00 2001 From: Yash Mayya Date: Tue, 5 Sep 2023 23:45:20 +0530 Subject: [PATCH 2/2] Add stopped state to list of possible states --- docs/connect.html | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/connect.html b/docs/connect.html index 16e4a92034934..6159570ceddd6 100644 --- a/docs/connect.html +++ b/docs/connect.html @@ -1065,6 +1065,7 @@

Kafka Connect
  • UNASSIGNED: The connector/task has not yet been assigned to a worker.
  • RUNNING: The connector/task is running.
  • PAUSED: The connector/task has been administratively paused.
    • +
  • STOPPED: The connector has been stopped. Note that this state is not applicable to tasks because the tasks for a stopped connector are shut down and won't be visible in the status API.
  • FAILED: The connector/task has failed (usually by raising an exception, which is reported in the status output).
  • RESTARTING: The connector/task is either actively restarting or is expected to restart soon