Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix segfault if assignor state is NULL, #4381

Merged
merged 9 commits into from
Sep 26, 2023
Merged

Conversation

emasab
Copy link
Contributor

@emasab emasab commented Aug 2, 2023

e.g. a destroy happens before the first assignment. Only affects the cooperative-sticky assignor.

fixes #4312

@mensfeld
Copy link

mensfeld commented Aug 2, 2023

🙏 was it a good report? 🤣

@emasab emasab marked this pull request as ready for review August 2, 2023 17:41
@milindl milindl self-requested a review August 3, 2023 11:04
@milindl
Copy link
Contributor

milindl commented Aug 9, 2023

Thanks for the PR @emasab , I think we should add a test for this before merging it

first rebalance with cooperative-stick assignor.
e.g. a destroy happens before the first assignment.
Only affects the cooperative-sticky assignor.

fixes #4312
@emasab emasab force-pushed the dev_fix_segfault_assignor_state branch from 81651aa to fce470f Compare August 11, 2023 15:43
@cla-assistant
Copy link

cla-assistant bot commented Aug 11, 2023

CLA assistant check
All committers have signed the CLA.

@emasab
Copy link
Contributor Author

emasab commented Aug 11, 2023

@milindl Sure this needed to be completed. I've added it.

Copy link
Contributor

@milindl milindl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this issue

@emasab emasab merged commit 73cf87c into master Sep 26, 2023
@emasab emasab deleted the dev_fix_segfault_assignor_state branch September 26, 2023 12:48
milindl added a commit that referenced this pull request Sep 27, 2023
commit bee8d6a80b81a8f31c823750e3e504537cd954ac
Author: Milind L <[email protected]>
Date:   Wed Sep 27 11:40:35 2023 +0530

    Squashed commit of the following:

    commit 73cf87c
    Author: Emanuele Sabellico <[email protected]>
    Date:   Tue Sep 26 14:48:57 2023 +0200

        Fix segfault if assignor state is NULL, (#4381)

        e.g. a destroy happens before the first assignment.
        Only affects the cooperative-sticky assignor

    commit 1cb8009
    Author: Emanuele Sabellico <[email protected]>
    Date:   Thu Sep 21 10:04:07 2023 +0200

        Add destroy call when leader epoch is stale (#4429)

        and partition is in state active

    commit 49f180a
    Author: Pranav Rathi <[email protected]>
    Date:   Tue Aug 15 23:47:28 2023 +0530

        Added new environment variable to tests - TESTS_SKIP_BEFORE (#4317)

    commit ff1aaf5
    Author: Pranav Rathi <[email protected]>
    Date:   Mon Aug 14 18:31:48 2023 +0530

        Fix ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0. (#4346)

commit f6f0a62
Author: Milind L <[email protected]>
Date:   Mon Sep 25 17:00:05 2023 +0530

    Fix case where broker RPC might not contain authorized operations

commit 63e0748
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:14:47 2023 +0530

    Fix possible initialization issue

commit 7205354
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:02:37 2023 +0530

    Differentiate between 0 and unrequested authorized ops

commit adc0c4f
Author: Milind L <[email protected]>
Date:   Mon Sep 25 09:35:36 2023 +0530

    Avoid malloc(0)

commit 4257dc9
Author: Milind L <[email protected]>
Date:   Fri Sep 22 10:35:33 2023 +0530

    Fix CI issues

commit 0e05dea
Author: Milind L <[email protected]>
Date:   Fri Sep 22 07:45:53 2023 +0530

    Change TopicCollection create method name

commit 74d98dc
Author: Milind L <[email protected]>
Date:   Thu Sep 21 15:02:51 2023 +0530

    Fix warning and add rack to cluster nodes

commit fd14d24
Author: Milind L <[email protected]>
Date:   Fri Sep 15 12:38:56 2023 +0530

    Retain topic ordering for DescribeTopics

commit d90bb98
Author: Milind L <[email protected]>
Date:   Thu Sep 14 12:41:10 2023 +0530

    Change rack_id to rack in Node public API

commit 0233c0f
Author: Milind L <[email protected]>
Date:   Wed Sep 6 15:50:37 2023 +0530

    Fix typo in result type

commit e6d4d63
Author: Milind L <[email protected]>
Date:   Mon Sep 4 17:26:44 2023 +0530

    Add brokers_sorted to internal metadata

commit 2c7f5e1
Author: Milind L <[email protected]>
Date:   Mon Sep 4 16:53:20 2023 +0530

    Address comments: AuthorizedOperations_parse uses array

commit 9a6ef10
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:47:41 2023 +0530

    Address comments: cosmetic and copyright changes

commit beccb0c
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:10:54 2023 +0530

    Address comments: Cosmetic changes

commit 6239372
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:59:20 2023 +0530

    Address comment and RD_IF_FREE fixes

commit bdd8a4b
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:55:26 2023 +0530

    Fix style and refactor MetadataRequest into op/cb

commit 53cd983
Author: Milind L <[email protected]>
Date:   Thu Aug 31 17:47:54 2023 +0530

    Address DescribeCluster/DescribeConsumerGroup changes, fix memleak

commit 59ab5bb
Author: Milind L <[email protected]>
Date:   Thu Aug 31 13:57:51 2023 +0530

    Address comments for DescribeTopics API

commit 85f7f91
Author: Milind L <[email protected]>
Date:   Fri Aug 11 15:08:51 2023 +0530

    Get rid of stray conflict marker

commit bd7f5a7
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:27:39 2023 +0530

    Style fixes

commit ef3f480
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:26:48 2023 +0530

    Fix examples

commit 0ed7250
Merge: 61840ab 07262c4
Author: Milind L <[email protected]>
Date:   Wed Aug 9 14:08:09 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 61840ab
Author: Milind L <[email protected]>
Date:   Wed Aug 9 13:53:33 2023 +0530

    Address review comments: Part 2

commit 45780f8
Author: Milind L <[email protected]>
Date:   Wed Aug 9 10:13:23 2023 +0530

    Address review comments: Part 1

commit 0bc97b3
Author: Milind L <[email protected]>
Date:   Wed Aug 2 11:31:37 2023 +0530

    Break into functions, update Supported KIPS

commit 6f8aa0d
Author: Milind L <[email protected]>
Date:   Wed Aug 2 10:40:36 2023 +0530

    Fix 0080 after addressing comments

commit 8d8a473
Author: Milind L <[email protected]>
Date:   Tue Aug 1 19:59:39 2023 +0530

    Address review comments: Refactor MetadataRequest into two parts

commit 28615cd
Author: Milind L <[email protected]>
Date:   Tue Aug 1 18:10:46 2023 +0530

    Address review comments - Part 2

commit e01ad41
Author: Milind L <[email protected]>
Date:   Tue Aug 1 15:09:46 2023 +0530

    Address review comments - Part 1

commit 3215a90
Author: Milind L <[email protected]>
Date:   Tue Aug 1 09:50:59 2023 +0530

    Fixed changelog

commit 4a4f503
Author: Milind L <[email protected]>
Date:   Mon Jul 31 20:41:57 2023 +0530

    Remove fanout from DescribeTopics

commit 9787d6c
Merge: 8c2b4d4 c07a335
Author: Milind L <[email protected]>
Date:   Mon Jul 31 14:47:43 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 8c2b4d4
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:44:24 2023 +0530

    Fix cluster_id allocation not on tmpabuf

commit 4a4d9df
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:29:26 2023 +0530

    Revert to macro to fix msvc compilation

commit 7fff6eb
Merge: 3a9a340 aa50e52
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:18:39 2023 +0530

    Remove warnings and add CHANGELOG

commit 3a9a340
Author: Milind L <[email protected]>
Date:   Sat Jun 24 15:36:22 2023 +0530

    Change API for common stuff

commit 6ddbaba
Merge: f497307 e52aa3b
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:24:15 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit f497307
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:22:39 2023 +0530

    Fix tests and change API to be similar to existing ones

commit 3fec50a
Author: Milind L <[email protected]>
Date:   Thu Jun 15 09:52:33 2023 +0530

    Fix style and docs issues

commit 99a368d
Merge: 7c15746 865cf60
Author: Ruchir Jain <[email protected]>
Date:   Wed Jun 14 18:52:29 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 7c15746
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:49:32 2023 +0530

    bug fix: request NULL, force_racks in MetadataReq

commit 88707a0
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:11:47 2023 +0530

    bug fixes and include auth ops false default

commit 060d694
Merge: 0e16f98 966b63d
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 15:54:10 2023 +0530

    Merge 'master' of librdkafka into dev_kip430_cp

commit 0e16f98
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 10:04:40 2023 +0530

    style-fix

commit b9fea3c
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 09:33:57 2023 +0530

    ClusterDescription destroy having double free bug removed

commit 7d512bf
Merge: 830d9b7 2c6dd0a
Author: Ruchir Jain <[email protected]>
Date:   Tue Apr 11 09:32:39 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 830d9b7
Author: jainruchir <[email protected]>
Date:   Thu Mar 30 14:36:25 2023 +0530

    bug fix for adminoptions

commit 0bfbf67
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 16:05:22 2023 +0530

    dev_KIP430 branch merged

commit c28152a
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 15:05:36 2023 +0530

    changes to describeCG tests

commit d2c902b
Author: jainruchir <[email protected]>
Date:   Tue Feb 21 15:55:50 2023 +0530

    added test for include auth ops in 0081 initial commit

commit 3a79ed6
Author: jainruchir <[email protected]>
Date:   Fri Feb 17 14:25:37 2023 +0530

    KIP-430 initial changes without checking tests

commit d0e2156
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 10:01:08 2023 +0530

    bug fix when include authorized ops is 0 desc Topics/Cluster

commit a542f4d
Author: jainruchir <[email protected]>
Date:   Wed Mar 15 10:25:40 2023 +0530

    integration tests for describecluster

commit 8df6f62
Author: jainruchir <[email protected]>
Date:   Tue Mar 14 14:04:44 2023 +0530

    describe topics and cluster

commit 069b8e8
Author: jainruchir <[email protected]>
Date:   Fri Feb 24 14:32:49 2023 +0530

    include authorized topic/cluster ops initial draft
milindl added a commit that referenced this pull request Sep 27, 2023
commit bee8d6a80b81a8f31c823750e3e504537cd954ac
Author: Milind L <[email protected]>
Date:   Wed Sep 27 11:40:35 2023 +0530

    Squashed commit of the following:

    commit 73cf87c
    Author: Emanuele Sabellico <[email protected]>
    Date:   Tue Sep 26 14:48:57 2023 +0200

        Fix segfault if assignor state is NULL, (#4381)

        e.g. a destroy happens before the first assignment.
        Only affects the cooperative-sticky assignor

    commit 1cb8009
    Author: Emanuele Sabellico <[email protected]>
    Date:   Thu Sep 21 10:04:07 2023 +0200

        Add destroy call when leader epoch is stale (#4429)

        and partition is in state active

    commit 49f180a
    Author: Pranav Rathi <[email protected]>
    Date:   Tue Aug 15 23:47:28 2023 +0530

        Added new environment variable to tests - TESTS_SKIP_BEFORE (#4317)

    commit ff1aaf5
    Author: Pranav Rathi <[email protected]>
    Date:   Mon Aug 14 18:31:48 2023 +0530

        Fix ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0. (#4346)

commit f6f0a62
Author: Milind L <[email protected]>
Date:   Mon Sep 25 17:00:05 2023 +0530

    Fix case where broker RPC might not contain authorized operations

commit 63e0748
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:14:47 2023 +0530

    Fix possible initialization issue

commit 7205354
Author: Milind L <[email protected]>
Date:   Mon Sep 25 16:02:37 2023 +0530

    Differentiate between 0 and unrequested authorized ops

commit adc0c4f
Author: Milind L <[email protected]>
Date:   Mon Sep 25 09:35:36 2023 +0530

    Avoid malloc(0)

commit 4257dc9
Author: Milind L <[email protected]>
Date:   Fri Sep 22 10:35:33 2023 +0530

    Fix CI issues

commit 0e05dea
Author: Milind L <[email protected]>
Date:   Fri Sep 22 07:45:53 2023 +0530

    Change TopicCollection create method name

commit 74d98dc
Author: Milind L <[email protected]>
Date:   Thu Sep 21 15:02:51 2023 +0530

    Fix warning and add rack to cluster nodes

commit fd14d24
Author: Milind L <[email protected]>
Date:   Fri Sep 15 12:38:56 2023 +0530

    Retain topic ordering for DescribeTopics

commit d90bb98
Author: Milind L <[email protected]>
Date:   Thu Sep 14 12:41:10 2023 +0530

    Change rack_id to rack in Node public API

commit 0233c0f
Author: Milind L <[email protected]>
Date:   Wed Sep 6 15:50:37 2023 +0530

    Fix typo in result type

commit e6d4d63
Author: Milind L <[email protected]>
Date:   Mon Sep 4 17:26:44 2023 +0530

    Add brokers_sorted to internal metadata

commit 2c7f5e1
Author: Milind L <[email protected]>
Date:   Mon Sep 4 16:53:20 2023 +0530

    Address comments: AuthorizedOperations_parse uses array

commit 9a6ef10
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:47:41 2023 +0530

    Address comments: cosmetic and copyright changes

commit beccb0c
Author: Milind L <[email protected]>
Date:   Mon Sep 4 15:10:54 2023 +0530

    Address comments: Cosmetic changes

commit 6239372
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:59:20 2023 +0530

    Address comment and RD_IF_FREE fixes

commit bdd8a4b
Author: Milind L <[email protected]>
Date:   Mon Sep 4 14:55:26 2023 +0530

    Fix style and refactor MetadataRequest into op/cb

commit 53cd983
Author: Milind L <[email protected]>
Date:   Thu Aug 31 17:47:54 2023 +0530

    Address DescribeCluster/DescribeConsumerGroup changes, fix memleak

commit 59ab5bb
Author: Milind L <[email protected]>
Date:   Thu Aug 31 13:57:51 2023 +0530

    Address comments for DescribeTopics API

commit 85f7f91
Author: Milind L <[email protected]>
Date:   Fri Aug 11 15:08:51 2023 +0530

    Get rid of stray conflict marker

commit bd7f5a7
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:27:39 2023 +0530

    Style fixes

commit ef3f480
Author: Milind L <[email protected]>
Date:   Wed Aug 9 19:26:48 2023 +0530

    Fix examples

commit 0ed7250
Merge: 61840ab 07262c4
Author: Milind L <[email protected]>
Date:   Wed Aug 9 14:08:09 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 61840ab
Author: Milind L <[email protected]>
Date:   Wed Aug 9 13:53:33 2023 +0530

    Address review comments: Part 2

commit 45780f8
Author: Milind L <[email protected]>
Date:   Wed Aug 9 10:13:23 2023 +0530

    Address review comments: Part 1

commit 0bc97b3
Author: Milind L <[email protected]>
Date:   Wed Aug 2 11:31:37 2023 +0530

    Break into functions, update Supported KIPS

commit 6f8aa0d
Author: Milind L <[email protected]>
Date:   Wed Aug 2 10:40:36 2023 +0530

    Fix 0080 after addressing comments

commit 8d8a473
Author: Milind L <[email protected]>
Date:   Tue Aug 1 19:59:39 2023 +0530

    Address review comments: Refactor MetadataRequest into two parts

commit 28615cd
Author: Milind L <[email protected]>
Date:   Tue Aug 1 18:10:46 2023 +0530

    Address review comments - Part 2

commit e01ad41
Author: Milind L <[email protected]>
Date:   Tue Aug 1 15:09:46 2023 +0530

    Address review comments - Part 1

commit 3215a90
Author: Milind L <[email protected]>
Date:   Tue Aug 1 09:50:59 2023 +0530

    Fixed changelog

commit 4a4f503
Author: Milind L <[email protected]>
Date:   Mon Jul 31 20:41:57 2023 +0530

    Remove fanout from DescribeTopics

commit 9787d6c
Merge: 8c2b4d4 c07a335
Author: Milind L <[email protected]>
Date:   Mon Jul 31 14:47:43 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 8c2b4d4
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:44:24 2023 +0530

    Fix cluster_id allocation not on tmpabuf

commit 4a4d9df
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:29:26 2023 +0530

    Revert to macro to fix msvc compilation

commit 7fff6eb
Merge: 3a9a340 aa50e52
Author: Milind L <[email protected]>
Date:   Mon Jun 26 11:18:39 2023 +0530

    Remove warnings and add CHANGELOG

commit 3a9a340
Author: Milind L <[email protected]>
Date:   Sat Jun 24 15:36:22 2023 +0530

    Change API for common stuff

commit 6ddbaba
Merge: f497307 e52aa3b
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:24:15 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit f497307
Author: Milind L <[email protected]>
Date:   Fri Jun 23 12:22:39 2023 +0530

    Fix tests and change API to be similar to existing ones

commit 3fec50a
Author: Milind L <[email protected]>
Date:   Thu Jun 15 09:52:33 2023 +0530

    Fix style and docs issues

commit 99a368d
Merge: 7c15746 865cf60
Author: Ruchir Jain <[email protected]>
Date:   Wed Jun 14 18:52:29 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 7c15746
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:49:32 2023 +0530

    bug fix: request NULL, force_racks in MetadataReq

commit 88707a0
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 18:11:47 2023 +0530

    bug fixes and include auth ops false default

commit 060d694
Merge: 0e16f98 966b63d
Author: jainruchir <[email protected]>
Date:   Wed Jun 14 15:54:10 2023 +0530

    Merge 'master' of librdkafka into dev_kip430_cp

commit 0e16f98
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 10:04:40 2023 +0530

    style-fix

commit b9fea3c
Author: jainruchir <[email protected]>
Date:   Tue Apr 11 09:33:57 2023 +0530

    ClusterDescription destroy having double free bug removed

commit 7d512bf
Merge: 830d9b7 2c6dd0a
Author: Ruchir Jain <[email protected]>
Date:   Tue Apr 11 09:32:39 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 830d9b7
Author: jainruchir <[email protected]>
Date:   Thu Mar 30 14:36:25 2023 +0530

    bug fix for adminoptions

commit 0bfbf67
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 16:05:22 2023 +0530

    dev_KIP430 branch merged

commit c28152a
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 15:05:36 2023 +0530

    changes to describeCG tests

commit d2c902b
Author: jainruchir <[email protected]>
Date:   Tue Feb 21 15:55:50 2023 +0530

    added test for include auth ops in 0081 initial commit

commit 3a79ed6
Author: jainruchir <[email protected]>
Date:   Fri Feb 17 14:25:37 2023 +0530

    KIP-430 initial changes without checking tests

commit d0e2156
Author: jainruchir <[email protected]>
Date:   Fri Mar 17 10:01:08 2023 +0530

    bug fix when include authorized ops is 0 desc Topics/Cluster

commit a542f4d
Author: jainruchir <[email protected]>
Date:   Wed Mar 15 10:25:40 2023 +0530

    integration tests for describecluster

commit 8df6f62
Author: jainruchir <[email protected]>
Date:   Tue Mar 14 14:04:44 2023 +0530

    describe topics and cluster

commit 069b8e8
Author: jainruchir <[email protected]>
Date:   Fri Feb 24 14:32:49 2023 +0530

    include authorized topic/cluster ops initial draft
axelandersson added a commit to axelandersson/librdkafka that referenced this pull request Oct 5, 2023
* upstream/master:
  librdkafka v2.3.0 (confluentinc#4455)
  Fix for idempotent producer fatal errors, triggered after a possibly persisted message state (confluentinc#4438)
  Move can_q_contain_fetched_msgs inside q_serve (confluentinc#4431)
  [KIP-580] Exponential Backoff with Mock Broker Changes to Automate Testing. (confluentinc#4422)
  Update only the mklove version of OpenSSL to 3.0.11 (confluentinc#4454)
  Permanent errors during offset validation should be retried (confluentinc#4447)
  Increased flexver request size for Metadata request to include topic_id size (confluentinc#4453)
  Fix loop of OffsetForLeaderEpoch requests on quick leader changes (confluentinc#4433)
  Fix for stored offsets not being committed if they lacked the leader epoch (confluentinc#4442)
  Add leader epoch to control messages (confluentinc#4434)
  Refactored tmpabuf and fixed an insufficient buffer allocation (confluentinc#4449)
  Work around KIP-700 restrictions for DescribeCluster [KIP-430]
  [admin] KIP-430: Add authorized operations to describe API
  Fix segfault if assignor state is NULL, (confluentinc#4381)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

assignor_state is NULL initially and it's only set after assignment but its value isn't checked before calling
3 participants