Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-topic subscription requires tenant admin access #12423

Closed
senax opened this issue Oct 19, 2021 · 1 comment · Fixed by #12600
Closed

Multi-topic subscription requires tenant admin access #12423

senax opened this issue Oct 19, 2021 · 1 comment · Fixed by #12600
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@senax
Copy link

senax commented Oct 19, 2021

Describe the bug
Since 2.8.1, using multi-topic subscriptions requires admin access; not just namespace-consume. This was working fine in 2.8.0.

To Reproduce
Steps to reproduce the behaviour:

  1. Run a cluster with JWT authentication enabled.
  2. Create a 'test_user' token. pulsar tokens create --private-key /data/certs/jwt-private.key --subject test_user > /data/certs/jwt-test_user.token
  3. Create a test tenant and namespace. pulsar-admin tenants create test; pulsar-admin namespaces create test/test
  4. Grant produce, consume to test_user. pulsar-admin namespaces grant-permissions --actions produce,consume --role test_user test/test
  5. Produce some messages to similar topics. pulsar-client produce test/test/topic-1 -m msg1 pulsar-client produce test/test/topic-2 -m msg2
  6. Consume messages from both topics with regex, using the test_user token. pulsar-client --auth-params file:///data/certs/jwt-test_user.token consume --regex 'test/test/topic-.*' -s all -p Earliest -n 0
  7. Consume will fail.
  8. Grant admin rights for 'test_user' to the tenant.pulsar-admin tenants update -r test_user test
  9. Now consume works.pulsar-client --auth-params file:///data/certs/jwt-test_user.token consume --regex 'test/test/topic-.*' -s all -n 0

Redacted Output of failure

2021-10-06 12:49:57,503 [pulsar-client-io-1-1] WARN  org.apache.pulsar.client.impl.ClientCnx - [id: 0x73dd935d, L:/172.30.0.33:48806 - R:broker1-] Received error from server: Proxy Client is not authorized to GetTopicsOfNamespace
2021-10-06 12:49:57,507 [pulsar-client-io-1-1] WARN  org.apache.pulsar.client.impl.PulsarClientImpl - [test/test] Failed to get topics under namespace
2021-10-06 12:49:57,508 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - Error while consuming messages
2021-10-06 12:49:57,508 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - Proxy Client is not authorized to GetTopicsOfNamespace
org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to GetTopicsOfNamespace
	at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:979) ~[org.apache.pulsar-pulsar-client-api-2.8.1.jar:2.8.1]

And in broker.log:
2021-10-06 12:49:57,463 [pulsar-io-4-5] INFO  org.apache.pulsar.broker.service.ServerCnx - New connection from /172.30.0.33:48806
2021-10-06 12:49:57,499 [pulsar-io-4-5] WARN  org.apache.pulsar.broker.service.ServerCnx - Role test_user is not authorized to perform operation GET_TOPICS on namespace test/test
2021-10-06 12:49:57,500 [pulsar-io-4-5] WARN  org.apache.pulsar.broker.service.ServerCnx - [/172.30.0.33:48806] Proxy Client is not authorized to GetTopicsOfNamespace with role test_user on namespace test/test
2021-10-06 12:49:57,867 [pulsar-io-4-5] INFO  org.apache.pulsar.broker.service.ServerCnx - Closed connection from /172.30.0.33:48806

Expected behavior
Running the same as above with version 2.8.0 , OR with admin rights granted on tenant works just fine.

2021-10-06 11:58:58,178 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x1e54fb1d, L:/172.30.0.33:53416 - R:broker2]] Connected to server
2021-10-06 11:58:58,189 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://test/test/topic-1][all] Subscribing to topic on cnx [id: 0xadad51dc, L:/172.30.0.33:50930 - R:broker1], consumerId 1
2021-10-06 11:58:58,240 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://test/test/topic-1][all] Subscribed to topic on broker1 -- consumer: 1
2021-10-06 11:58:58,241 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.MultiTopicsConsumerImpl - [MultiTopicsConsumer-e38eb] [all] Success subscribe new topic persistent://test/test/topic-1 in topics consumer, partitions: 0, allTopicPartitionsNumber: 2
2021-10-06 11:58:58,891 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://test/test/topic-2][all] Subscribing to topic on cnx [id: 0x1e54fb1d, L:/172.30.0.33:53416 - R:broker2], consumerId 0
2021-10-06 11:58:59,256 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://test/test/topic-2][all] Subscribed to topic on broker2 -- consumer: 0
2021-10-06 11:58:59,267 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.MultiTopicsConsumerImpl - [MultiTopicsConsumer-e38eb] [all] Created topics consumer with 2 sub-consumers
2021-10-06 11:58:59,268 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.MultiTopicsConsumerImpl - [MultiTopicsConsumer-e38eb] [all] Success subscribe new topic persistent://test/test/topic-2 in topics consumer, partitions: 0, allTopicPartitionsNumber: 2
...

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • CentOS7

Additional context
Add any other context about the problem here.

@MarvinCai
Copy link
Contributor

it's introduced in this patch: #11172
multi-topic subscribe will need to list topic under namesapce first to determine what topics to subscribe which is a namesapce operation and will need admin permission afterward.
namespace operation will always need admin permission expect for packages management.

hangc0276 pushed a commit that referenced this issue Nov 5, 2021
Fixes #12423

### Motivation
Regex subscription requires to get the topics list of given namespace with GetTopicsOfNamespace request, but this request requires tenant admin permission which will block the regex consumers who only have consume permission.

### Modifications
This PR added the consume permission check for GetTopicsOfNamespace, which allows consumers get the topics list with consume permission.
eolivelli pushed a commit that referenced this issue Nov 9, 2021
Fixes #12423

### Motivation
Regex subscription requires to get the topics list of given namespace with GetTopicsOfNamespace request, but this request requires tenant admin permission which will block the regex consumers who only have consume permission.

### Modifications
This PR added the consume permission check for GetTopicsOfNamespace, which allows consumers get the topics list with consume permission.

(cherry picked from commit 7e078aa)
codelipenghui pushed a commit that referenced this issue Nov 18, 2021
Fixes #12423

### Motivation
Regex subscription requires to get the topics list of given namespace with GetTopicsOfNamespace request, but this request requires tenant admin permission which will block the regex consumers who only have consume permission.

### Modifications
This PR added the consume permission check for GetTopicsOfNamespace, which allows consumers get the topics list with consume permission.

(cherry picked from commit 7e078aa)
eolivelli pushed a commit to eolivelli/pulsar that referenced this issue Nov 29, 2021
Fixes apache#12423

### Motivation
Regex subscription requires to get the topics list of given namespace with GetTopicsOfNamespace request, but this request requires tenant admin permission which will block the regex consumers who only have consume permission.

### Modifications
This PR added the consume permission check for GetTopicsOfNamespace, which allows consumers get the topics list with consume permission.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants