Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple pulsar clusters use the same bookkeeper cluster ? #5935

Closed
SunDapeng1 opened this issue Dec 25, 2019 · 4 comments · Fixed by #5985
Closed

Multiple pulsar clusters use the same bookkeeper cluster ? #5935

SunDapeng1 opened this issue Dec 25, 2019 · 4 comments · Fixed by #5985
Labels
type/feature The PR added a new feature or issue requested a new feature

Comments

@SunDapeng1
Copy link

Can multiple pulsar clusters use the same bookkeeper cluster? If I had deployed the bk separately in advance and the metadata was stored under /ledgers, how would I configure the pulsar so that every pulsar recognizes /ledgers ?
(According to my test: pulsar initializes metadata to /pulsar, then the pulasr cluster can only recognize /pulsar/ledger to store metadata of bk)

@SunDapeng1 SunDapeng1 added the type/feature The PR added a new feature or issue requested a new feature label Dec 25, 2019
@jiazhai
Copy link
Member

jiazhai commented Dec 26, 2019

Thanks @SunDapeng1 for open this issue. currently class PulsarClusterMetadataSetup, which is called by command bin/pulsar initialize-cluster-metadata, will do all the metadata setup for Pulsar.
As discussed here with @murong00 ,
#5879 (comment)
we should make this configuration more easy.

But, Seems in current code, of PulsarClusterMetadataSetup, it is using /ledgers for bookies by default? In your deployment, are you using the localzk as: zkhost:2181/pulsar ? maybe just change it back to zkhost:2181 would solve this issue?

@SunDapeng1
Copy link
Author

Thank you for your reply. Maybe I didn't describe it clearly. What I want to express is that if the bk metadata is in /ledgers, then when the first pulsar is deployed, the metadata can be initialized to /. At this time, the first pulsar cluster can normally identify /ledgers Bk cluster, but how can I deploy a second pulsar cluster, so that the second pulsar cluster also uses the same bk cluster /ledgers?

@jiazhai
Copy link
Member

jiazhai commented Dec 26, 2019

@SunDapeng1 , as we discussed, the main issue is make bookie service (/ledgers) configurable. This may involves some change like PulsarClusterMetadataSetup InternalConfigurationData.
at present, ZkLedgersRootPath that passed into InternalConfigurationData is some kind of hard coded.

@SunDapeng1
Copy link
Author

@jiazhai How is this issue going?

sijie pushed a commit that referenced this issue Apr 21, 2020
…#5985)

### Motivation

Fixes #5935

Support multi pulsar clusters to use the specified bookkeeper cluster by pointing `BookKeeper Client` to the zookeeper connection string of bookkeeper cluster.

### Modifications

- Add a config `bookkeeperServiceUri` to discover bookkeeper cluster metadata store.
- Use metadata service uri to init bookkeeper client.
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this issue Aug 24, 2020
…apache#5985)

### Motivation

Fixes apache#5935

Support multi pulsar clusters to use the specified bookkeeper cluster by pointing `BookKeeper Client` to the zookeeper connection string of bookkeeper cluster.

### Modifications

- Add a config `bookkeeperServiceUri` to discover bookkeeper cluster metadata store.
- Use metadata service uri to init bookkeeper client.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature The PR added a new feature or issue requested a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants