Restricting Indices Access to Kibana Users

[cks333]

Hi,

I have created 3 kibana users namely admin, user1 and user2. while admin will have access to all indices in ES, user1 / user2 will have access to specific indices only.

I have whitelisted the IP address so that kibana can communicate with the ES without any issues and it is working fine. I can pump data in ES and read without any issues.

Now coming to Kibana UI, I can login as admin and see data belonging to all the indices. perfect. But when i login as user1 /user2, it is not restricting the data according to the indices. I have specified the indices using the user management console at _plugin/elasticfence/ for the users like

/[INDEX_NAME_1],/.kibana for user1
/[INDEX_NAME_2],/.kibana for user2

In this case, i expect user1 not to see the data in [INDEX_NAME_2] and user2 not to see the data in [INDEX_NAME_1]

Am i doing something wrong here ? or do i have to specify the index filter somewhere in the config file ?

Any inputs on this is greatly appreciated.

Thanks.

[ryanlutgen]

Whitelisting an IP means any request originating from that particular IP will bypass authentication checks. How I use it with my work is to not whitelist Kibana IPs, but have the authentication for a user inside the kibana config:
elasticsearch.url: "http://root:[email protected]:9200"

Perhaps there is a way to have the Kibana auth plugin do dynamic users, I am not familiar with the Kibana plugin ecosystem, I would suggest making a feature request there if the feature is needed.

[cks333]

Hi Ryan,

Thanks for your quick response. I have now modified it in the way you said in my kibana.yml config file. so rather than whitelisting, my kibana is connecting to ES through this root:rootpassword credential.

But still the user1 can still see [INDEX_NAME_2] and user2 can see [INDEX_NAME_1]

This gives me the doubt about the usage of indices in the user management console of elasticfence for different users really work ? I mean does it really restrict the indices view for various kibana users.

I created a user with indices such that the index does not exist in ES. Then tried to login, it logged in fine and could still see all the available indices in ES. When i delete the user, i could not login. Am i doing anything wrong here ?

Not sure whether it is related to this #2 where _msearch got implemented.

Thanks for your time and I salute for your inputs.

thanks.

[ryanlutgen]

Its possible that its related to that issue, but I am not 100% at this point. From what I can see, the code does some blanket checks for Kibana requests, so it might be that something there is not working as intended and the code thinking it is a generic Kibana request. The evaluation of permissions in particular is done in the following chunk of code: https://github.com/elasticfence/elasticsearch-http-user-auth/blob/5.1.2/src/main/java/org/elasticsearch/plugin/elasticfence/UserAuthenticator.java#L38

1. What version of ES and Kibana are you running?
2. Are you running queries via the Kibana dev tools against specific indexes when you encounter this issue?
3. Using root credentials, what is your output for the following URL: http://your.elasticsearch.hostname:9200/_httpuserauth?mode=list? You can censor your indexes if they are sensitively named.
4. It might be an issue with how the index names are evaluated relative to how the permission names are specified. i.e. If you are using wildcards, it might not be evaluating them as expected, if you can preserve as much as you can about the index names, be it by replacing sensitive strings with random characters and whatnot it might help solve this issue.

[cks333]

Hi,

Thanks for your inputs. My answers are below. Thanks for your time !

1. What version of ES and Kibana are you running?

ES 2.4.2 and Kibana 4.4.2

I had a look at the same java file mentioned by you in 2.4.2 but could not see any difference between the two.

2. Are you running queries via the Kibana dev tools against specific indexes when you encounter this issue?

I am accessing kibana using my internal kibana URL setup by me XXX.XX.XX.XX:5601

It displays the kibana-auth login page perfectly fine and the login process works fine too.

My ES contains two indices - customers and orders

Coming to users, I have 3 users namelu admin, user1 and user2. All created using the elasticfence user management console.

admin has the filter setting - /customers,/_*,/.kibana,/orders
user1 has the filter setting - /.kibana,/orders
user2 has the filter setting - /customers,/.kibana

Now all three users are able to login fine. But user1 can still see customers index in the kibana page (left side top) and able to click to see the data. Same things happens for user2 who can see the orders data.
This is what the problem is. I dont want user1 to be able to see customers in the kibana main page and same for user2 who should not be able to see orders. admin having access to all should be able to see all including the _ stuffs.

3. Using root credentials, what is your output for the following URL: http://your.elasticsearch.hostname:9200/_httpuserauth?mode=list? You can censor your indexes if they are sensitively named.

It is in dev stage only. You can see below.

[
{"username":"admin","password":"5203eef2913fbbb8ba2caf9fc88492eb163f725ba1fd08f0ddc5efd142110b33","indices":["/customers","/_*","/.kibana","/orders"],"created":"2017-01-24T08:28:31+0000"},
{"username":"user1","password":"5203eef2913fbbb8ba2caf9fc88492eb163f725ba1fd08f0ddc5efd142110b33","indices":["/.kibana","/orders"],"created":"2017-01-24T02:33:07+0000"},
{"username":"user2","password":"5203eef2913fbbb8ba2caf9fc88492eb163f725ba1fd08f0ddc5efd142110b33","indices":["/customers","/.kibana"],"created":"2017-01-24T02:32:32+0000"}
]

4. It might be an issue with how the index names are evaluated relative to how the permission names are specified. i.e. If you are using wildcards, it might not be evaluating them as expected, if you can preserve as much as you can about the index names, be it by replacing sensitive strings with random characters and whatnot it might help solve this issue.

As seen above, I am not using any wildcards here ! Thats why I am confused :-)

[ryanlutgen]

Are you running Kibana on the same box as any Elasticsearch node in your cluster?

I did a quick smoke test on my 5.1.1 branch and cluster using your users and index names and have noticed that I experience your issue if I run Kibana locally with an ES node. However, if I connect Kibana to a remote cluster, it works as intended: For user1, I cannot query customers via the dev tools, and I cannot define an index pattern for customers to view its mapping.

I was also able to dig up an old 2.3.5 configuration I had and ran a quick test locally and notice the same issue. I am able to add new index patterns and view their fields via Kibana.

The only real change that happened between 2.x and 5.x is how the plugin is registered with Elasticsearch and how the filters are put into Elasticsearch's pipeline. Because the authentication and index filter validation were not touched, I have reason to believe the same issue is present for 2.x.

Very odd, because it seems like Kibana is not going trough the filters this plugin sets up when it is running locally. I will look into this more as I am able.

[cks333]

Hi Ryan, Yes I am running kibana in the same machine as ES. I can configure kibana in another machine to give it a try as you said. will update you. Thanks Sent from mobile On 27 Jan 2017 1:10 a.m., "Ryan" <[email protected]> wrote: Are you running Kibana on the same box as any Elasticsearch node in your cluster? I did a quick smoke test using your users and index names and have noticed that I experience your issue if I run Kibana locally with an ES node. However, if I connect Kibana to a remote cluster, it works as intended. Very odd, because it seems like Kibana is not going trough the filters this plugin sets up when it is running locally. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#61 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AI7vAdt9zboMz1PksnTZputIyY749tbNks5rWNN4gaJpZM4LsIAr> .

[cks333]

I was so eager to test kibana in a different VM and then connecting to ES cluster. I just tried this with the same result. Not sure whether you used 4.4 or the newer 5.1 though. I tried 5.1 but complained saying incompatible with ES 2.4.2 and hence used 4.4

[ryanlutgen]

I think I might have reproduced what you were talking about.

In my Kibana config:
I first set it to use user1, created an index pattern for orders no problem, but am not able to create one for customers, as expected.
I then set it to use user2, restarted Kibana, and am able to create an index pattern for customers now, as expected. However, I still see the index pattern for orders and can access it: http://i.imgur.com/Hr2bcag.png

However, when I attempt to visualize data or refresh the fields, I get an authorization error, as expected. I can still see the fields for orders on user 2 though. Is this the issue you are talking about?

[cks333]

Hi Ryan,

You said "In my Kibana config: I first set it to use user 1" - a bit confusing for me.

So before i comment, I want to know whether you are adding anything in the kibana config file i.e kibana.yml file for all the users created using the elasticfence plugin as I am not adding anything to that config file except for the elasticsearch.url pointing to the ES cluster with the root password.
i.e elasticsearch.url: http://root:[email protected]:9200

If so, can you please give me the snippet of your kibana.yml file.

---

Another thing I wanna say is that you are trying to create an index pattern after creating users while I have the index pattern created before implementing this elasticfence. It was like we need something and i started pumping data from various apps to elastic and started using kibana dashboards. All good until now. But we have two different audiences here now and one should not see other. This is where i got stuck !

Thanks for your time and patience.

[ryanlutgen]

By "I first set it to user 1", I mean I set the elasticsearch.url to http://user1:[email protected]:9200, to ensure any requests I am making are indeed coming from that user in Kibana. When I do this, I am able to add an index pattern for orders, but not customers.

When I switch the elasticsearch.url to http://user2:[email protected]:9200, I can then add an index pattern for customers, but the orders one is still there. If I were on a new instance of Kibana, I would not be able to add an orders index pattern, but since one is already there I can see it.

Essentially what it looks like the issue is, is that there are pre-existing index patterns and one Kibana instance is meant to be shared between multiple users. What I've observed is that everything Kibana manages is within the /.kibana index, so any index patterns, or any other Kibana data is in that index and freely accessible. The reason I am not able to add a new index pattern depending on which user I have the URL set to is because when Kibana tries to access the mapping(s) for the pattern, it issues a request (with the supplied credentials in the URL) to the associated elasticsearch host. Elasticfence then does its thing and blocks the request if the user does not have access to the index.

I am not 100% sure this will work, but you could try to run two separate Kibana instances to keep everything separated. Create another copy of Kibana somewhere on the machine and alter the kibana.index and server.port in the second instance's kibana.yml. For example:

kibana.index: ".user2-kibana"
server.port: 5602

And specify the elasticsearch.url as http://user2:[email protected]:9200
Then alter user2's permissions to remove `.kibana' and add '.user2-kibana'
There are some hard-coded checks in the code that check for .kibana requests (https://github.com/elasticfence/elasticsearch-http-user-auth/blob/2.4.2/src/main/java/org/elasticsearch/plugin/elasticfence/UserAuthenticator.java#L59-L62 -> https://github.com/elasticfence/elasticsearch-http-user-auth/blob/2.4.2/src/main/java/org/elasticsearch/plugin/elasticfence/UserAuthenticator.java#L128-L148) so those might mess with this scheme, but its worth a shot I think. I will attempt this approach as well to verify when I have more time.

[ryanlutgen]

I had a bit of time to mess with my approach. All requests coming from the second instance of Kibana were blocked. The check for Kibana requests has a hard-coded .kibana index name in the code but I built a new Elasticfence package to allow Kibana index names to be configurable and for more than one.

If you want to try this approach to see if it fits your needs, I have a JAR here:
https://github.com/ryanlutgen/elasticsearch-http-user-auth/tree/Lutgen-2.4.2

A new config option needs to be added to the elasticsearch.yml file for this to work:
elasticfence.kibana.indices: [".kibana", ".kibana-user2"]

For reference, these were the options I had specified in the kibana configs:

elasticsearch.url: "http://user1:[email protected]:9200"

and

server.port: 5602
elasticsearch.url: "http://user2:[email protected]:9200"
kibana.index: ".kibana-user2"

[lmangani]

@ryanlutgen thanks for getting to the bottom of the problem, I am away for a few more days but it seems you got this better than I would have for sure! let me know if there's anything i can help with

[cks333]

@ryanlutgen sorry for the delayed reply. I was bit off and back now. I can see that you are using two different instances of kibana running in two different ports to make it work. There is a bottleneck i am noticing here. Actually we planned to dump data into elastic for every project we do and this means we will have different audiences for different project. Going by this way, we have to create multiple instances of kibana to make it work. Say 5 projects, 5 different audeinces and 5 different instances. Do you think it is elegant ? it can be used as a stop gap approach for now but in the longer run, it may be not be the elegant option. What do you think ?

Thanks for your time.

[ryanlutgen]

I think if you want to scope what users have access to what visualizations/dashboards, you will need seperate instances, since everything is stored in a .kibana index by default. From what I understand about Lucene, adding 5 or so more indices is not that big of a deal. In terms of whether or not multiple instances of elegant, the biggest cost will be maintaining those different Kibana instances and knowing which one is for what. Kibana seems to be meant for people who have access to all of the data, since even X-Pack Security does not have security controls around dashboards currently

If a user loads a Kibana dashboard that accesses data in an index that they are not authorized to view, they get an error that indicates the index does not exist. X-Pack Security does not currently provide a way to control which users can load which dashboards.

So I don't think it is as elegant as it could be, but for your case, I presently don't see another way around it.

Since each user would have its own .kibana instance, it is important to not give user1 access to user2's Kibana, since they would not only be able to see existing views, but could also make their own and thus give user2 access to their stuff. If user1 were to get access to user2's Kibana instance though, I am not sure how you could lock them out, since I am pretty sure the Kibana auth plugin works with Elasticfence users.

[cks333]

@ryanlutgen Thanks for your inputs. Appreciated. I will be pondering on this during the week and update you how it goes.