Add pkg gluon-geolocator from google summer of code 2017

[2tata]

This package makes able to receive positions base on surrounding WIFIs.

the geolocator program takes surrounding BSSIDs and send this to the openwifi project.
As an answer the router will get a position base on the above sent BSSIDs. This will be done
in a defined interval, which can be set by site.conf or by configMode.

Following site.conf parameters a definable for community's:

• autolocation enable flag. Can be 1 or 0 means enabled or disabled. Default is disabled.
• interval in minutes, means how often should the geolocator refresh its position. Default is every 12 hours.

Here an example site.conf part:

geolocator = {
                autolocation = true,
                interval = 1000,
             },

This package is over a year tested in the Nordwest Freifunk network.

This is one of 4 other packages there should follow in the next few weeks.
2 of the other packages are responsible for the configuration over the configMode.
https://blog.freifunk.net/2017/07/26/geolocator-software-defined-gps-second-evaluation/

[rubo77]

Is it possible for the user to select, that he/she doesn't want to be discovered?

Can you add some screenshots of the admin interface?

[2tata]

This pkg contains only the geolocator. The admin interface comes in the 3 other above mentioned pkgs.
Users can set share_location [0] to discover there positions.

As well users have to be enable the service by setting auto_location [1] otherwise the geolocator will be disabled.

[0] https://github.com/freifunk-gluon/gluon/blob/master/package/gluon-node-info/files/etc/config/gluon-node-info
[1] https://github.com/freifunk-gluon/gluon/pull/1201/files#diff-ca9821c5dfe71d5edd884409cceef57bR30

[rubo77]

Why is the text below the dropdown with such a big padding?

[2tata]

On 08/21/17 07:56, Ruben Barkow wrote: Why is the text below the dropdown with such a big padding?

I dont know which text do you mean? vg Tarek

[2tata]

@NeoRaider now every requested changes should be done. :-)

[rotanid]

status update from dev meeting:

• needs rebase for check_site
• awaiting review

[neocturne]

I have started some code cleanup, unfortunately I can't test it at the moment, as http://openwifi.su/api/v1/bssids/... only returns "502 Bad Gateway"...

Edit: It's back online, I'll try to get this finished today.

[neocturne]

Btw, is the quality of the openwifi.su data high enough so that the geolocator package will return even marginally useful data? I just tested this with a node in Leipzig, and I got a location in Braunschweig (the scan returned 46 BSSIDs on 2.4GHz, so there should be sufficient data to avoid getting the location that wrong?)

Anyways, I've just pushed the first part of my rework (which has turned out to be mostly a rewrite...) as freifunk-gluon/packages@53a659a, a reusable Lua library for geolocation. In addition to code cleanup, I changed the following:

• use ubus to get the interface list from netifd instead of using UCI (nicer interface, and we don't depend on the optional ifname UCI option)
• only query one interface per radio

[2tata]

Btw, is the quality of the openwifi.su data high enough so that the geolocator package will return even marginally useful data? I just tested this with a node in Leipzig, and I got a location in Braunschweig (the scan returned 46 BSSIDs on 2.4GHz, so there should be sufficient data to avoid getting the location that wrong?)

You need to add the mesh BSSIDs to the blacklist as i wrote above. The openwifi.su backend needs atleast one known WiFi network in the database. For places where nearly no wifi is known in the database, you can just use the openWLANmap wardriving App for Android [0]. The wardriving App will may replace in the coming months because this is in the GSoC18 as a Project for Freifunk. I started work on a new backend witch will also use Mozilla Location service in addition to the own database. This was my last Google Summer of Code project last year. I just got the idea if I change the the request url to a siteconf variable local communitis without v6 can use my GSoC17 project as a MLS proxy [1]. So I will make the request url variable.

[0] http://openwifi.su/owlmapatandroid.php?lang=en
[1] https://github.com/openwifi-su/sgps-core

Anyways, I've just pushed the first part of my rework (which has turned out to be mostly a rewrite...) as freifunk-gluon/packages@53a659a, a reusable Lua library for geolocation. In addition to code cleanup,

I explicitly change it from a C-Lib into the geolocator corn to avoid airtime wasting by multiple access on scanning. For example before the geolocator, the hoodselector and an autolocator (witch place the location in the uci config) where using the C-Lib that means scanns twice for the same result. The geolocator includes the idea to update location position in an intervall where other programs can access. So maybe we should keep the lib out to force using the uci interface.

I changed the following:

use ubus to get the interface list from netifd instead of using UCI (nicer interface, and we don't depend on the optional ifname UCI option)

ubus was also my plan but I drop this idea while I was discussing with people from the Mozilla Location Service project and Zeeshan Ali who already develop a software calls gps-share to provide positions via ubus by using GPS HW in the background.

[neocturne]

Btw, is the quality of the openwifi.su data high enough so that the geolocator package will return even marginally useful data? I just tested this with a node in Leipzig, and I got a location in Braunschweig (the scan returned 46 BSSIDs on 2.4GHz, so there should be sufficient data to avoid getting the location that wrong?)

You need to add the mesh BSSIDs to the blacklist as i wrote above. The openwifi.su backend needs atleast one known WiFi network in the database.

Okay, after blacklisting the FFHL IBSS cell ID I get 0 results now (why is the FFHL IBSS located in Braunschweig though...?). I think I will just change it to ignore all non-AP WLANs, there should be no need to blacklist anything by default then.

I explicitly change it from a C-Lib into the geolocator corn to avoid airtime wasting by multiple access on scanning. For example before the geolocator, the hoodselector and an autolocator (witch place the location in the uci config) where using the C-Lib that means scanns twice for the same result. The geolocator includes the idea to update location position in an intervall where other programs can access. So maybe we should keep the lib out to force using the uci interface.

Did you even look at my commit? I built a small Lua library from the code of this PR, it has nothing to do with a C library.

ubus was also my plan but I drop this idea while I was discussing with people from the Mozilla Location Service project and Zeeshan Ali who already develop a software calls gps-share to provide positions via ubus by using GPS HW in the background.

I only use ubus to get the interface list from netifd, instead of getting them from /etc/config/wireless via UCI.

I have a few more ideas regarding this PR. WIll probably replace my Lua library again with one implementing the standard GLS API provided by Google and Mozilla (I assume I will implement it in C, to be able to HTTP POST). To use openwifi.su, they would need to provide a GLS endpoint (or a simple proxy would need to be written); I don't really want to support their non-standard API after all.

Furthermore, I'm unsure about what to do with the determined locations. Your PR currently writes it to the flash, which is not acceptable. I think it would be a nice solution to use a UCI state file in /var/state for it, so the results are only kept in RAM (a command to explicitly commit this state to flash can still be provided).

[2tata]

Btw, is the quality of the openwifi.su data high enough so that the geolocator package will return even marginally useful data? I just tested this with a node in Leipzig, and I got a location in Braunschweig (the scan returned 46 BSSIDs on 2.4GHz, so there should be sufficient data to avoid getting the location that wrong?)

You need to add the mesh BSSIDs to the blacklist as i wrote above. The openwifi.su backend needs atleast one known WiFi network in the database.

Okay, after blacklisting the FFHL IBSS cell ID I get 0 results now (why is the FFHL IBSS located in Braunschweig though...?).

Because some ware has use the ffhl IBSS cell ID there or just setup a FFHL router and its Cell ID got catch by the war driving app.

I think I will just change it to ignore all non-AP WLANs, there should be no need to blacklist anything by default then.

Ah yes this is a better idea. The new war driving app, which is planed as a GSoC18 project, should also ignore non-AP WLANs and even mobile hotspots.

I explicitly change it from a C-Lib into the geolocator corn to avoid airtime wasting by multiple access on scanning. For example before the geolocator, the hoodselector and an autolocator (witch place the location in the uci config) where using the C-Lib that means scanns twice for the same result. The geolocator includes the idea to update location position in an intervall where other programs can access. So maybe we should keep the lib out to force using the uci interface.

Did you even look at my commit? I built a small Lua library from the code of this PR, it has nothing to do with a C library.

I think you miss understood me. I look at your commit and you created a lua lib witch can be called from other lua programs. Above I explained that each program witch use your lib will also call a wifi scan witch will waist airtime, in case if more than one program will request its position in an interval. This problem was happens before the geolocator exists by using a C-lib witch did a similar work like your lua lib on our FFNW routers. That is why I developed the geolocator and why I use the uci interface for providing actual coordinates. I think you lib is not a good way because of the airtime issue or do you think this is not relevant?

ubus was also my plan but I drop this idea while I was discussing with people from the Mozilla Location Service project and Zeeshan Ali who already develop a software calls gps-share to provide positions via ubus by using GPS HW in the background.

I only use ubus to get the interface list from netifd, instead of getting them from /etc/config/wireless via UCI.

Ah pardon, it was late yesterday ;)

I have a few more ideas regarding this PR. WIll probably replace my Lua library again with one implementing the standard GLS API provided by Google and Mozilla (I assume I will implement it in C, to be able to HTTP POST). To use openwifi.su, they would need to provide a GLS endpoint (or a simple proxy would need to be written); I don't really want to support their non-standard API after all.

The problem by the GLS services by google was form me it is not opensource and I think you will need a API key

The key request was also a problem for GLS service by Mozilla

Both above projects did only provide HTTPS request witch means we have to put SSL on gluon. (The main reason why I didn´t use MLS)

Furthermore, I'm unsure about what to do with the determined locations. Your PR currently writes it to the flash, which is not acceptable. I think it would be a nice solution to use a UCI state file in /var/state for it, so the results are only kept in RAM (a command to explicitly commit this state to flash can still be provided).

Yes I was also not sure about that. Another Idea which I got was to write a position only for once into the flash and refresh this position only if a tolerance value is overreach. Holding a position only in the RAM is not really optimal on reboots for programs for example the hoodselector. Because that can mean in the worse case the hoodselector will change its hood 2 times after reboot. But at this point I am also not sure..

[neocturne]

I think you miss understood me. I look at your commit and you created a lua lib witch can be called from other lua programs. Above I explained that each program witch use your lib will also call a wifi scan witch will waist airtime, in case if more than one program will request its position in an interval. This problem was happens before the geolocator exists by using a C-lib witch did a similar work like your lua lib on our FFNW routers. That is why I developed the geolocator and why I use the uci interface for providing actual coordinates. I think you lib is not a good way because of the airtime issue or do you think this is not relevant?

The library is just code refactoring. There will be only a single program using the library in Gluon.

Regarding the HTTPS issue: I'm not sure how to proceed here. The reasoning of Mozilla etc. not to support HTTP makes sense; for privacy reasons the scan data should not be sent over unencrypted channels.

[2tata]

I think you miss understood me. I look at your commit and you created a lua lib witch can be called from other lua programs. Above I explained that each program witch use your lib will also call a wifi scan witch will waist airtime, in case if more than one program will request its position in an interval. This problem was happens before the geolocator exists by using a C-lib witch did a similar work like your lua lib on our FFNW routers. That is why I developed the geolocator and why I use the uci interface for providing actual coordinates. I think you lib is not a good way because of the airtime issue or do you think this is not relevant?

The library is just code refactoring. There will be only a single program using the library in Gluon.

Ah ok.

Regarding the HTTPS issue: I'm not sure how to proceed here. The reasoning of Mozilla etc. not to support HTTP makes sense; for privacy reasons the scan data should not be sent over unencrypted channels.

I am also not sure if MLS provides v6 as you probably read in the IRC

regarding to the privacy reasons, but what about all the respondd datas from gluon there available over public v6 ?

like:
http://[fd74:fdaa:9dc4:6800:12fe:edff:fe7b:2836]/cgi-bin/interfaces
http://[2a03:2260:1001:6800:12fe:edff:fe7b:2836]/cgi-bin/nodeinfo
etc. ?

[2tata]

@NeoRaider I have change the geolocator to only request with Master mode driven scanned devices.
And I applyed your ubus request. Still the question how to handle the flash writing issue..

[rubo77]

You could Set-up a Proxy service somewhere that accepts HTTP requests and Redirects them to https requests to the Mozilla Service.

The requests to that self created service should use some other kind of encryption though.

[2tata]

https://github.com/openwifi-su/sgps-core
An another project of my is already an MLS proxy. But without security.

But I dont like MLS that much... they keep all data closed and even for the service you need to ask them for a API key and you have to explain them why do you need it. I didint got a Key when I have told them I have build an MLS SSL Proxy... So I have not so good experience with there service.

[ngobibibnbe]

goodmornig i am a student in ENSP in yaounde; our teacher talked us about the work of gluon and Gsoc for the evolution of freifunk which really intrest me and for now i want to integrate the geolocalisation packages in the sources codes of openwrt and i have many difficulties to find the road of the work that i should do to achieve this integration.i am needing advices please .tks

[mweinelt]

Hey @2tata,

we just talked this through and decided that with its privacy impact we are not going to merge this package.

hexa-