ZEPPELIN-157: Adding Map Visualization for Zeppelin #152

Madhuka · 2015-07-12T19:01:30Z

Madhuka · 2015-07-12T19:02:44Z

Map Visualization for Zeppelin

vgmartinez · 2015-07-12T20:51:24Z

Hi @Madhuka....
when you try to resize the paragraph the map does not update to the new size ... I have seen that you have:
<leaflet width="100%" height="480px"></leaflet>
It is possible to do dynamic

thanks for you contributions

Madhuka · 2015-07-13T02:37:35Z

@vgmartinez

Thanks, Yes, I will make it dynamic

Madhuka · 2015-07-20T14:22:09Z

Add it as in tutorial, where it can be run from the clean environment

Loading sample GIS Data for tutorial

Markers looks like (static look)

Madhuka · 2015-07-20T14:46:43Z

Due to -len it is breaking

Madhuka · 2015-07-24T17:42:54Z

Loading sample data from internet

Madhuka · 2015-07-27T11:47:12Z

In Data validatorservices will return error status in boolean true or false. And also it will return error msg with explanation which data record is not validated under data schema.

Madhuka · 2015-08-03T16:06:28Z

Data validator message will contain {'error':true/false,'msg':'error msg'}.
Data schema can be set as config file.
Data validator is expose as service and it is extendable (factory model).

Here is current screen shoot of PR.

heruka-urgyen · 2015-08-04T03:28:58Z

zeppelin-web/src/components/data-validator/data-validator-factory.js

rowCheck returns a boolean, however you never use its output. is it a bug?

fixed,
with setError()

Not sure how it addresses the issue. Could you elaborate?

@felixcheung :
With updated code basicCheck[1] function return bool and and it is used in checkData()[2].

rowCheck() do not return any bool at now. When ever you found validator issue you can call new function setError();. It will handle it by making errorStatus to true. I will added this function for documentation too. So I will be easy for new developers who will writting new data valitors.

[1] https://github.com/Madhuka/incubator-zeppelin/blob/milestone05-map/zeppelin-web/src/components/data-validator/data-validator-factory.js#L59
[2] https://github.com/Madhuka/incubator-zeppelin/blob/milestone05-map/zeppelin-web/src/components/data-validator/data-validator-factory.js#L32

Madhuka · 2015-08-15T15:16:24Z

Test are added on data validation.

PR covers all the chart type data models in zeppelin.

eranwitkon · 2015-08-15T20:00:39Z

Hi,
I tried testing your PR (152) but I could not see the map icon on the output page.

Did I miss something?

Madhuka · 2015-08-16T10:36:34Z

I don't think you missed anything if you have built it correctly. Please clear the browser cache.

eranwitkon · 2015-08-16T12:13:36Z

Cool that did the trick.
Now that I can run the code I found this behavior :

Try the following code:
%sql
select Name from myMap
where Country = "${Country='United States'}"

The data output of this one have only one column - name but when I press
the map icon I get it to show the data as before. (other view respect the
select and present it as expected)
Eran

On Sun, Aug 16, 2015 at 1:36 PM madhuka [email protected] wrote:

I don't think you missed anything if you have built it correctly. Please
clear the browser cache.

—
Reply to this email directly or view it on GitHub
#152 (comment)
.

corneadoug · 2015-08-17T08:01:57Z

The Build runs multiple time on the CI

Madhuka · 2015-08-17T12:36:27Z

yes, Build has run multiple times and final build was successful according to log files. But there is still one failing check.

Madhuka · 2015-08-20T05:25:21Z

The Travis CI is only check first Build only, isn't it? First build test failure which is bind to CI and In next build It was fine. According to log second build (final build) says "BUILD SUCCESS" in the log.

[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 6.055 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 3.867 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 2.659 s]
[INFO] Zeppelin: Spark .................................... SUCCESS [ 35.690 s]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 0.699 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 0.316 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 0.315 s]
[INFO] Zeppelin: Hive interpreter ......................... SUCCESS [ 3.282 s]
[INFO] Zeppelin: Apache Geode interpreter ................. SUCCESS [ 7.297 s]
[INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 0.413 s]
[INFO] Zeppelin: Tajo interpreter ......................... SUCCESS [ 1.088 s]
[INFO] Zeppelin: Flink .................................... SUCCESS [ 4.955 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SUCCESS [ 0.680 s]
[INFO] Zeppelin: Lens interpreter ......................... SUCCESS [ 2.181 s]
[INFO] Zeppelin: Cassandra ................................ SUCCESS [ 6.649 s]
[INFO] Zeppelin: web Application .......................... SUCCESS [ 37.351 s]
[INFO] Zeppelin: Server ................................... SUCCESS [ 18.031 s]
[INFO] Zeppelin: Packaging distribution ................... SUCCESS [ 26.737 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:39 min
[INFO] Finished at: 2015-08-17T05:10:28+00:00
[INFO] Final Memory: 56M/360M
[INFO] ------------------------------------------------------------------------
travis_time:end:2f3eec76:start=1439788067730573331,finish=1439788229232148013,duration=161501574682
�[0K

Madhuka · 2015-08-21T18:16:53Z

CI Build is test with PR[1], It contain data validations (fully) and leaflet support with map and tutorial with squashing commits

[1] #238

eranwitkon · 2015-08-21T18:31:59Z

Did you see my comment from 5 days ago?

Now that I can run the code I found this behavior :

Try the following code:
%sql
select Name from myMap
where Country = "${Country='United States'}"

The data output of this one have only one column - name but when I press
the map icon I get it to show the data as before. (other view respect the
select and present it as expected)
Eran
בתאריך יום ו׳, 21 באוג׳ 2015, 21:17 מאת madhuka [email protected]:

CI Build is test with PR[1], It contain data validations (fully) and
leaflet support with map and tutorial with squashing commits

[1] #238 #238

—
Reply to this email directly or view it on GitHub
#152 (comment)
.

Madhuka · 2015-09-24T05:03:00Z

@bzz : ZeppelinIT which is failed in CI run fine in local environment. I have re-base (update) the PR. is there any more things to do for this PR?

T E S T S

Running org.apache.zeppelin.ZeppelinIT
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.075 sec - in org.apache.zeppelin.ZeppelinIT

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- scalatest-maven-plugin:1.0:test (test) @ zeppelin-server ---

friedhelmvictor · 2015-09-29T14:12:38Z

Map Visualization for Zeppelin would be a great addition!
I've tried the code and ran into several problems.

I don't think it's good to select the lat and lon statically like so:
https://github.com/Madhuka/incubator-zeppelin/blob/milestone05-map/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js#L866
It would be nice if they were detected automatically. Maybe if there are columns lat, latitude, lon, long or longitude, pick them?

The same goes for the key:
https://github.com/Madhuka/incubator-zeppelin/blob/milestone05-map/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js#L863
If a key exists twice, it obviously breaks. Key and message are the same though, but I may be interested in the same message at different locations.

There should be a settings drop down like for the other chart types where this can be configured. With the current version, that doesn't seem to work. (Only for me?)

Madhuka · 2015-09-30T04:34:14Z

@datagiraffe : thanks for the interest

Current zeppelin is written on order_of_key level. It is not written to detected automatically at the moment.
key is local variable and it is not twice. And all the variable name can have issue. That is the reason 'jshint' in zeppelin. It is not reason to break.

breaking happen in build due to CI binding test. you can find the points in this thread.

Madhuka · 2015-09-30T04:36:19Z

@bzz : I have build PR for data validator. At the same time I will create new PR for leaflet mapping as lot of users are interested.

randerzander · 2015-12-09T04:52:30Z

@Madhuka @bzz what's holding this one up? Is there something someone else can do to get this across the goal line?

Madhuka · 2015-12-09T05:02:47Z

@randerzander: Thanks for ping!
@bzz : I will create new PR for adding leaflet.

corneadoug · 2015-12-09T05:05:10Z

@randerzander @Madhuka
I'm reviewing #262 first, already took a look at it.
I'm planning on finishing review on those 3 PR this week.
But I think this will need a new PR without data validation and rebased at least.

Madhuka · 2015-12-09T06:15:49Z

@corneadoug : Thanks, get #262, it contain data validation.
So new PR will have only Map Vis.

corneadoug · 2015-12-15T09:49:04Z

@Madhuka Are you working on a new PR?
Tell me if you need some help

felixcheung · 2016-02-06T19:14:25Z

any update on this? @Madhuka this would be very cool to have

ankurmitujjain · 2016-02-09T11:27:37Z

Hi @Madhuka any update on this? really appreciate your effort for this.
Thanks

corneadoug · 2016-02-19T18:08:31Z

After discussion with @Madhuka, this should be rebased and finished by 1st week of March. (since he is busy)

There has been a new PR opened recently (#728), however I'd like Madhuka to finish his contribution for a few reasons:

This PR was part of Google Summer of Code, and is all thats left of all the work Madhuka did, so it would be nice for him to get his contribution
This PR has been around for quite some time, and I took way too long to properly review it

However if it is not done by then, we should get things moving and go with #728, @eddies fine with that?

eddies · 2016-02-19T23:53:21Z

@corneadoug absolutely. As I mentioned in #728, I only made my PR out of expediency because I didn't see any movement here. If @Madhuka will be revisiting this PR soon, that would be ideal--he definitely deserves the credit for all his work.

ankurmitujjain · 2016-04-08T11:42:40Z

@Madhuka Any update on this?

yanglei99 · 2016-04-27T03:35:24Z

When will this be in the main stream？It is a pity to see some great function missing while the solution is already available for more than half a year. Thanks.

felixcheung · 2016-04-27T07:42:37Z

Based on the earlier comment, are we moving to #728 then?

Madhuka · 2016-04-27T07:49:34Z

Referee #765

felixcheung · 2016-04-27T15:45:16Z

Great - will you have time to get that ready then?

_____________________________

jfuni · 2016-06-16T07:50:21Z

Nice work ! When we will be able to use these nice maps?

corneadoug · 2016-09-27T07:08:58Z

@Madhuka Let's close this one since you opened another one

Madhuka · 2016-10-03T04:45:31Z

@corneadoug As we have #765 We can close this, I am willing to finish that #765 as lot of people need this feature. Please let me know when I can continue to finish that from your end?

Madhuka force-pushed the milestone05-map branch 4 times, most recently from f4aaa1c to cd15db8 Compare July 23, 2015 09:13

heruka-urgyen reviewed Aug 4, 2015
View reviewed changes

Madhuka force-pushed the milestone05-map branch 4 times, most recently from e27b6b6 to 9a5ce12 Compare August 12, 2015 04:08

Madhuka force-pushed the milestone05-map branch from ff727b8 to d1a0ef0 Compare August 15, 2015 15:25

Madhuka force-pushed the milestone05-map branch from d1a0ef0 to 6cf4a61 Compare August 17, 2015 04:35

Madhuka force-pushed the milestone05-map branch from 6cf4a61 to 7c87b70 Compare August 20, 2015 19:23

Madhuka added 4 commits September 24, 2015 10:12

connecting services and d3 chart validation

21fad0c

completing validations with logs (demo)

2a8a3e6

adding test on data validator

e1855c0

removing console logs

58a1de1

Madhuka force-pushed the milestone05-map branch from cee3c6b to 58a1de1 Compare September 24, 2015 04:43

randerzander mentioned this pull request Jan 9, 2016

[ZEPPELIN-554] Streaming interpreter output to front-end #611

Closed

eddies mentioned this pull request Feb 19, 2016

Leaflet support, based on PR #152 #728

Closed

Madhuka mentioned this pull request Mar 7, 2016

[ZEPPELIN-157] Adding Map Visualization for Zeppelin #765

Closed

4 tasks

asfgit closed this in 3982184 Oct 3, 2016

ZEPPELIN-157: Adding Map Visualization for Zeppelin #152

ZEPPELIN-157: Adding Map Visualization for Zeppelin #152

Uh oh!

Conversation

Madhuka commented Jul 12, 2015

Uh oh!

Madhuka commented Jul 12, 2015

Uh oh!

vgmartinez commented Jul 12, 2015

Uh oh!

Madhuka commented Jul 13, 2015

Uh oh!

Madhuka commented Jul 20, 2015

Uh oh!

Madhuka commented Jul 20, 2015

Uh oh!

Madhuka commented Jul 24, 2015

Uh oh!

Madhuka commented Jul 27, 2015

Uh oh!

Madhuka commented Aug 3, 2015

Uh oh!

heruka-urgyen Aug 4, 2015

Choose a reason for hiding this comment

Uh oh!

Madhuka Aug 15, 2015

Choose a reason for hiding this comment

Uh oh!

heruka-urgyen Aug 15, 2015

Choose a reason for hiding this comment

Uh oh!

heruka-urgyen Aug 17, 2015

Choose a reason for hiding this comment

Uh oh!

Madhuka Aug 19, 2015

Choose a reason for hiding this comment

Uh oh!

Madhuka commented Aug 15, 2015

Uh oh!

eranwitkon commented Aug 15, 2015

Uh oh!

Madhuka commented Aug 16, 2015

Uh oh!

eranwitkon commented Aug 16, 2015

Uh oh!

corneadoug commented Aug 17, 2015

Uh oh!

Madhuka commented Aug 17, 2015

Uh oh!

Madhuka commented Aug 20, 2015

Uh oh!

Madhuka commented Aug 21, 2015

Uh oh!

eranwitkon commented Aug 21, 2015

Uh oh!

Madhuka commented Sep 24, 2015

T E S T S

Uh oh!

friedhelmvictor commented Sep 29, 2015

Uh oh!

Madhuka commented Sep 30, 2015

Uh oh!

Madhuka commented Sep 30, 2015

Uh oh!

randerzander commented Dec 9, 2015

Uh oh!

Madhuka commented Dec 9, 2015

Uh oh!

corneadoug commented Dec 9, 2015

Uh oh!

Madhuka commented Dec 9, 2015

Uh oh!

corneadoug commented Dec 15, 2015

Uh oh!

felixcheung commented Feb 6, 2016

Uh oh!

ankurmitujjain commented Feb 9, 2016

Uh oh!

corneadoug commented Feb 19, 2016

Uh oh!