Skip to content

Conversation

@cloverhearts
Copy link
Member

@cloverhearts cloverhearts commented Oct 17, 2016

What is this PR for?

Currently, if you have a Notebook large number of,
There is a sharp decrease in performance of the Code editor.
The number and Paragraph creation of the Notebook does not have a relationship.
We are should always use the Code editor of the same performance.

I had to print a Notebook list only if there is a request.
There was quite a lot of performance improvements.

What type of PR is it?

Improvement

Todos

  • - change Notebook list render logic

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-1559
https://issues.apache.org/jira/browse/ZEPPELIN-1513

How should this be tested?

  1. Create a notebook over 1000. (tested 10,000)
    I've used script on this. (important! please, backup for notebooks directory.)
#/bin/bash

idx="0"

while [ $idx -lt 1000 ] # notebooks count 1000
do
  mkdir $idx
  echo """
{
    \"paragraphs\": [
    {
        \"text\": \"fsdkljfowiejfowejfoiwefiowejfiojlkdsjfklsdjflkjsdflkjwlkjefewf\",
            \"dateUpdated\": \"Oct 13, 2016 5:02:44 PM\",
            \"config\": {
                \"colWidth\": 12.0,
            \"graph\": {
                \"mode\": \"table\",
            \"height\": 300.0,
            \"optionOpen\": false,
            \"keys\": [],
            \"values\": [],
            \"groups\": [],
            \"scatter\": {},
            \"map\": {
                \"baseMapType\": \"Streets\",
            \"isOnline\": true,
            \"pinCols\": []
    }
    },
        \"enabled\": true,
        \"editorMode\": \"ace/mode/scala\"
    },
        \"settings\": {
            \"params\": {},
        \"forms\": {}
    },
        \"apps\": [],
        \"jobName\": \"paragraph_1476345687682_-1459653828\",
        \"id\": \"20161013-170127_68316618\",
        \"dateCreated\": \"Oct 13, 2016 5:01:27 PM\",
        \"status\": \"READY\",
        \"progressUpdateIntervalMs\": 500
    }
    ],
        \"name\": \""${idx}"\",
        \"id\": \""${idx}"\",
        \"angularObjects\": {
            \"2BZSC9D3G:shared_process\": [],
        \"2BXGWF5TF:shared_process\": [],
        \"2BYKTADMA:shared_process\": [],
        \"2BYFT9HTZ:shared_process\": [],
        \"2BXDTJMED:shared_process\": [],
        \"2BZHPZ6NS:shared_process\": [],
        \"2BYKRWYB3:shared_process\": [],
        \"2BZH2UAT4:shared_process\": [],
        \"2BZ38J35G:shared_process\": [],
        \"2BXJ2X464:shared_process\": [],
        \"2BYQ57ED9:shared_process\": [],
        \"2BXDSA2SN:shared_process\": [],
        \"2BWZCTD2B:shared_process\": [],
        \"2BWT81MNU:shared_process\": [],
        \"2BWA7ZTRD:shared_process\": [],
        \"2BXW6X5KN:shared_process\": [],
        \"2BVWDQPXH:shared_process\": [],
        \"2BZGAF8KG:shared_process\": []
},
    \"config\": {},
    \"info\": {}
    }
  """ > $idx/note.json

  idx=$[$idx+1]
done

  1. zeppelin restart or refresh notebook.
  2. Try coding in any notebook. (fast)

Screenshots (if appropriate)

before ( 5000 notebooks)

optimization_before_codeditor

after ( 5000 notebooks)

optimization_for_codeeidtor

Questions:

  • Does the licenses files need update? no
  • Is there breaking changes for older versions? no
  • Does this needs documentation? no

@cloverhearts
Copy link
Member Author

@Leemoonsoo
Copy link
Member

Tested and confirmed angular watcher reduced from 5xxx -> 3xx in case of 1k notes. Great work!

<ul class="nav navbar-nav" ng-if="ticket">
<li class="dropdown" dropdown>
<a href="#" class="dropdown-toggle" dropdown-toggle>Notebook <span class="caret"></span></a>
<li id="notebook-list-button" class="dropdown notebook-list-dropdown" dropdown>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like that 'id' is being used

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corneadoug
thank you!
Modification was completed. :)

@Leemoonsoo
Copy link
Member

LGTM and merge if there're no more discussions.

@asfgit asfgit closed this in a3a2e4d Oct 19, 2016
darionyaphet pushed a commit to darionyaphet/zeppelin that referenced this pull request Oct 27, 2016
…tebooks

### What is this PR for?
Currently, if you have a Notebook large number of,
There is a sharp decrease in performance of the Code editor.
The number and Paragraph creation of the Notebook does not have a relationship.
We are should always use the Code editor of the same performance.

I had to print a Notebook list only if there is a request.
There was quite a lot of performance improvements.

### What type of PR is it?
Improvement

### Todos
- [x] - change Notebook list render logic

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1559
https://issues.apache.org/jira/browse/ZEPPELIN-1513

### How should this be tested?
1. Create a notebook over 1000. (tested 10,000)
   I've used script on this.  (important! please, backup for notebooks directory.)
```
#/bin/bash

idx="0"

while [ $idx -lt 1000 ] # notebooks count 1000
do
  mkdir $idx
  echo """
{
	\"paragraphs\": [
	{
		\"text\": \"fsdkljfowiejfowejfoiwefiowejfiojlkdsjfklsdjflkjsdflkjwlkjefewf\",
			\"dateUpdated\": \"Oct 13, 2016 5:02:44 PM\",
			\"config\": {
				\"colWidth\": 12.0,
			\"graph\": {
				\"mode\": \"table\",
			\"height\": 300.0,
			\"optionOpen\": false,
			\"keys\": [],
			\"values\": [],
			\"groups\": [],
			\"scatter\": {},
			\"map\": {
				\"baseMapType\": \"Streets\",
			\"isOnline\": true,
			\"pinCols\": []
	}
	},
		\"enabled\": true,
		\"editorMode\": \"ace/mode/scala\"
	},
		\"settings\": {
			\"params\": {},
		\"forms\": {}
	},
		\"apps\": [],
		\"jobName\": \"paragraph_1476345687682_-1459653828\",
		\"id\": \"20161013-170127_68316618\",
		\"dateCreated\": \"Oct 13, 2016 5:01:27 PM\",
		\"status\": \"READY\",
		\"progressUpdateIntervalMs\": 500
	}
	],
		\"name\": \""${idx}"\",
		\"id\": \""${idx}"\",
		\"angularObjects\": {
			\"2BZSC9D3G:shared_process\": [],
		\"2BXGWF5TF:shared_process\": [],
		\"2BYKTADMA:shared_process\": [],
		\"2BYFT9HTZ:shared_process\": [],
		\"2BXDTJMED:shared_process\": [],
		\"2BZHPZ6NS:shared_process\": [],
		\"2BYKRWYB3:shared_process\": [],
		\"2BZH2UAT4:shared_process\": [],
		\"2BZ38J35G:shared_process\": [],
		\"2BXJ2X464:shared_process\": [],
		\"2BYQ57ED9:shared_process\": [],
		\"2BXDSA2SN:shared_process\": [],
		\"2BWZCTD2B:shared_process\": [],
		\"2BWT81MNU:shared_process\": [],
		\"2BWA7ZTRD:shared_process\": [],
		\"2BXW6X5KN:shared_process\": [],
		\"2BVWDQPXH:shared_process\": [],
		\"2BZGAF8KG:shared_process\": []
},
	\"config\": {},
	\"info\": {}
	}
  """ > $idx/note.json

  idx=$[$idx+1]
done

```

2. zeppelin restart or refresh notebook.
3. Try coding in any notebook. (fast)

### Screenshots (if appropriate)
#### before ( 5000 notebooks)
![optimization_before_codeditor](https://cloud.githubusercontent.com/assets/10525473/19457150/f18845a0-94ff-11e6-876d-4386dbf1e7de.gif)

#### after ( 5000 notebooks)
![optimization_for_codeeidtor](https://cloud.githubusercontent.com/assets/10525473/19457184/1f7d5036-9500-11e6-8b0c-91b301641c73.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: CloverHearts <[email protected]>

Closes apache#1533 from cloverhearts/ZEPPELIN-1559-Notebook-broadcast-opti and squashes the following commits:

bd58102 [CloverHearts] id Delete unused for navbar
14e88bb [CloverHearts] selectively modified to render the Notebook list.
pedrozatta pushed a commit to pedrozatta/zeppelin that referenced this pull request Oct 27, 2016
…tebooks

### What is this PR for?
Currently, if you have a Notebook large number of,
There is a sharp decrease in performance of the Code editor.
The number and Paragraph creation of the Notebook does not have a relationship.
We are should always use the Code editor of the same performance.

I had to print a Notebook list only if there is a request.
There was quite a lot of performance improvements.

### What type of PR is it?
Improvement

### Todos
- [x] - change Notebook list render logic

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1559
https://issues.apache.org/jira/browse/ZEPPELIN-1513

### How should this be tested?
1. Create a notebook over 1000. (tested 10,000)
   I've used script on this.  (important! please, backup for notebooks directory.)
```
#/bin/bash

idx="0"

while [ $idx -lt 1000 ] # notebooks count 1000
do
  mkdir $idx
  echo """
{
	\"paragraphs\": [
	{
		\"text\": \"fsdkljfowiejfowejfoiwefiowejfiojlkdsjfklsdjflkjsdflkjwlkjefewf\",
			\"dateUpdated\": \"Oct 13, 2016 5:02:44 PM\",
			\"config\": {
				\"colWidth\": 12.0,
			\"graph\": {
				\"mode\": \"table\",
			\"height\": 300.0,
			\"optionOpen\": false,
			\"keys\": [],
			\"values\": [],
			\"groups\": [],
			\"scatter\": {},
			\"map\": {
				\"baseMapType\": \"Streets\",
			\"isOnline\": true,
			\"pinCols\": []
	}
	},
		\"enabled\": true,
		\"editorMode\": \"ace/mode/scala\"
	},
		\"settings\": {
			\"params\": {},
		\"forms\": {}
	},
		\"apps\": [],
		\"jobName\": \"paragraph_1476345687682_-1459653828\",
		\"id\": \"20161013-170127_68316618\",
		\"dateCreated\": \"Oct 13, 2016 5:01:27 PM\",
		\"status\": \"READY\",
		\"progressUpdateIntervalMs\": 500
	}
	],
		\"name\": \""${idx}"\",
		\"id\": \""${idx}"\",
		\"angularObjects\": {
			\"2BZSC9D3G:shared_process\": [],
		\"2BXGWF5TF:shared_process\": [],
		\"2BYKTADMA:shared_process\": [],
		\"2BYFT9HTZ:shared_process\": [],
		\"2BXDTJMED:shared_process\": [],
		\"2BZHPZ6NS:shared_process\": [],
		\"2BYKRWYB3:shared_process\": [],
		\"2BZH2UAT4:shared_process\": [],
		\"2BZ38J35G:shared_process\": [],
		\"2BXJ2X464:shared_process\": [],
		\"2BYQ57ED9:shared_process\": [],
		\"2BXDSA2SN:shared_process\": [],
		\"2BWZCTD2B:shared_process\": [],
		\"2BWT81MNU:shared_process\": [],
		\"2BWA7ZTRD:shared_process\": [],
		\"2BXW6X5KN:shared_process\": [],
		\"2BVWDQPXH:shared_process\": [],
		\"2BZGAF8KG:shared_process\": []
},
	\"config\": {},
	\"info\": {}
	}
  """ > $idx/note.json

  idx=$[$idx+1]
done

```

2. zeppelin restart or refresh notebook.
3. Try coding in any notebook. (fast)

### Screenshots (if appropriate)
#### before ( 5000 notebooks)
![optimization_before_codeditor](https://cloud.githubusercontent.com/assets/10525473/19457150/f18845a0-94ff-11e6-876d-4386dbf1e7de.gif)

#### after ( 5000 notebooks)
![optimization_for_codeeidtor](https://cloud.githubusercontent.com/assets/10525473/19457184/1f7d5036-9500-11e6-8b0c-91b301641c73.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: CloverHearts <[email protected]>

Closes apache#1533 from cloverhearts/ZEPPELIN-1559-Notebook-broadcast-opti and squashes the following commits:

bd58102 [CloverHearts] id Delete unused for navbar
14e88bb [CloverHearts] selectively modified to render the Notebook list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants