Speed up progressive rendering #396
Conversation
martinRenou
changed the title
|
@maartenbreddels what do you think? It improves perfomances but the logic was also wrong before (processing each cell like it's a single Notebook) |
|
Part of the slowness was coming from the fact that internally nbconvert creates new sockets for connecting with the kernel at each |
martinRenou
changed the title
|
Reviewed this in person with @martinRenou and this looks great. |
martinRenou
changed the title
|
I'm wondering if the |
|
I now reimplement |
martinRenou
changed the title
There was a problem hiding this comment.
Pretty nice! I see the same performance increase in the asyncio PR #374, but lets get this in first, it is much simpler. Then we make in multithreaded and after that worry about true asyncio.
Would you mind adding a unittest that tests that the output is removed (since I think we can consider this a security risk if it contains a stacktrace).?
| try: | ||
| result = super(VoilaExecutePreprocessor, self).preprocess_cell(cell, resources, cell_index, store_history) | ||
| except CellExecutionError as e: | ||
| self.log.error(e) | ||
| result = [cell] | ||
| return result | ||
|
|
There was a problem hiding this comment.
To mimic the VoilaExecutePreprocessor.preprocess behavior. I change this behavior a bit in a separate PR concerning error outputs BTW.
There was a problem hiding this comment.
The try except prevents a 505 server error when the Notebook contains an error
Should fix #395
Instead of using the
preprocessmethod (which is for preprocessing an entire Notebook) for each cell, I usepreprocess_cellfor each cell