Skip to content

[SPARK-17243] [Web UI] Spark 2.0 History Server won't load with very large application history #14835

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ajbozarth wants to merge 7 commits into from
Closed

[SPARK-17243] [Web UI] Spark 2.0 History Server won't load with very large application history #14835

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9bcb02d
Added limit param to applications list api and maxApplications conf t…
ajbozarth Aug 26, 2016
fab32f9
Switched to use ConfigBuilder
ajbozarth Aug 26, 2016
6163f13
Added tests and fixed a typos
ajbozarth Aug 26, 2016
172dc56
Switched to js solution and updated docs
ajbozarth Aug 26, 2016
bbb9718
Fixed some wording in the docs
ajbozarth Aug 29, 2016
aa1c6f9
Merge branch 'master' into spark17243
ajbozarth Aug 30, 2016
8da22e3
updated doc wording
ajbozarth Aug 30, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,16 @@ The history server can be configured as follows:
<td>spark.history.retainedApplications</td>
<td>50</td>
<td>
The number of application UIs to retain. If this cap is exceeded, then the oldest
applications will be removed.
The number of application UIs to retain in the cache. If this cap is exceeded, then the oldest
applications will be removed from the cache.
Copy link
Contributor

@tgravescs tgravescs Aug 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of a nit here but perhaps we should change this to "The number of applications to retain in the cache". I'm wondering if UIs could still be confusing with the number displayed. And perhaps after the last sentence add something like: If an application is not in the cache, it will have to be loaded from disk if its accessed from the UI.

Copy link
Member Author

@ajbozarth ajbozarth Aug 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that'll help me reword it a bit

</td>
</tr>
<tr>
<td>spark.history.ui.maxApplications</td>
<td>Int.MaxValue</td>
<td>
The number of applications to display on the history summary page. Application UIs are
still available even if they are not displayed on the history summary page.
The number of applications to display on the history summary page. Application UIs are still
available by accessing their URLs directly even if they are not displayed on the history summary page.
</td>
</tr>
<tr>
Expand Down