Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce default size of diagnostic buffer for FlightRecorderInputStream from 1MiB to 1KiB #770

Merged
merged 3 commits into from
Oct 21, 2024

Conversation

dwnusbaum
Copy link
Member

@dwnusbaum dwnusbaum commented Oct 18, 2024

One of the main side effects of the bug fixed by jenkinsci/google-compute-engine-plugin#478 was that each failed connection caused a 1MiB buffer to be retained by a FlightRecorderInputStream, leading to high memory usage. This buffer is only used to print a hex dump of the stream contents in case of a stream corruption error, and so having it be a full 1MiB by default seems unnecessary, and if there is something interesting in the dump, it is probably right at the end.

I considered disabling the buffer completely by making the default value 0, but that requires a few changes to avoid ArrayIndexOutOfBoundsException and improve the error messages, so it seemed more expedient to just reduce the default buffer size.

In the real-world case we observed, there were around 600 instances of this class in memory, so this patch would have cut ~600MB of memory used by these buffers down to around ~600KB.

See jenkinsci/jenkins#9887 for the matching core PR.

Testing done

Only tested using the existing automated tests in FlightRecorderInputStreamTest and DiagnosedStreamCorruptionExceptionTest.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

@dwnusbaum dwnusbaum requested review from jglick and Vlatombe October 18, 2024 19:30
@@ -23,7 +23,7 @@ class FlightRecorderInputStream extends InputStream {
* @since 2.41
*/
static final int BUFFER_SIZE =
Integer.getInteger("hudson.remoting.FlightRecorderInputStream.BUFFER_SIZE", 1024 * 1024);
Integer.getInteger("hudson.remoting.FlightRecorderInputStream.BUFFER_SIZE", 1024);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think this is too aggressive, we could increase to 10KiB.

@jglick jglick added the enhancement For changelog: An enhancement providing new capability. label Oct 18, 2024
@jglick jglick merged commit 4cfe589 into jenkinsci:master Oct 21, 2024
14 checks passed
@dwnusbaum dwnusbaum deleted the reduce-diagnostic-buffer-size branch October 23, 2024 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement For changelog: An enhancement providing new capability.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants