CpsFlowExecution: parseScript(): log "Method Too Large" situations more readably #817
Conversation
|
Out of curiosity, what output does this give you and how is it helpful? |
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
…to avoid catching by full name Signed-off-by: Jim Klimov <[email protected]>
…argeExceptionRealistic() tests Signed-off-by: Jim Klimov <[email protected]>
…sException as a carrier of MethodTooLargeException; re-throw with just a compact message to appear in the build log Signed-off-by: Jim Klimov <[email protected]>
|
Tests helped me realize that the PR updated with handling of both eventualities, and if the only exception in view is the MTL - re-throw with a short message avoiding a wall of text in pipeline log. From self-test console (showing both server log Still got a bit of stack trace from generic handling, but it is reasonably short - the ultimate pipeline devs won't have to scroll up a couple of screenfuls just to learn that their script was too long or complex. Now that the message is visible to end users, I am open to suggestions how to make it more actionable, and/or if the e.g. to explain about script length or its coding complexity and what to do about it (nesting, amount of stages, separation of methods into JSL classes, etc.); maybe point to a knowledge-base URL for a better maintainable article on the subject (don't want to replace one wall of text in the logs with another, right?) |
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
…unt to satisfy different CI platforms Signed-off-by: Jim Klimov <[email protected]>
…t importing the class Signed-off-by: Jim Klimov <[email protected]>
…uggestions for pipeline devs Signed-off-by: Jim Klimov <[email protected]>
… expected matching line Signed-off-by: Jim Klimov <[email protected]>
Until today, it was helpful to myself and a few colleagues who may/can/do follow traces of the Jenkins server logs when developing pipelines (testing in private Jenkins instances and those wrapped by IDEs notwithstanding). After the review comments some months ago (thanks, and sorry for not noticing) and a coding effort today, this should be better visible and more "actionable" to a more general population - now shown as a smaller wall of text right in the pipeline build logs: ...which is IMHO a bit more comprehensible than the original (especially the first time you see it): ...and requires less scrolling to get to the crux of the issue =D |
…ethods (class complexity) from maxTryCatch (nesting depth) Signed-off-by: Jim Klimov <[email protected]>
There was a problem hiding this comment.
The logic here is pretty convoluted and seems excessive when we do not really need to make things particularly pretty, we just need to offer some clue beyond what there is today. Would suffice to do something along the lines of
if (Functions.printThrowable(x).containsString("MethodTooLargeException")) {
throw new IllegalArgumentException("…your hint here…", x);
} else {
throw x;
}
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
…wExecution.java Do not log to server console log (not all server admins are those who run the projects with issues). Co-authored-by: Jesse Glick <[email protected]>
| if (ecCount > 1) { | ||
| // Not squashing with explicit MethodTooLargeException | ||
| // re-thrown below, in this codepath we have other errors. | ||
| throw new RuntimeException(msg, x); | ||
| } else { | ||
| // Do not confuse pipeline devs by a wall of text in the | ||
| // build console, but let the full context be found in | ||
| // server log with some dedication. | ||
| LOGGER.log(Level.FINE, mtlEx.getMessage()); | ||
| //throw new RuntimeException(msg, mtlEx); | ||
| throw new RuntimeException(msg); | ||
| } |
There was a problem hiding this comment.
Regarding the concern of "pretty convoluted logic" - this block is largely why I did it so: groovy can return a MultipleCompilationErrorsException with an attached collection of one or more errors (broadly speaking), so we need to count if we only had the MethodTooLargeException one way or another here (and then print the pretty log), or also something else and then better print everything for human devs to sift through it diligently.
plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
Outdated
Show resolved
Hide resolved
|
Sorry it took a while to notice that there was finally a new review here - too many browser tabs open to keep track of such :D Thanks for doing it! |
|
Hm, got another use-case to handle: managed to get a step definition too large (aka "global variable", anyhow in JSL as So the pipeline chugged along until it called that step - and that's where the job crashed, with a different exception: Note there are actually two lines Gotta look into the structure of |
Notably, the latter is a Throwable but not an Exception so e.g. LinkageError might be not handled. Signed-off-by: Jim Klimov <[email protected]>
…clude x.getMessage(), a copy is already there Signed-off-by: Jim Klimov <[email protected]>
…ationErrorsException.getMessage() is too long with all the original stack trace Introduce an xMsgStart with curated start of original Throwable's message (until the first "\tat somewhere(file:line)" match). Signed-off-by: Jim Klimov <[email protected]>
…ee in pretty-printed overflowedClassName Signed-off-by: Jim Klimov <[email protected]>
…SNAME_PATTERN and CLASSNAME_MENTIONS_PATTERN expectations/assumptions in code Signed-off-by: Jim Klimov <[email protected]>
…e may have slashes for class or not for step Signed-off-by: Jim Klimov <[email protected]>
…ine scripts named like "Script<NUM>" Signed-off-by: Jim Klimov <[email protected]>
…excerpts into snip-markers to separate visibly Signed-off-by: Jim Klimov <[email protected]>
…eMsg as a StringBuilder Signed-off-by: Jim Klimov <[email protected]>
…dClassNameBreadcrumbs as a StringBuilder Signed-off-by: Jim Klimov <[email protected]>
…verflowedClassName for overflowedClassNameBreadcrumbs Signed-off-by: Jim Klimov <[email protected]>
…excerpts - only add a newline before final wrapper if not present in wrapped text Signed-off-by: Jim Klimov <[email protected]>
…eraction" with ContinuationGroup.fixupStackTrace() Signed-off-by: Jim Klimov <[email protected]>
…x in the end, use a dedicated "RuntimeException rtex" object Signed-off-by: Jim Klimov <[email protected]>
…line Signed-off-by: Jim Klimov <[email protected]>
|
Updated screenshots, now with some detail learned about MTL issues happening in the JSL also:
Notably, code was made to trace the messages I've seen (and posted earlier above) with "bread-crumbs" to see the code path through the pipeline script and groovy (JSL) files and line numbers (for steps/classes with code too large), only to discover this is slapped on later via Probably some refactoring can be due to take advantage of that bit. At least, the groovy-cps code is now in the same plugin. |
…ding for logged message Signed-off-by: Jim Klimov <[email protected]>
| } | ||
| actionableMsg.append("; please refactor to simplify code structure"); | ||
| if (overflowedClassNameReport.contains("WorkflowScript") || CLASSNAME_SCRIPTNUM_PATTERN.matcher(overflowedClassName).find()) | ||
| actionableMsg.append(" and/or move logic to a Jenkins Shared Library"); |
There was a problem hiding this comment.
BTW the terminology in https://www.jenkins.io/doc/book/pipeline/shared-libraries/ is unfortunate. By definition a “library” is “shared”; why else would you create a library? The plugin itself uses the term Pipeline Groovy library.
| @@ -466,6 +470,8 @@ Timing time(TimingKind kind) { | |||
|
|
|||
| static final Logger TIMING_LOGGER = Logger.getLogger(CpsFlowExecution.class.getName() + ".timing"); | |||
|
|
|||
| static final Logger METHOD_TOO_LARGE_LOGGER = Logger.getLogger(CpsFlowExecution.class.getName() + ".MethodTooLargeLogging"); | |||
There was a problem hiding this comment.
Is there some reason not to simply use
?
| //rtex.addSuppressed(mtlEx); | ||
|
|
||
| return rtex; | ||
| } |
There was a problem hiding this comment.
I am just going to note the irony of a 285-line method here. 😁
| Throwable mtlEx = null; | ||
| int ecCount = 0; | ||
| String xStr = Functions.printThrowable(x); // includes x.getMessage() contents | ||
| final Pattern LINE_SEP_PATTERN = Pattern.compile("\\R"); |
There was a problem hiding this comment.
I had to look this one up.
Any Unicode linebreak sequence, is equivalent to
\u000D\u000A|[\u000A\u000B\u000C\u000D\u0085\u2028\u2029]
AFAICT
| final Pattern LINE_SEP_PATTERN = Pattern.compile("\\R"); | |
| final Pattern LINE_SEP_PATTERN = Pattern.compile("\r?\n"); |
suffices.
| .append(METHOD_TOO_LARGE_LOGGER.getName()) | ||
| .append(" is required)"); | ||
|
|
||
| //return new RuntimeException(actionableMsg.toString(), mtlEx); |
There was a problem hiding this comment.
Do not leave commented-out code in a final PR (unless there is a real possibility it should be resurrected some day soonish and a preceding comment explains those circumstances).
| } else if (x instanceof CpsCompilationErrorsException) { | ||
| // Defined in this plugin, to clone a message and stack trace | ||
| // from a MultipleCompilationErrorsException and be serializable. | ||
| // Grep it as text for "MethodTooLargeException" and "1 error" |
There was a problem hiding this comment.
Would it not make more sense to just enrich CpsCompilationErrorsException with whatever information you need? AFAICT all you are looking for ultimately is the MethodTooLargeException.message. So, we could just keep that in a field?
| @Test public void methodTooLargeExceptionFabricated() throws Exception { | ||
| // Fabricate a MethodTooLargeException which "normally" happens when evaluated |
There was a problem hiding this comment.
Is there a reason to keep this test case, given the “realistic” test below?
| // * java.lang.StackOverflowError varies per JDK platform | ||
| // (CI on Linux was unhappy with 255, on Windows with 1023) |
There was a problem hiding this comment.
You can set a per-thread stack size, but unfortunately the unit of measurement is unspecified and presumably specific to JVM implementation details, so this is probably not that helpful.
There was a problem hiding this comment.
(Why would you would be getting a JVM-level SOE, anyway? This is all dealing with the CPS VM that does not use the native stack.)
There was a problem hiding this comment.
OK I think, though I can barely follow most of the code here. @dwnusbaum did you expect to review?
|
I have no plans to review this. My impression from a very brief look at the code was that it seemed quite complicated, but I am not blocking it. IMO creating a page like https://www.jenkins.io/doc/book/pipeline/cps-method-mismatches/ explaining the situation and recommendations in detail seems like it could help, and then perhaps this code could be simplified to only check the stack trace for the relevant class types and add a suppressed exception that links to that page, accepting potential false positives and not adding the additional context currently added by this PR (but maybe that context is critically useful, IDK). |
#817 (review) more or less. IIUC this PR offers the same actionable information as was already there, but formatted differently?
Actually a redirect URL like in jenkins-infra/jenkins.io#2290. |
A small almost-cosmetic change which makes life easier for my colleagues when they develop (and debug failures of) Jenkins scripted pipelines.
UPDATE: Grew to be not so small, so evicted into a method of its own to constrain changes to other parts of the codebase. Learned a few more tricks over time, as new MTL situations were found in the field.