Skip to content

GET /_migration/deprecations doesn't report node-level failures properly #137010

@DaveCTurner

Description

@DaveCTurner

As mentioned in #137004, we log (but otherwise ignore) node-level failures to check for deprecations:

logger.warn("nodes failed to run deprecation checks: {}", failedNodeIds);
for (FailedNodeException failure : response.failures()) {
logger.debug("node {} failed to run deprecation checks: {}", failure.nodeId(), failure);
}

This is very problematic since it means users may incorrectly believe their cluster to be ready to upgrade, only to discover during the upgrade that it is now broken. If the response to the API is incomplete then users need to know about it.

It's probably a bad idea to report the details at DEBUG, and even worse we actually only report the result of failure.toString() which is something useless like Failed node [RiRNhBZARO-v7peMV0rQDg] org.elasticsearch.action.FailedNodeException: Failed node [RiRNhBZARO-v7peMV0rQDg] rather than reporting the whole stack trace:

diff --git a/x-pack/plugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/NodeDeprecationChecker.java b/x-pack/plugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/NodeDeprecationChecker.java
index a2e9ed12a229..53bd7e492d17 100644
--- a/x-pack/plugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/NodeDeprecationChecker.java
+++ b/x-pack/plugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/NodeDeprecationChecker.java
@@ -52,7 +52,9 @@ public class NodeDeprecationChecker {
                         .collect(Collectors.toList());
                     logger.warn("nodes failed to run deprecation checks: {}", failedNodeIds);
                     for (FailedNodeException failure : response.failures()) {
-                        logger.debug("node {} failed to run deprecation checks: {}", failure.nodeId(), failure);
+                        logger.atDebug()
+                            .withThrowable(failure)
+                            .log("node {} failed to run deprecation checks: {}", failure.nodeId(), failure.getMessage());
                     }
                 }
                 l.onResponse(reduceToDeprecationIssues(response));

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions