NIFI-16174 - Treat a stateless process group as a single lifecycle unit when starting/stopping a controller service's referencing components - #11515
Conversation
| * process group is not part of a stateless group. A group whose execution engine is {@code INHERITED} resolves to | ||
| * its nearest ancestor that explicitly declares an execution engine. | ||
| */ | ||
| private ProcessGroup getStatelessGroup(final ProcessGroup start) { |
There was a problem hiding this comment.
I think there's a minor bug here. This returns the inner-most Process Group that is marked as STATELESS. But we need to operate on the outermost group. So if you have something like STANDARD -> STANDARD -> INHERITED -> STATELESS -> INHERITED -> INHERITED this works fine. But if it's STANDARD -> STANDARD -> INHERITED -> STATELESS -> STATELESS -> STATELESS we will have an issue because we'll attempt to operate on the inner-most group, which will fail because it is an inner Stateless group.
There was a problem hiding this comment.
Yeah you're right, good catch. And it fails silently rather than erroring out: startProcessing() logs "Cannot start Process Group ... because its parent is configured to run using the Stateless Engine" and returns, and stopProcessing() returns an already completed future without stopping anything, so the caller thinks the stop worked.
Fixed in the latest commit. The helper now walks up while the group resolves to STATELESS and keeps the last one, so it returns the top-most group. Switched to resolveExecutionEngine() too so the INHERITED collapsing is handled there rather than recursing on it here, and documented the rule on ProcessGroup.startProcessing() / stopProcessing(). Tests cover your exact chain, plus one pinning the no-op itself since stopComponents() relies on it to terminate the recursion during stateless shutdown. The system test now has a nested subtree whose only referencing processor is in the inner group, it wouldnt catch this otherwise since a referencing processor in the outer group masks the nested no-op.
Also worth flagging, the same innermost-resolution helper is copied in AffectedComponentSet (line 461) and StandardAssetComponentManager (291), and StandardProcessGroupDAO.scheduleComponents() (310) calls start/stopProcessing directly on the requested group so it has the same exposure. Left those alone to keep this PR focused, happy to do a follow up if you'd prefer.
Let me know what you think, thanks!
There was a problem hiding this comment.
Oh good catch. Yeah, would be a good idea to do that as a follow-on, thanks!
|
@ncover21 thanks for the fix. Changes look good to me. Looks like build validation is failing because it's not up to date with |
…it when starting/stopping a controller service's referencing components StandardControllerServiceProvider scheduled processors that reference a controller service individually, even when they belong to a stateless process group. Because a stateless group is a single scheduling unit, this left the group with a mixed running/stopped processor state and a group node stuck RUNNING, from which it could not recover. Resolve each referenced processor's owning stateless group (STATELESS -> self, INHERITED -> nearest explicit ancestor) and, for stateless members, stop the group once via ProcessGroup.stopProcessing() / start it once via ComponentScheduler.startStatelessGroup(), mapping the group's single future to every affected member. Standard processors are unchanged. Public ControllerServiceProvider signatures are unchanged. Adds unit coverage in StandardControllerServiceProviderTest and an end-to-end regression (ConnectorTroubleshootingIT) backed by a stateless controller-service reference in the ComponentLifecycleConnector test fixture.
…wning stateless group A referenced processor's process group can report a null execution engine (e.g. in unit-test fixtures backed by mock process groups). Treat a null engine as non-stateless so getStatelessGroup returns null and the processor is handled on the standard per-component path, rather than throwing an NPE in the switch.
…ing a controller service's referencing components The previous helper returned the inner-most Process Group marked STATELESS. Only the top-most stateless group may be started or stopped directly, so on a nested stateless chain both paths became silent no-ops: startProcessing() logs a warning and returns, and stopProcessing() returns an already-completed Future without stopping anything. Walk up while the group resolves to STATELESS and operate on the last one. Document the rule on ProcessGroup.startProcessing()/stopProcessing(), and pin the no-op with a test, since it is what terminates the recursive stopComponents() walk during stateless shutdown. The system test now carries a second stateless subtree whose only referencing processor lives in a nested stateless group, which is what makes it discriminate: when a referencing processor exists in the outer group too, the outer group's transition masks the nested no-op.
2 participants
Summary
NIFI-16174
StandardControllerServiceProviderscheduled controller-service referencing processors individually even when they belong to a stateless process group. Because a stateless group is a single scheduling unit, this left the group in a mixed running/stopped state with the group node stuckRUNNINGand unable to recover. It surfaces during break-glass/troubleshooting on connectors whose stateless subgroup references a root-scoped controller service.The start path already skips stateless members (
DefaultComponentScheduler.startNowreturns early whenresolveExecutionEngine() == STATELESS); the stop path (unscheduleReferencingComponents) did not, so it stopped members individually. Simply ignoring stateless members on stop is not sufficient either: a running stateless member reportsgetPhysicalScheduledState() == RUNNING, so it counts as an active reference inverifyCanDisable, which would then block the controller service from being disabled.This change treats the owning stateless group as a single unit on both paths:
STATELESS-> self,INHERITED-> nearest explicit ancestor).ProcessGroup.stopProcessing()once per distinct owning stateless group (its members transition to STOPPED together, so the service can then be disabled), mapping the group's single future to every affected member.ComponentScheduler.startStatelessGroup()once per distinct owning stateless group, so "enable service and start referencing components" restarts the group.Standard (non-stateless) referencing components are unchanged, and the public
ControllerServiceProvidermethod signatures are unchanged.Tracking
NIFI-16174mainbuilds on the changed modules (checkstyle + PMD pass)