fix(groups): a paused cadence discussion no longer wedges a standing team forever

docs/changelog.md

@@ -5,6 +5,23 @@
 
 ---
 
+## ⏱️ fix(groups): a paused cadence discussion no longer wedges a standing team forever (2026-08-09)
+
+**Repo:** EDDI (`fix/cadence-claim-expiry`)
+
+Wave C of the Agent / Group Agent review — a liveness defect in I13 Standing Teams, the newest part of the group subsystem.
+
+`TeamCadenceService.reconcile` releases a workspace's `runningDiscussionId` claim when its discussion reaches a terminal state. `AWAITING_APPROVAL` and `AWAITING_HUMAN_INPUT` are not terminal, so they fell into the `default -> false` ("still running") arm — correctly, for a discussion that will be approved. But the default group HITL timeout policy is `WAIT_INDEFINITELY`, so a pause that nobody resolves never becomes terminal either, and the claim was held **forever**: every subsequent cadence fire for that group was skipped as "still running", and the backlog tasks that run had pulled stayed `IN_PROGRESS`. There was no claim TTL and no reaper anywhere — out of character for a subsystem whose task-force half carries an explicit no-progress fingerprint guard precisely to guarantee termination.
+
+- `GroupWorkspace` gains a nullable `claimedAt` stamp, written next to `runningDiscussionId` and cleared next to it in `settle()` — the two are one fact. Nullable on purpose: documents written before the field existed have no stamp, and reclaiming those on a missing timestamp would be a guess, so they simply get one on their next claim.
+- `reconcile`'s non-terminal arm now routes through `reclaimIfStale`, which cancels the stranded discussion and runs the **ordinary failure writeback** — pulled tasks back to `PENDING`, claim cleared — rather than a bespoke path. Cancel before release, so a reclaimed run cannot keep spending against a budget nobody is tracking.
+- TTL is `eddi.groups.cadence.claim-ttl`, default `PT24H`. Deliberately generous: an approval arriving the next business morning must land on the discussion it belongs to, not on a reclaimed corpse. Non-positive disables reclaiming, for an operator who would rather wedge than risk abandoning a pause.
+- New counter `eddi_team_cadence_claims_reclaimed_total`, and a WARN naming the stranded discussion and how long the claim was held.
+- `POST /groupstore/groups/{id}/workspace/cadences` now warns when a group combines `requiresApproval` phases with `WAIT_INDEFINITELY` — that combination is what makes the backstop reachable, and the operator almost certainly wanted a finite `timeoutPolicy`. A warning, not a rejection: the combination is legitimate for a team whose approver really is always available, and rejecting it would break existing configs.
+
+Also: `cancelQuietly` now passes `ControlSignal.CANCEL_GRACEFUL` explicitly. `null` already resolved to graceful (only `CANCEL_IMMEDIATE` takes the other branch), but the method has a second caller now and "which cancel is this?" should not require reading `GroupHitlCoordinator`.
+
+Suites: 585 tests green across `TeamCadence*`, `GroupWorkspace*`, `RestGroupWorkspace*`, `GroupConversationService*`; 9 new.
 ## 🔐 fix(agents): dynamic-agent guardrails — permissive fallback on resume, per-member caps, duplicate recruits, V7 (2026-08-09)
 
 **Repo:** EDDI (`fix/dynamic-agent-guardrails`)

src/main/java/ai/labs/eddi/configs/groups/model/GroupWorkspace.java

@@ -74,6 +74,23 @@ public class GroupWorkspace {
      * state.
      */
     private String runningDiscussionId = NO_RUNNING_DISCUSSION;
+    /**
+     * When {@link #runningDiscussionId} was claimed, or {@code null} when idle.
+     * <p>
+     * A claim is held until the discussion reaches a terminal state, and an
+     * {@code AWAITING_APPROVAL} / {@code AWAITING_HUMAN_INPUT} pause is not one.
+     * With the default {@code WAIT_INDEFINITELY} timeout policy such a pause never
+     * resolves on its own, so a single unapproved cadence discussion used to hold
+     * the claim forever: every subsequent fire for that group was skipped as "still
+     * running" and the tasks it pulled stayed IN_PROGRESS on the backlog with no
+     * reaper anywhere. This stamp is what lets {@code TeamCadenceService#reconcile}
+     * decide a claim has gone stale and reclaim it.
+     * <p>
+     * Nullable on purpose: workspaces written before this field existed have no
+     * stamp, and reclaiming those on a missing timestamp would be a guess. They are
+     * stamped on their next claim.
+     */
+    private Instant claimedAt;
     /**
      * Backlog task ids the running discussion pulled — what writeback returns to
      * PENDING if the discussion fails, and matches outcomes against when it
@@ -275,6 +292,14 @@ public String getRunningDiscussionId() {
         return runningDiscussionId;
     }
 
+    public Instant getClaimedAt() {
+        return claimedAt;
+    }
+
+    public void setClaimedAt(Instant claimedAt) {
+        this.claimedAt = claimedAt;
+    }
+
     public void setRunningDiscussionId(String runningDiscussionId) {
         this.runningDiscussionId = runningDiscussionId != null ? runningDiscussionId : NO_RUNNING_DISCUSSION;
     }

src/main/java/ai/labs/eddi/configs/groups/rest/RestGroupWorkspace.java

@@ -7,10 +7,12 @@
 import ai.labs.eddi.configs.groups.IAgentGroupStore;
 import ai.labs.eddi.configs.groups.IGroupWorkspaceStore;
 import ai.labs.eddi.configs.groups.IRestGroupWorkspace;
+import ai.labs.eddi.configs.groups.model.AgentGroupConfiguration.DiscussionPhase;
 import ai.labs.eddi.configs.groups.model.GroupWorkspace;
 import ai.labs.eddi.configs.groups.model.GroupWorkspace.Cadence;
 import ai.labs.eddi.configs.groups.model.SharedTaskList;
 import ai.labs.eddi.configs.groups.model.SharedTaskList.TaskItem;
+import ai.labs.eddi.configs.hitl.HitlTimeoutPolicy;
 import ai.labs.eddi.datastore.IResourceStore;
 import ai.labs.eddi.engine.runtime.internal.CronParser;
 import ai.labs.eddi.engine.runtime.internal.TeamCadenceService;
@@ -188,6 +190,11 @@ public Response addCadence(String groupId, CadenceRequest request) {
                         .entity(Map.of("error", "inputTemplate exceeds " + MAX_INPUT_TEMPLATE_LENGTH + " characters"))
                         .build();
             }
+            String indefinitePause = indefinitePauseWarning(groupId);
+            if (indefinitePause != null) {
+                LOG.warnf("Cadence added to group %s which pauses for approval under WAIT_INDEFINITELY: %s",
+                        sanitize(groupId), indefinitePause);
+            }
             String cadenceId = UUID.randomUUID().toString();
             String principal = identity != null && identity.getPrincipal() != null
                     && identity.getPrincipal().getName() != null && !identity.getPrincipal().getName().isBlank()
@@ -283,4 +290,49 @@ private void requireGroupExists(String groupId) throws IResourceStore.ResourceNo
             throw new IResourceStore.ResourceNotFoundException("Group not found.");
         }
     }
+
+    /**
+     * Warns when a group combines automated cadences with an approval gate that
+     * never times out.
+     * <p>
+     * A cadence discussion that pauses for an approval nobody gives is not a
+     * terminal state, so writeback never releases the workspace claim. The claim
+     * TTL in {@code TeamCadenceService} reclaims it eventually, but "eventually,
+     * after a day, by abandoning the run" is a backstop — the operator almost
+     * certainly wanted either a finite {@code hitlConfig.timeoutPolicy} or no
+     * approval gate on an unattended recurring run. Warn rather than reject: the
+     * combination is legitimate for a team whose approver genuinely is always
+     * available, and refusing it would break existing configs.
+     *
+     * @return a human-readable reason, or {@code null} when the combination is not
+     *         present (or the config cannot be read — never fail an otherwise valid
+     *         cadence over a diagnostic)
+     */
+    private String indefinitePauseWarning(String groupId) {
+        try {
+            var resourceId = groupStore.getCurrentResourceId(groupId);
+            if (resourceId == null) {
+                return null;
+            }
+            var config = groupStore.read(groupId, resourceId.getVersion());
+            if (config == null || config.getPhases() == null) {
+                return null;
+            }
+            boolean gatesForApproval = config.getPhases().stream().anyMatch(DiscussionPhase::requiresApproval);
+            if (!gatesForApproval) {
+                return null;
+            }
+            var hitl = config.getHitlConfig();
+            boolean waitsForever = hitl == null || hitl.getTimeoutPolicy() == null
+                    || hitl.getTimeoutPolicy() == HitlTimeoutPolicy.WAIT_INDEFINITELY;
+            if (!waitsForever) {
+                return null;
+            }
+            return "the group has requiresApproval phase(s) and hitlConfig.timeoutPolicy=WAIT_INDEFINITELY, so an "
+                    + "unapproved run holds this team's cadence claim until the claim TTL reclaims it; set a finite "
+                    + "timeoutPolicy to resolve such pauses properly";
+        } catch (Exception e) {
+            return null;
+        }
+    }
 }

src/main/java/ai/labs/eddi/engine/runtime/internal/TeamCadenceService.java

@@ -16,15 +16,18 @@
 import ai.labs.eddi.configs.groups.model.SharedTaskList.TaskItem;
 import ai.labs.eddi.configs.groups.model.SharedTaskList.TaskStatus;
 import ai.labs.eddi.engine.internal.GroupConversationService;
+import ai.labs.eddi.engine.lifecycle.model.ControlSignal;
 import ai.labs.eddi.modules.templating.ITemplatingEngine;
 import ai.labs.eddi.utils.LogSanitizer;
 import io.micrometer.core.instrument.Counter;
 import io.micrometer.core.instrument.MeterRegistry;
 import jakarta.annotation.PostConstruct;
 import jakarta.enterprise.context.ApplicationScoped;
 import jakarta.inject.Inject;
+import org.eclipse.microprofile.config.inject.ConfigProperty;
 import org.jboss.logging.Logger;
 
+import java.time.Duration;
 import java.time.Instant;
 import java.util.ArrayList;
 import java.util.Comparator;
@@ -79,32 +82,60 @@ public class TeamCadenceService {
     /** Fallback identity when a cadence predates {@code createdBy}. */
     public static final String FALLBACK_USER_ID = "system:team-cadence";
 
+    /**
+     * How long a cadence run may hold its workspace claim before {@link #reconcile}
+     * treats it as stale and reclaims it.
+     * <p>
+     * A claim is released by writeback when its discussion reaches a terminal
+     * state, and an {@code AWAITING_APPROVAL} / {@code AWAITING_HUMAN_INPUT} pause
+     * is not one. The default group HITL timeout policy is
+     * {@code WAIT_INDEFINITELY}, so one unapproved cadence discussion held the
+     * claim forever: every later fire for that group was skipped as "still
+     * running", and the tasks it pulled stayed IN_PROGRESS on the backlog. Nothing
+     * anywhere reaped it.
+     * <p>
+     * 24 hours is deliberately generous — an approval that arrives the next
+     * business morning must still land on the discussion it belongs to, not on a
+     * reclaimed corpse. This is a liveness backstop for a wedged team, not an SLA;
+     * an operator who wants a tighter bound should set a finite
+     * {@code hitlConfig.timeoutPolicy} on the group, which resolves the pause
+     * properly instead of abandoning it.
+     */
+    static final Duration DEFAULT_CLAIM_TTL = Duration.ofHours(24);
+
     private final IGroupWorkspaceStore workspaceStore;
     private final IGroupConversationStore conversationStore;
     private final GroupConversationService groupConversationService;
     private final ITemplatingEngine templatingEngine;
     private final MeterRegistry meterRegistry;
+    private final Duration claimTtl;
 
     private Counter cadenceRunsStarted;
     private Counter cadenceRunsSkipped;
     private Counter cadenceWritebacks;
+    private Counter cadenceClaimsReclaimed;
 
     @Inject
     public TeamCadenceService(IGroupWorkspaceStore workspaceStore, IGroupConversationStore conversationStore,
             GroupConversationService groupConversationService, ITemplatingEngine templatingEngine,
-            MeterRegistry meterRegistry) {
+            MeterRegistry meterRegistry,
+            @ConfigProperty(name = "eddi.groups.cadence.claim-ttl", defaultValue = "PT24H") Duration claimTtl) {
         this.workspaceStore = workspaceStore;
         this.conversationStore = conversationStore;
         this.groupConversationService = groupConversationService;
         this.templatingEngine = templatingEngine;
         this.meterRegistry = meterRegistry;
+        // Non-positive disables reclaiming entirely, for an operator who would rather
+        // wedge than risk abandoning a pause.
+        this.claimTtl = claimTtl != null ? claimTtl : DEFAULT_CLAIM_TTL;
     }
 
     @PostConstruct
     void initMetrics() {
         cadenceRunsStarted = meterRegistry.counter("eddi_team_cadence_runs_started_total");
         cadenceRunsSkipped = meterRegistry.counter("eddi_team_cadence_runs_skipped_total");
         cadenceWritebacks = meterRegistry.counter("eddi_team_cadence_writebacks_total");
+        cadenceClaimsReclaimed = meterRegistry.counter("eddi_team_cadence_claims_reclaimed_total");
     }
 
     /** True if the given schedule metadata marks a team-cadence schedule. */
@@ -193,6 +224,10 @@ public CadenceResult processScheduledFire(Map<String, Object> metadata) {
             // 4. Claim: conditional on the persisted idle marker.
             String before = workspace.getRunningDiscussionId();
             workspace.setRunningDiscussionId(gc.getId());
+            // Stamped with the claim so reconcile can tell a long-running discussion
+            // from a wedged one. Set here, next to the id it belongs to, and cleared
+            // in settle() alongside it — the two are one fact.
+            workspace.setClaimedAt(Instant.now());
             workspace.setPulledTaskIds(pulled.stream().map(TaskItem::id).toList());
             for (TaskItem task : pulled) {
                 workspace.getBacklog().updateTask(withStatus(task, TaskStatus.IN_PROGRESS));
@@ -259,10 +294,55 @@ public boolean reconcile(GroupWorkspace workspace) {
             // the next fire read fresh state.
             case COMPLETED -> writebackCompleted(workspace, gc);
             case FAILED, CANCELLED -> writebackFailure(workspace, gc);
-            default -> false; // IN_PROGRESS, SYNTHESIZING, AWAITING_* — still running
+            // IN_PROGRESS, SYNTHESIZING, AWAITING_* — still running, unless the claim
+            // has been held past its TTL. See reclaimIfStale.
+            default -> reclaimIfStale(workspace, gc);
         };
     }
 
+    /**
+     * Releases a claim held past {@link #claimTtl}, cancelling the discussion that
+     * held it.
+     * <p>
+     * Without this a cadence discussion that pauses for an approval nobody gives
+     * wedges its team permanently: the pause is not a terminal state, so writeback
+     * never runs; the claim is never released; every subsequent fire is skipped as
+     * "still running"; and the tasks that run pulled stay IN_PROGRESS on the
+     * backlog. The default group HITL policy is {@code WAIT_INDEFINITELY}, so this
+     * needs no unusual configuration to happen — one unapproved run is enough.
+     * <p>
+     * The discussion is cancelled before the claim is released, so a run cannot
+     * keep spending against a budget nobody is tracking any more, and its tasks go
+     * back to PENDING through the ordinary failure writeback rather than a bespoke
+     * path.
+     *
+     * @return {@code true} when the claim was released and the workspace is idle
+     *         again, {@code false} while the run is still legitimately in flight
+     */
+    private boolean reclaimIfStale(GroupWorkspace workspace, GroupConversation gc) {
+        if (claimTtl == null || claimTtl.isZero() || claimTtl.isNegative()) {
+            return false;
+        }
+        Instant claimedAt = workspace.getClaimedAt();
+        if (claimedAt == null) {
+            // A workspace written before the stamp existed. Reclaiming on a missing
+            // timestamp would be a guess; it gets a stamp on its next claim.
+            return false;
+        }
+        if (Instant.now().isBefore(claimedAt.plus(claimTtl))) {
+            return false;
+        }
+
+        LOGGER.warnf("Cadence claim on group %s has been held since %s (past the %s TTL) by discussion %s in state %s — "
+                + "cancelling it and releasing the claim so the team's cadences can fire again",
+                LogSanitizer.sanitize(workspace.getGroupId()), claimedAt, claimTtl, gc.getId(), gc.getState());
+        cancelQuietly(gc.getId());
+        cadenceClaimsReclaimed.increment();
+        // The ordinary failure writeback: returns the pulled tasks to PENDING and
+        // clears the claim, exactly as a FAILED/CANCELLED discussion would.
+        return writebackFailure(workspace, gc);
+    }
+
     /**
      * Writeback for a COMPLETED cadence discussion: match every pulled backlog task
      * against the discussion's task list by subject — VERIFIED outcomes stay
@@ -343,6 +423,7 @@ private boolean settle(GroupWorkspace workspace, GroupConversation gc, String se
             metrics.setTotalCost(metrics.getTotalCost() + gc.getTotalCost());
         }
         workspace.setRunningDiscussionId(GroupWorkspace.NO_RUNNING_DISCUSSION);
+        workspace.setClaimedAt(null);
         workspace.setPulledTaskIds(List.of());
         try {
             if (!workspaceStore.casRunningDiscussion(workspace, settledDiscussionId)) {
@@ -435,9 +516,14 @@ private String renderQuestion(Cadence cadence, GroupWorkspace workspace, List<Ta
 
     private void cancelQuietly(String discussionId) {
         try {
-            groupConversationService.cancelDiscussion(discussionId, null);
+            // Explicitly graceful. A null mode already resolved to CANCEL_GRACEFUL —
+            // only CANCEL_IMMEDIATE takes the other branch — but this method now has a
+            // second caller (the stale-claim reclaim), and "which cancel is this?" is
+            // not a question either call site should have to answer by reading
+            // GroupHitlCoordinator.
+            groupConversationService.cancelDiscussion(discussionId, ControlSignal.CANCEL_GRACEFUL);
         } catch (Exception e) {
-            LOGGER.warnf("Could not cancel discussion %s after a lost cadence claim: %s", discussionId, e.getMessage());
+            LOGGER.warnf("Could not cancel discussion %s after a lost or expired cadence claim: %s", discussionId, e.getMessage());
         }
     }
 }