Skip to content

docs(hitl): Plan to implement human in the loop - #164

Merged
murdore merged 1 commit into
juspay:releasefrom
naynisinghal1008:BZ-43988-plan-to-implement-human-in-the-loop-in-neurolink
Sep 11, 2025
Merged

murdore merged 1 commit into
juspay:releasefrom
naynisinghal1008:BZ-43988-plan-to-implement-human-in-the-loop-in-neurolink

Conversation

@naynisinghal1008

@naynisinghal1008 naynisinghal1008 commented Sep 11, 2025 •

Copy link
Copy Markdown
Contributor

Pull Request

Description

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🧹 Code refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test coverage improvement
  • 🔧 Build/CI configuration change

Related Issues

  • Fixes #
  • Related to #

Changes Made

AI Provider Impact

  • OpenAI
  • Anthropic
  • Google AI/Vertex
  • AWS Bedrock
  • Azure OpenAI
  • Hugging Face
  • Ollama
  • Mistral
  • All providers
  • No provider-specific changes

Component Impact

  • CLI
  • SDK
  • MCP Integration
  • Streaming
  • Tool Calling
  • Configuration
  • Documentation
  • Tests

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • E2E tests added/updated
  • Manual testing performed
  • All existing tests pass

Test Environment

  • OS:
  • Node.js version:
  • Package manager:

Performance Impact

  • No performance impact
  • Performance improvement
  • Minor performance impact (acceptable)
  • Significant performance impact (needs discussion)

Breaking Changes

Screenshots/Demo

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

Summary by CodeRabbit

  • Documentation
    • Added a comprehensive Human-in-the-Loop (HITL) implementation guide for NeuroLink.
    • Covers architecture, core components, configuration options (including enterprise setups and environment-based settings), and integration patterns with tools and external services.
    • Details event-driven workflows, timeout/cancellation handling, and user approval flows.
    • Includes security guidance (authentication/authorization, audit logging, input sanitization).
    • Provides testing strategies (unit/integration) and end-to-end usage examples with frontend-backend coordination.
    • Offers practical setup examples and best practices for safe, monitored tool execution.

@coderabbitai

coderabbitai Bot commented Sep 11, 2025 •

Copy link
Copy Markdown

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Adds a new documentation file detailing a Human-in-the-Loop (HITL) design for NeuroLink, covering architecture, configuration, event contracts, tool registry and external MCP integration patterns, security/testing guidance, and illustrative code snippets for a conceptual HITLManager and related errors.

Changes

Cohort / File(s) Summary
HITL documentation
docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
New doc specifying HITL architecture, public API concepts (HITLManager, HITLConfig, rules, errors), event model, Tool Registry and external MCP interception patterns, configuration examples, security considerations, and testing strategy with example snippets.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant Frontend
  participant EventBus as Event Bus
  participant App as NeuroLink App
  participant HITL as HITL Manager
  participant Tools as Tool Registry

  App->>HITL: requiresConfirmation(tool, args)
  alt Confirmation required
    App->>HITL: requestConfirmation(tool, args)
    HITL-->>EventBus: emit hitl:confirmation-request
    EventBus-->>Frontend: deliver request
    Frontend->>User: Display request UI
    User-->>Frontend: Approve/Reject/Modify args
    Frontend-->>EventBus: hitl:confirmation-response
    EventBus-->>HITL: response payload
    alt Approved
      HITL-->>App: ConfirmationResult(approved, args?)
      App->>Tools: executeTool(tool, approvedArgs)
      Tools-->>App: result
    else Rejected
      HITL-->>App: HITLUserRejectedError
    end
  else No confirmation
    App->>Tools: executeTool(tool, args)
    Tools-->>App: result
  end
Loading
sequenceDiagram
  autonumber
  participant App as NeuroLink App
  participant HITL as HITL Manager
  participant MCP as External MCP Server

  App->>HITL: requiresConfirmation(externalTool, args)
  alt Required
    App->>HITL: requestConfirmation(...)
    HITL-->>App: approved? (or error/timeout)
  end
  opt On approval
    App->>MCP: executeExternalTool(args)
    MCP-->>App: response
  end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested reviewers

  • murdore

Pre-merge checks (3 passed)

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "docs(hitl): Plan to implement human in the loop" succinctly and accurately describes the primary change—adding a documentation plan for a human-in-the-loop implementation. It is concise, uses a standard scope prefix, and communicates the PR’s intent clearly without extraneous detail. A reviewer scanning history can understand the main purpose from this title alone.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.

Poem

A hop, a pause—“Proceed?” I ask,
In loops where humans share the task.
I thump approval, nose-twitch fast,
Events hop by, the checks are passed.
With tools in line and logs to keep,
HITL now guards while engines leap. 🐇✨

✨ Finishing touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (8)
docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md (8)

218-247: Use one internal executor name consistently

Earlier snippet calls executeToolInternal; later section uses executeToolOriginal. Pick one to avoid confusion.

-  return this.executeToolInternal(toolName, args, context);
+  return this.executeToolOriginal(toolName, args, context);

527-585: Expose confirmationId to callers or document how to retrieve it

Callers often need the ID for tracing/tests. Either return it or document listening for the request event to capture it.

Option A (return id + promise):

-async requestConfirmation(...): Promise<ConfirmationResult> {
+async requestConfirmation(...): Promise<ConfirmationResult> { /* keep for compat */ }
+
+requestConfirmationWithId(
+  toolName: string,
+  arguments_: unknown,
+  context?: { serverId?: string; sessionId?: string; userId?: string },
+): { confirmationId: string; wait: Promise<ConfirmationResult> } {
+  const confirmationId = this.generateConfirmationId();
+  const wait = this._requestWithId(confirmationId, toolName, arguments_, context);
+  return { confirmationId, wait };
+}

925-933: Integrate HITLSecurityConfig into HITLConfig

Security options are defined but not wired into HITLConfig; fold them in to avoid orphaned config.

 export interface HITLConfig {
   enabled: boolean;
   dangerousActions: string[];
   timeout: number;
   confirmationMethod: "event";
   allowArgumentModification: boolean;
   auditLogging?: boolean;
   customRules?: HITLRule[];
+  security?: HITLSecurityConfig;
 }

1033-1070: Complete the integration test and await execution

The snippet is truncated and doesn’t await tool completion after approval.

-// Execute tool (should trigger HITL)
-const executionPromise = neurolink.executeTool('deleteFile', { path: '/test.txt' });
+const executionPromise = neurolink.executeTool('deleteFile', { path: '/test.txt' });
@@
-      });
-    });
-```
+      });
+    });
+
+    const result = await executionPromise;
+    expect(result).toBe('Deleted: /test.txt');
+  });
+});

289-307: Event schema: timestamp type clarification

Explicitly state that metadata.timestamp is ISO string, and consumers must parse to number for arithmetic.


706-716: Action description coverage

Consider including “truncate” to match dangerousActions examples for better UX consistency.

 if (lowerToolName.includes("drop")) return "Drop Operation";
+if (lowerToolName.includes("truncate")) return "Truncate Operation";

748-758: Statistics placeholders

Call out that totals/averages are stubs and indicate where to source data (e.g., in-memory counters or audit store).


575-583: Timeout audit parity

You log “confirmation-requested” and “confirmation-responded”; add explicit “confirmation-approved/rejected” for parity or clarify mapping.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 75d0c6d and fb6dceb.

📒 Files selected for processing (1)
  • docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md (1 hunks)

Comment thread docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
Comment thread docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
Comment thread docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
Comment thread docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
Comment thread docs/HUMAN-IN-THE-LOOP-IMPLEMENTATION.md
@naynisinghal1008
naynisinghal1008 force-pushed the BZ-43988-plan-to-implement-human-in-the-loop-in-neurolink branch from fb6dceb to 5d3fd7e Compare September 11, 2025 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants