Skip to content

Ai attribution tracking - #4561

Merged
jrf0110 merged 1 commit into
mainfrom
ai-attribution
Dec 22, 2025
Merged

Ai attribution tracking#4561
jrf0110 merged 1 commit into
mainfrom
ai-attribution

Conversation

@jrf0110

@jrf0110 jrf0110 commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Adds tracking to AI contributions so we can attribute code e2e from development -> main branch

┌─────────────────────────────────────────────────────────────────────────────┐
│                     AI CONTRIBUTION TRACKING FLOW                            │
└─────────────────────────────────────────────────────────────────────────────┘

1. USER INTERACTION WITH AI-GENERATED CODE
   ┌──────────────────────────────────────────────────────────────┐
   │  User accepts/rejects AI-generated file changes              │
   │  (via ApplyDiffTool, WriteToFileTool, MultiApplyDiffTool,    │
   │   editFileTool, etc.)                                        │
   └────────────────────┬─────────────────────────────────────────┘
                        │
                        ▼
2. FIRE-AND-FORGET TRACKING CALL
   ┌──────────────────────────────────────────────────────────────┐
   │  trackContribution({                                         │
   │    cwd: '/path/to/repo',                                     │
   │    filePath: 'src/file.ts',                                  │
   │    unifiedDiff: '@@...',                                     │
   │    status: 'accepted' | 'rejected',                          │
   │    taskId: 'task_123',                                       │
   │    organizationId: 'org_456',                                │
   │    kilocodeToken: 'token_789'                                │
   │  })                                                          │
   └────────────────────┬─────────────────────────────────────────┘
                        │ (never blocks user workflow)
                        ▼
3. CONTRIBUTION TRACKING SERVICE (Singleton)
   ┌──────────────────────────────────────────────────────────────┐
   │  ContributionTrackingService.getInstance()                   │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ A. Validate Organization ID                            │ │
   │  │    └─> Skip if missing                                 │ │
   │  └────────────────────────────────────────────────────────┘ │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ B. Get Git Context (parallel)                          │ │
   │  │    ├─> getCurrentBranch(cwd)                           │ │
   │  │    └─> getGitRepositoryInfo(cwd)                       │ │
   │  └────────────────────────────────────────────────────────┘ │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ C. Get Project ID                                      │ │
   │  │    └─> getProjectId(cwd, repositoryUrl)                │ │
   │  │        └─> Skip if missing                             │ │
   │  └────────────────────────────────────────────────────────┘ │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ D. Parse Unified Diff                                  │ │
   │  │    extractLineChanges(unifiedDiff)                     │ │
   │  │    ├─> Parse @@ hunk headers for line numbers          │ │
   │  │    ├─> Extract added lines (+)                         │ │
   │  │    │   └─> Compute SHA-1 hash of each line            │ │
   │  │    └─> Extract removed lines (-)                       │ │
   │  │        └─> Compute SHA-1 hash of each line            │ │
   │  │                                                          │ │
   │  │    Returns:                                             │ │
   │  │    linesAdded: [{line_number, line_hash}, ...]         │ │
   │  │    linesRemoved: [{line_number, line_hash}, ...]       │ │
   │  └────────────────────────────────────────────────────────┘ │
   └────────────────────┬─────────────────────────────────────────┘
                        │
                        ▼
4. TOKEN MANAGEMENT (Cached, Short-lived JWT)
   ┌──────────────────────────────────────────────────────────────┐
   │  getValidToken(organizationId, kilocodeToken)                │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ Check cached token:                                    │ │
   │  │  ├─> Valid & same org? ──> Return cached token        │ │
   │  │  └─> Expired/missing? ──> Fetch new token             │ │
   │  └────────────────────────────────────────────────────────┘ │
   │                                                              │
   │  ┌────────────────────────────────────────────────────────┐ │
   │  │ Fetch new token (if needed):                           │ │
   │  │  POST /api/organizations/{orgId}/user-tokens           │ │
   │  │  Authorization: Bearer {kilocodeToken}                 │ │
   │  │                                                          │ │
   │  │  Response:                                              │ │
   │  │  {                                                      │ │
   │  │    token: "short-lived-jwt",                           │ │
   │  │    expiresAt: "2024-01-01T12:00:00Z",                  │ │
   │  │    organizationId: "org_456"                           │ │
   │  │  }                                                      │ │
   │  │                                                          │ │
   │  │  Cache with 60s refresh buffer                         │ │
   │  └────────────────────────────────────────────────────────┘ │
   └────────────────────┬─────────────────────────────────────────┘
                        │
                        ▼
5. BUILD & SEND PAYLOAD
   ┌──────────────────────────────────────────────────────────────┐
   │  POST https://ai-attribution.kiloapps.io/attributions/track  │
   │  Authorization: Bearer {short-lived-jwt}                     │
   │  Content-Type: application/json                              │
   │                                                              │
   │  Payload:                                                    │
   │  {                                                           │
   │    "project_id": "proj_123",                                 │
   │    "branch": "main",                                         │
   │    "file_path": "src/file.ts",                               │
   │    "lines_added": [                                          │
   │      {                                                       │
   │        "line_number": 42,                                    │
   │        "line_hash": "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3"│
   │      }                                                       │
   │    ],                                                        │
   │    "lines_removed": [                                        │
   │      {                                                       │
   │        "line_number": 41,                                    │
   │        "line_hash": "b94a8fe5ccb19ba61c4c0873d391e987982fbbd4"│
   │      }                                                       │
   │    ],                                                        │
   │    "status": "accepted",                                     │
   │    "task_id": "task_123"                                     │
   │  }                                                           │
   └────────────────────┬─────────────────────────────────────────┘
                        │
                        ▼
6. ATTRIBUTIONS WORKER (External Service)
   ┌──────────────────────────────────────────────────────────────┐
   │  Receives and processes contribution data                    │
   │  - Tracks AI-generated code at line level                    │
   │  - Uses SHA-1 hashes for line matching                       │
   │  - Records acceptance/rejection status                       │
   │  - Associates with project, branch, and task                 │
   └──────────────────────────────────────────────────────────────┘

@changeset-bot

changeset-bot Bot commented Dec 19, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3c18860

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
kilo-code Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@jrf0110
jrf0110 requested a review from kilocode-bot December 19, 2025 02:20

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 2 Issues Found

Severity Issue Location
WARNING Missing mock for getGitRepositoryInfo - tests will fail ContributionTrackingService.spec.ts:6-8
SUGGESTION Unused import TokenProvisionResponseSchema ContributionTrackingService.ts:7-8

Recommendation: Address the missing mock before merge to ensure tests pass.

Review Details (7 files)

Files Reviewed:

  • src/core/tools/ApplyDiffTool.ts - tracking integration ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking integration ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration ✓
  • src/core/tools/kilocode/editFileTool.ts - tracking integration ✓
  • src/services/contribution-tracking/ContributionTrackingService.ts (1 suggestion)
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts (1 issue)
  • src/services/contribution-tracking/contribution-tracking-types.ts

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file

Fix these issues in Kilo Cloud

Comment thread src/services/contribution-tracking/ContributionTrackingService.ts Outdated

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 1 New Issue Found

Severity Issue Location
WARNING Batch operations not tracked MultiApplyDiffTool.ts:639-650

Recommendation: Address this issue along with the 2 previously flagged issues before merge.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration ✓
  • src/core/tools/MultiApplyDiffTool.ts (1 new issue)
  • src/core/tools/WriteToFileTool.ts - tracking integration ✓
  • src/core/tools/kilocode/editFileTool.ts - tracking integration ✓
  • src/services/contribution-tracking/ContributionTrackingService.ts (1 prior issue - unused import)
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts (1 prior issue - missing mock)
  • src/services/contribution-tracking/contribution-tracking-types.ts

Checked: Security, bugs, error handling, test coverage

Prior issues still open:

  • Missing mock for getGitRepositoryInfo in tests
  • Unused import TokenProvisionResponseSchema

Fix these issues in Kilo Cloud

Comment thread src/core/tools/MultiApplyDiffTool.ts

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Previous Issues Addressed

The following issues from previous reviews have been resolved:

  • Missing mock for getGitRepositoryInfo - Now properly mocked on line 9 of the test file
  • Unused import TokenProvisionResponseSchema - Import removed (comment now outdated)

1 issue remains open:

  • ⚠️ Batch operations not tracked (MultiApplyDiffTool.ts:650) - Contribution tracking is only called for single file operations. When multiple files are modified in a batch, none are tracked.
Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration ✓
  • src/core/tools/MultiApplyDiffTool.ts (1 open issue)
  • src/core/tools/WriteToFileTool.ts - tracking integration ✓
  • src/core/tools/kilocode/editFileTool.ts - tracking integration ✓
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts

Checked: Security, bugs, error handling, test coverage

Fix these issues in Kilo Cloud

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ All Previous Issues Resolved

The following issues from previous reviews have been addressed:

Issue Status Resolution
Missing mock for getGitRepositoryInfo ✅ Fixed Mock added on line 9 of test file
Unused import TokenProvisionResponseSchema ✅ Fixed Import removed from ContributionTrackingService.ts
Batch operations not tracked ✅ Fixed Tracking added for batch operations (lines 675-688 in MultiApplyDiffTool.ts)

No new issues found in the latest changes.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking for both single and batch operations ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration ✓
  • src/core/tools/kilocode/editFileTool.ts - tracking integration ✓
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file
  • Batch operations now properly tracked alongside single file operations

Recommendation: Ready for merge

@catrielmuller catrielmuller left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good!
Just a quick note: we have some similar functionality at https://github.com/Kilo-Org/kilocode/blob/main/src/core/context-tracking/FileContextTracker.ts.
Could you take a look and see if the trackContribution calls are a bit cleaner in that class?
Also, it would be really helpful if you could add the kilocode_change comments to make merging with upstream a breeze. Thanks so much!

Comment thread src/core/tools/ApplyDiffTool.ts Outdated
Comment thread src/core/tools/ApplyDiffTool.ts
Comment thread src/core/tools/ApplyDiffTool.ts
Comment thread src/core/tools/ApplyDiffTool.ts
Comment thread src/core/tools/ApplyDiffTool.ts
Comment thread src/core/tools/WriteToFileTool.ts Outdated
Comment thread src/core/tools/WriteToFileTool.ts
Comment thread src/core/tools/WriteToFileTool.ts
Comment thread src/core/tools/WriteToFileTool.ts
Comment thread src/core/tools/WriteToFileTool.ts
@jrf0110

jrf0110 commented Dec 20, 2025

Copy link
Copy Markdown
Contributor Author

Need to respect telemetry settings

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 3 Issues Found

Severity Issue Location
SUGGESTION Unused Zod import and schema contribution-tracking-types.ts:2-43
SUGGESTION Missing kilocode_change comments on imports editFileTool.ts:16-17
SUGGESTION Missing kilocode_change start/end block comments editFileTool.ts:184-196

Recommendation: Address catrielmuller's feedback about kilocode_change comments in editFileTool.ts for consistency with other files.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration with proper comments ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking for both single and batch operations ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration with proper comments ✓
  • src/core/tools/kilocode/editFileTool.ts (2 suggestions - missing kilocode_change comments)
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts (1 suggestion - unused code)

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file
  • Batch operations now properly tracked alongside single file operations
  • Previous issues (missing mock, unused import in service) have been resolved

Fix these issues in Kilo Cloud

Comment thread src/services/contribution-tracking/contribution-tracking-types.ts
Comment thread src/core/tools/kilocode/editFileTool.ts
Comment thread src/core/tools/kilocode/editFileTool.ts

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No New Issues Found

8 files reviewed | Confidence: 95% | Recommendation: Address existing feedback before merge

The latest commit (rebase on Dec 21) contains no new code changes since the previous review. The following issues from previous reviews remain unaddressed:

Severity Issue Location Status
SUGGESTION Unused Zod import and TokenProvisionResponseSchema contribution-tracking-types.ts:2,38-43 Open
SUGGESTION Missing kilocode_change comments on imports editFileTool.ts:16-17 Open
SUGGESTION Missing kilocode_change start/end block comments editFileTool.ts:184-196 Open

Additionally, there are 13 unresolved comments from @catrielmuller requesting kilocode_change comments for consistency across other files.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration with proper comments ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking for both single and batch operations ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration with proper comments ✓
  • src/core/tools/kilocode/editFileTool.ts (missing kilocode_change comments)
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts (unused Zod code)

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file
  • Batch operations properly tracked alongside single file operations
  • Previous critical issues (missing mock, batch tracking) have been resolved

Note: All remaining issues are SUGGESTION-level (code style/consistency). The core functionality is sound.

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No New Issues Found

8 files reviewed | Confidence: 95% | Recommendation: Ready for merge

Correction on previous feedback: The Zod schema TokenProvisionResponse in contribution-tracking-types.ts is NOT unused - it's imported as TokenProvisionResponseSchema in ContributionTrackingService.ts (line 13) and used for runtime validation on line 106: TokenProvisionResponseSchema.parse(await response.json()). This is good practice for validating external API responses.

Remaining open comments (SUGGESTION-level only):

  • Missing kilocode_change comments on imports in editFileTool.ts:16-17
  • Missing kilocode_change start/end block comments in editFileTool.ts:184-196

These are minor style consistency issues per @catrielmuller's feedback and don't affect functionality.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration with proper comments ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking for both single and batch operations ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration with proper comments ✓
  • src/core/tools/kilocode/editFileTool.ts (2 minor style suggestions remain)
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts ✓ (Zod schema IS used)

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Zod schema provides runtime validation of external API responses
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file
  • Batch operations properly tracked alongside single file operations

Note: All remaining issues are SUGGESTION-level (code style/consistency). The core functionality is sound and all previous critical issues have been resolved.

@kilo-code-bot kilo-code-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No New Issues Found

8 files reviewed | Confidence: 95% | Recommendation: Ready for merge

The latest commit (80ca2c1) is a rebase with no new code changes since the previous review. All previously flagged issues have been resolved.

Remaining open items (SUGGESTION-level only):

  • Missing kilocode_change comments on imports in editFileTool.ts:16-17
  • Missing kilocode_change start/end block comments in editFileTool.ts:184-196

These are minor style consistency items per @catrielmuller's feedback and don't affect functionality.

Review Details (8 files)

Files Reviewed:

  • .changeset/giant-buckets-clap.md
  • src/core/tools/ApplyDiffTool.ts - tracking integration with proper comments ✓
  • src/core/tools/MultiApplyDiffTool.ts - tracking for both single and batch operations ✓
  • src/core/tools/WriteToFileTool.ts - tracking integration with proper comments ✓
  • src/core/tools/kilocode/editFileTool.ts (2 minor style suggestions remain)
  • src/services/contribution-tracking/ContributionTrackingService.ts
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts
  • src/services/contribution-tracking/contribution-tracking-types.ts ✓ (Zod schema IS used)

Checked: Security, bugs, error handling, test coverage

Positive observations:

  • Fire-and-forget pattern correctly implemented - tracking never blocks user workflow
  • Proper error handling with try-catch that logs but doesn't throw
  • Token caching with expiry buffer prevents unnecessary API calls
  • Zod schema provides runtime validation of external API responses
  • Comprehensive test coverage for core functionality
  • Clean separation of concerns with types in separate file
  • Batch operations properly tracked alongside single file operations
  • Telemetry opt-out properly respected

Note: All remaining issues are SUGGESTION-level (code style/consistency). The core functionality is sound and all previous critical issues have been resolved.

Comment thread src/services/contribution-tracking/ContributionTrackingService.ts
@kilo-code-bot

kilo-code-bot Bot commented Dec 22, 2025

Copy link
Copy Markdown
Contributor

✅ No New Issues Found

8 files reviewed | Confidence: 95% | Recommendation: Merge

The previous review comment about "Debug console.log should be removed" at line 222 was a false positive. The code at that location is a legitimate telemetry check:

if (TelemetryService.hasInstance() && !TelemetryService.instance.isTelemetryEnabled()) {
    return
}

This is not debug code - it properly respects user privacy preferences by skipping tracking when telemetry is disabled.

Review Details

Files Reviewed:

  • .changeset/giant-buckets-clap.md - Changeset entry
  • src/core/tools/ApplyDiffTool.ts - Contribution tracking integration
  • src/core/tools/MultiApplyDiffTool.ts - Contribution tracking integration (single + batch)
  • src/core/tools/WriteToFileTool.ts - Contribution tracking integration
  • src/core/tools/kilocode/editFileTool.ts - Contribution tracking integration
  • src/services/contribution-tracking/ContributionTrackingService.ts - New service (330 lines)
  • src/services/contribution-tracking/__tests__/ContributionTrackingService.spec.ts - Tests (349 lines)
  • src/services/contribution-tracking/contribution-tracking-types.ts - Type definitions

Checked: Security (token handling, API calls), bugs (null handling, async patterns), error handling (fire-and-forget pattern), test coverage

Notes:

  • Token caching with expiry buffer is well implemented
  • Fire-and-forget pattern correctly prevents blocking user workflow
  • Zod schema validation is used for API response parsing
  • All 19 previous review comments have been resolved

@jrf0110
jrf0110 merged commit 77997cb into main Dec 22, 2025
12 checks passed
@jrf0110
jrf0110 deleted the ai-attribution branch December 22, 2025 16:43
@github-actions github-actions Bot mentioned this pull request Dec 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants