test(ai): enforce complete audit assurance gates - #111
Merged
Conversation
📝 WalkthroughWalkthroughAI 서비스의 감사 오류 소유권과 PostgreSQL 검증을 정리했다. 공개 부트스트랩 API와 서버 진입점을 추가했다. JSDoc·100% 커버리지 게이트, 운영 보증 문서, capability manifest와 CI 검증을 갱신했다. ChangesAI 서비스 품질 및 감사 경계
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
Contributor
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
apps/ai-service/package.json (1)
8-8: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win패키지 lint가 리포지터리 루트 문서를 포맷 검사합니다.
lint가../../docs/...경로의 문서 3개를 검사합니다. 이 방식은 패키지 스크립트를 리포지터리 배치에 결합합니다. 문서 경로가 바뀌거나 파일이 삭제되면apps/ai-service의lint가 실패합니다. 또한 Turborepo 캐시 입력이 패키지 외부 파일 변경을 추적하지 못하면 결과가 신뢰할 수 없게 됩니다. 문서 포맷 검사는 루트format/lint스크립트로 옮기고, 이 스크립트는 패키지 내부 파일만 검사하도록 유지하십시오.♻️ 제안 변경
- "lint": "tsc --noEmit && prettier --single-quote --check package.json tsconfig.json vitest.config.ts \"src/**/*.ts\" ../../docs/operations/ai-proposal-audit-assurance.md ../../docs/superpowers/specs/2026-08-04-ai-service-quality-gates-design.md ../../docs/superpowers/plans/2026-08-04-ai-service-quality-gates.md", + "lint": "tsc --noEmit && prettier --single-quote --check package.json tsconfig.json vitest.config.ts \"src/**/*.ts\"",문서 파일은 루트
package.json의 포맷 대상에 추가하십시오.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ai-service/package.json` at line 8, Update the ai-service lint script to remove the three ../../docs document paths, keeping it limited to package-owned files and existing TypeScript checks. Add those documents to the repository-root package.json format/lint target so their formatting is validated centrally.apps/ai-service/src/postgres-proposal-audit-repository.ts (1)
303-316: 🚀 Performance & Scalability | 🔵 Trivial
listProposals는 워크스페이스의 전체 행을 한 번에 반환한다.
ai.proposal_audit_records는 append-only 감사 테이블이다. 행 수는 계속 증가한다.listProposals에는LIMIT이나 커서가 없다. 도메인 검증은 레코드 하나의 크기만 제한하고 행 개수는 제한하지 않는다. 워크스페이스 하나의 제안 수가 커지면 이 조회는 요청 스레드에서 대용량 결과를 파싱한다.운영 관점의 후속 작업으로 두 가지를 권한다.
created_at, proposal_id기준 키셋 페이지네이션을 추가한다. 현재ORDER BY와 정렬 순서가 같으므로 커서 도입이 단순하다.(workspace_id, created_at, proposal_id)복합 인덱스를 확인한다. 이 인덱스는 현재 정렬과 필터를 함께 처리한다.이번 변경은 JSDoc 추가이므로 이 PR에서 반드시 수정할 항목은 아니다.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ai-service/src/postgres-proposal-audit-repository.ts` around lines 303 - 316, Review the listProposals implementation for unbounded result handling, but no code change is required for this JSDoc-only PR. Track keyset pagination using the existing created_at, proposal_id ordering and verify a composite index on workspace_id, created_at, proposal_id as follow-up work.apps/ai-service/src/quality-coverage.test.ts (1)
165-184: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
SequencedSqlClient가 큐 소진 시 조용히 빈 행을 반환한다. 호출 수를 확정하기 바란다.line 178의
this.responses.shift() ?? []는 스크립트한 응답이 모두 소비된 뒤에도 빈 결과를 반환한다. line 629-633과 line 664-669의 테스트는 이 동작에 의존하므로 의도된 설계다.부작용이 하나 있다. 대상 코드가 스크립트한 개수보다 많은 질의를 보내도 테스트는 실패하지 않는다. 추가 질의는 빈 행을 받고, 테스트는 다른 이유로 통과할 수 있다. 이 클래스는
calls를 기록하지만calls.length를 확인하는 assertion이 보이지 않는다.이 PR의 목표는 정확한 보증 증거다. 최소한
appendDecision의 insert-후-replay 경로처럼 질의 순서가 계약인 테스트에서는client.calls.length를 확정하기를 권한다. 이렇게 하면 질의 횟수 변화가 회귀로 드러난다.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ai-service/src/quality-coverage.test.ts` around lines 165 - 184, Update the tests covering appendDecision’s insert-then-replay path to assert the expected client.calls.length, using SequencedSqlClient’s recorded calls to enforce the scripted query count and order. Keep the existing empty-response fallback unchanged because the tests at the referenced scenarios intentionally depend on it.apps/ai-service/src/server.test.ts (1)
4-13: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win모듈 mock 정리를
afterEach로 옮기기 바란다.
vi.doUnmock('./main')이 테스트 본문 마지막 줄에 있다. line 11의 assertion이 실패하면 이 줄은 실행되지 않는다. 그러면 './main' mock이 파일의 남은 실행 구간에 계속 등록된 상태로 남는다.현재 이 파일에는 테스트가 하나뿐이므로 실제 영향은 없다. 다만 누구든 두 번째 테스트를 추가하면 그 테스트가 mock된 './main'을 받게 된다.
afterEach는 assertion 결과와 무관하게 정리를 보장한다.vi.resetModules()도 함께 호출하면 모듈 레지스트리 캐시까지 정리된다.♻️ 테스트 격리 보장 제안
-import { describe, expect, it, vi } from 'vitest'; +import { afterEach, describe, expect, it, vi } from 'vitest'; describe('AI process entrypoint', () => { + afterEach(() => { + vi.doUnmock('./main'); + vi.resetModules(); + }); + it('delegates process startup to the tested bootstrap boundary', async () => { vi.resetModules(); const bootstrapAiService = vi.fn().mockResolvedValue(undefined); vi.doMock('./main', () => ({ bootstrapAiService })); await import('./server'); expect(bootstrapAiService).toHaveBeenCalledOnce(); - vi.doUnmock('./main'); }); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ai-service/src/server.test.ts` around lines 4 - 13, Move the ./main mock cleanup from the test body into an afterEach hook, and call vi.resetModules() there as well so cleanup runs even when assertions fail and clears the module registry for subsequent tests. Remove the now-redundant inline vi.doUnmock('./main') while preserving the existing bootstrapAiService test behavior.apps/ai-service/src/server.ts (1)
4-4: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win명시적 시작 실패 처리 추가
void bootstrapAiService()는 반환된 Promise를 버린다.resolveAiServicePort()가 유효하지 않은 포트 값으로 오류를 던지거나app.listen()이 EADDRINUSE로 거부되면, 미처리 거부(unhandled rejection)가 발생한다.Node.js 22.0.0 이상은 기본값
--unhandled-rejections=throw로 미처리 거부 시 프로세스를 0이 아닌 코드로 종료한다. 이 경우 프로세스는 종료되지만 운영자는 미처리 거부 스택 추적만 본다. 명시적.catch()핸들러를 추가하면 실패 원인을 구조화된 로그로 기록하고 종료 코드를 명확히 설정한다.제안된 수정
import 'reflect-metadata'; +import { Logger } from '`@nestjs/common`'; import { bootstrapAiService } from './main'; -void bootstrapAiService(); +void bootstrapAiService().catch((error: unknown) => { + new Logger('AiServiceEntrypoint').error( + 'AI service failed to start', + error instanceof Error ? error.stack : undefined, + ); + process.exitCode = 1; +});🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ai-service/src/server.ts` at line 4, Update the bootstrapAiService invocation to attach an explicit catch handler for startup failures, including errors from resolveAiServicePort and app.listen. Log the failure through the service’s established structured logger and terminate the process with a non-zero exit code after recording the error.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/ai-service/src/quality-coverage.test.ts`:
- Around line 870-895: Update the logging redaction assertion in the audit
failure test around logger.mock.calls so it inspects each logged argument,
including Error.message and Error.stack, rather than relying only on
JSON.stringify(logger.mock.calls). Ensure the assertion rejects password=secret
in both ordinary logged values and Error details, while preserving the existing
test cases and logger cleanup.
---
Nitpick comments:
In `@apps/ai-service/package.json`:
- Line 8: Update the ai-service lint script to remove the three ../../docs
document paths, keeping it limited to package-owned files and existing
TypeScript checks. Add those documents to the repository-root package.json
format/lint target so their formatting is validated centrally.
In `@apps/ai-service/src/postgres-proposal-audit-repository.ts`:
- Around line 303-316: Review the listProposals implementation for unbounded
result handling, but no code change is required for this JSDoc-only PR. Track
keyset pagination using the existing created_at, proposal_id ordering and verify
a composite index on workspace_id, created_at, proposal_id as follow-up work.
In `@apps/ai-service/src/quality-coverage.test.ts`:
- Around line 165-184: Update the tests covering appendDecision’s
insert-then-replay path to assert the expected client.calls.length, using
SequencedSqlClient’s recorded calls to enforce the scripted query count and
order. Keep the existing empty-response fallback unchanged because the tests at
the referenced scenarios intentionally depend on it.
In `@apps/ai-service/src/server.test.ts`:
- Around line 4-13: Move the ./main mock cleanup from the test body into an
afterEach hook, and call vi.resetModules() there as well so cleanup runs even
when assertions fail and clears the module registry for subsequent tests. Remove
the now-redundant inline vi.doUnmock('./main') while preserving the existing
bootstrapAiService test behavior.
In `@apps/ai-service/src/server.ts`:
- Line 4: Update the bootstrapAiService invocation to attach an explicit catch
handler for startup failures, including errors from resolveAiServicePort and
app.listen. Log the failure through the service’s established structured logger
and terminate the process with a non-zero exit code after recording the error.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: ea08fed1-ded8-4070-89a6-219923c8bc33
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (23)
.github/workflows/ci.yml.gitignoreCHANGELOG.mdapps/ai-service/package.jsonapps/ai-service/src/ai-runtime.tsapps/ai-service/src/docstring-coverage.test.tsapps/ai-service/src/main.tsapps/ai-service/src/postgres-proposal-audit-repository.tsapps/ai-service/src/proposal-audit-application.tsapps/ai-service/src/proposal-audit-domain.test.tsapps/ai-service/src/proposal-audit-domain.tsapps/ai-service/src/proposal-service.tsapps/ai-service/src/quality-coverage.test.tsapps/ai-service/src/server.test.tsapps/ai-service/src/server.tsapps/ai-service/vitest.config.tsapps/notification-service/src/postgres-reminder-repository.integration.test.tsdocs/operations/ai-proposal-audit-assurance.mddocs/superpowers/plans/2026-08-04-ai-service-quality-gates.mddocs/superpowers/specs/2026-08-04-ai-service-quality-gates-design.mdpackage.jsonproduct/capabilities.jsonturbo.json
Comment on lines
+870
to
+895
| it('maps every audit application failure without credential details', async () => { | ||
| const logger = vi | ||
| .spyOn(Logger.prototype, 'error') | ||
| .mockImplementation(() => undefined); | ||
| const cases: Array<[unknown, number, string]> = [ | ||
| [new ProposalValidationError(), 400, 'invalid_request'], | ||
| [new ProposalAuditValidationError(), 400, 'invalid_request'], | ||
| [new ProposalAuditNotFoundError(), 404, 'proposal_not_found'], | ||
| [new ProposalDigestMismatchError(), 409, 'stale_proposal'], | ||
| [new ProposalDecisionConflictError(), 409, 'idempotency_conflict'], | ||
| [new ProposalAuditPersistenceError(), 503, 'audit_unavailable'], | ||
| [new Error('password=secret'), 503, 'audit_unavailable'], | ||
| ['password=secret', 503, 'audit_unavailable'], | ||
| ]; | ||
| for (const [error, status, code] of cases) { | ||
| await expectProblem( | ||
| new AiProposalAuditController(throwingApplication(error)).listProposals( | ||
| WORKSPACE_ID, | ||
| ), | ||
| status, | ||
| code, | ||
| ); | ||
| } | ||
| expect(JSON.stringify(logger.mock.calls)).not.toContain('password=secret'); | ||
| logger.mockRestore(); | ||
| }); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- Logger.error usage and test assertion context ---'
rg -n -A8 -B8 "JSON\.stringify\(logger\.mock\.calls\)|Logger\.prototype\.error|password=secret" apps/ai-service/src
printf '%s\n' '--- Standalone Error serialization probe ---'
node - <<'JS'
const error = new Error('password=secret');
console.log(JSON.stringify([error]));
console.log(Object.prototype.propertyIsEnumerable.call(error, 'message'));
console.log(Object.prototype.propertyIsEnumerable.call(error, 'stack'));
console.log(JSON.stringify(error, ['name', 'message', 'stack']));
JSRepository: ContextualWisdomLab/life-os
Length of output: 6494
Sensitive Data Exposure (CWE-532): Insertion of Sensitive Information into Log File
Reachability: Internal · Exploitability: Theoretical
Reachability path
● Entry
apps/ai-service/src/server.test.ts
│
▼
● Hop
apps/ai-service/src/server.ts
│
▼
● Hop
apps/ai-service/src/main.ts
│
▼
● Hop
apps/ai-service/src/proposal-audit-domain.ts
│
▼
● Hop
apps/ai-service/src/proposal-service.ts:282
ProposalService: Creates the generator with explicit model, clock, and identifier seams.
│
▼
● Sink
apps/ai-service/src/quality-coverage.test.ts
Error의 message와 stack도 redaction assertion에 포함하세요.
JSON.stringify(logger.mock.calls)는 Error를 {}로 직렬화합니다. 따라서 new Error('password=secret')가 raw 객체로 로깅되어도 assertion이 통과합니다. 로깅 인자를 펼쳐 Error.message와 Error.stack을 검사해야 합니다.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/ai-service/src/quality-coverage.test.ts` around lines 870 - 895, Update
the logging redaction assertion in the audit failure test around
logger.mock.calls so it inspects each logged argument, including Error.message
and Error.stack, rather than relying only on JSON.stringify(logger.mock.calls).
Ensure the assertion rejects password=secret in both ordinary logged values and
Error details, while preserving the existing test cases and logger cleanup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Buyer-visible outcome
Makes the production AI proposal-audit bounded context independently verifiable: every production declaration is documented, every executable production path is covered, and the operator assurance boundary is grounded in current AI-governance and HTTP standards.
Implemented outcome
apps/ai-service/srcserver.tsentrypointquality.ai-audit-assurancecommercial-readiness capability and matchingCHANGELOG.mdevidenceAI_TEST_DATABASE_URL, so the full monorepo gate executes rather than silently skipping the real AI PostgreSQL/HTTP suiteArchitecture boundary
The AI service remains independently deployable and receives no planning, calendar, habit, identity, notification, generic mutation repository, command bus, or execution dependency. An accepted proposal decision remains append-only audit evidence only; it cannot apply or execute an operation.
No NVIDIA NIM or external model call is introduced in this slice because the assurance gate is deterministic and provider-independent. External-model integration remains subject to separate accuracy, robustness, privacy, prompt-injection, latency, and cost evidence using the configured secret boundary.
Fresh verification before PR
The complete repository gate passed before the temporary repair automation removed itself:
Exact-head merge gate
CI, AppGuardrail, SAST Semgrep, Security Scan, Commercial Readiness, CodeRabbit, and every actionable review thread must pass on the exact current head before squash merge.
Closes #107.
Summary by CodeRabbit
새 기능
품질 개선
문서
빌드 및 테스트