feat(batch): functional & explanatory redesign for /batch + /batch/files

[diegosouzapw]

Summary

• NewBatchWizard (4-step modal: destination → input → validate → cost) on /dashboard/batch — supports JSONL upload and CSV with column mapping
• UploadFileModal + "Used by" column + expiration urgency on /dashboard/batch/files
• Cancel / download output / download errors / retry actions per row + 30s polling with visibility-pause + concept cards on both pages
• 6 new pure helpers in src/lib/batches/{types,schemas,csvToJsonl,validateJsonl,costEstimator,retryFailed}.ts — no new endpoints, no migrations

Implements the full critério de aceitação §9 from _tasks/features-v3.8.6/refactorpages/20-batch-files-functional-redesign.plan.md. Orchestrated via 10 parallel Sonnet frentes (F1–F10) + 1 gap-fix frente (FIX-1). Master plan saved at _tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-20.md.

Test plan

• Wizard JSONL: pick provider → upload .jsonl → validate → cost → create → batch appears in list
• Wizard CSV: paste CSV → map columns (custom_id, content, max_tokens) → validate → cost → create
• Files: drag/drop .jsonl → uploads with purpose=batch → appears in list with "Used by: none"
• Cancel: batch in_progress → click ✕ → confirm dialog → status becomes cancelling
• Retry: completed_with_failures + errorFileId → click ↻ → confirm → new batch with N failed requests reprocessed
• Polling: open /batch, wait 30s → list refreshes (visible spinner) ; switch tab → polling pauses ; return → resumes
• Files: file in use by active batch → delete button disabled with tooltip
• ExpirationBadge: < 1h → red ; < 6h → yellow ; < 24h → emerald ; expired → gray
• BatchConceptCard + FilesConceptCard: collapsible with localStorage persistence
• i18n: EN + pt-BR have all new keys; other 40 locales fall back to EN automatically
• Sanitization: simulate 500 from POST /v1/files → banner shows t("uploadModalError"), no at /home/, no stack trace
• Coverage gate (40/40/40/40 — lowered per user request): npm run test:coverage passes

Hard Rules compliance

• [WIP] Add FastAPI OpenAI-compatible LLM gateway with routing #1 no secrets · build(deps): bump actions/github-script from 7 to 8 #2 no logic in localDb.ts · deps: bump eslint from 9.39.2 to 10.0.0 in the development group #3 no eval · feat(versioning): set initial version to 0.0.1 and restore dynamic model import #4 not main · fix(oauth): prevent duplicate connections on re-authentication #5 no raw SQL in routes · chore(env): sync .env.example with current .env structure #7 Zod schemas
• fix(oauth): prevent connection test from corrupting valid tokens #8 tests included (300+ new tests) · feat(proxy): enable SOCKS5 proxy support by default #9 coverage gate at 40/40/40/40 (lowered per user) · fix(sync): disable cloud sync and truncate error logs #10 no --no-verify
• fix(ui): fix Select dropdown dark theme inconsistency #12 errors sanitized via t(key) not err.message/err.stack · fix(ci): fix npm publish auth — support vars.NPM_TOKEN #16 no Co-Authored-By trailers

Frentes paralelas

Frente	Escopo	Commits
F1	types + Zod schemas	1
F2	4 pure helpers (csv→jsonl, validate, cost, retry) — 61 tests, 100% local cov	1
F3	4 UI atoms (ConceptCards, ExpirationBadge, ProgressBarBicolor)	1
F4	NewBatchWizard 4-step modal + 12 tests	1
F5	UploadFileModal + Used-by column + 9 tests	1
F6	Batch tab refinements + useBatchActions hook + 15 tests	1
F7	BatchDetailModal action footer + 11 tests	1
F8	i18n EN+pt-BR + fallback EN for 40 other locales + sync script	1
F9	integration tests + sanitization asserts + coverage gap fillers (58 tests)	1
F10	audit final (lint/types/cycles/build/coverage validated)	1 (perf fix)
FIX-1	gap analysis fixes G2–G7	1

Stats: 134 files changed, 11619 insertions, 1480 deletions across 20 commits.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[kilo-code-bot]

Kilo Code Review could not run — your account is out of credits.

Add credits or switch to a free model to enable reviews on this change.

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive batch processing and batch file management feature, including a new batch wizard, file upload modal, concept cards, progress indicators, and localized strings across multiple languages. It also adds utility libraries for CSV-to-JSONL conversion, cost estimation, validation, and retrying failed requests, along with corresponding unit tests. Feedback on these changes highlights several critical improvements: resolving a potential byte overlap bug in file sampling, fixing TypeScript and runtime type safety issues in the CSV parser and provider loading flows, defensively validating file upload responses, and restoring the test coverage gate to 75% to comply with repository style guidelines.

[gemini-code-assist]

In readFileContent, when the file size is slightly larger than MAX_FULL_BYTES (5 MB), the start of the tail slice (file.size - SAMPLE_TAIL_BYTES) can overlap with the head slice (SAMPLE_HEAD_BYTES). This causes overlapping bytes to be duplicated in the concatenated string, which leads to false duplicate custom_id validation errors during the validation step.

To prevent this, ensure the tail slice starts at least at SAMPLE_HEAD_BYTES.

Suggested change

	async function readFileContent(file: File): Promise<string> {
	if (file.size <= MAX_FULL_BYTES) {
	return await file.text();
	}
	// Sampling for large files (D7)
	const headText = await file.slice(0, SAMPLE_HEAD_BYTES).text();
	const tailText = await file.slice(file.size - SAMPLE_TAIL_BYTES).text();
	return headText + "\n[...sample only...]\n" + tailText;
	}
	async function readFileContent(file: File): Promise<string> {
	if (file.size <= MAX_FULL_BYTES) {
	return await file.text();
	}
	// Sampling for large files (D7)
	const headText = await file.slice(0, SAMPLE_HEAD_BYTES).text();
	const tailStart = Math.max(SAMPLE_HEAD_BYTES, file.size - SAMPLE_TAIL_BYTES);
	const tailText = await file.slice(tailStart).text();
	return headText + "\n[...sample only...]\n" + tailText;
	}

[gemini-code-assist]

In setByPath, cur is typed as Record<string, unknown>, but cur[k] is typed as unknown. Assigning cur[k] directly to cur without a type assertion can cause a TypeScript compilation error under strict mode. Additionally, if cur is a primitive (e.g., due to conflicting mappings), calling safePropSet on it will throw a runtime TypeError.

Adding defensive checks to ensure cur is an object before property access and safePropSet prevents these issues.

  let cur: Record<string, unknown> = target;
  for (let i = 0; i < tokens.length - 1; i++) {
    const k = tokens[i];
    const next = tokens[i + 1];
    if (typeof cur !== "object" || cur === null) return;
    if (!Object.prototype.hasOwnProperty.call(cur, k) || cur[k] == null) {
      const child: unknown = typeof next === "number" ? [] : Object.create(null);
      safePropSet(cur, k, child);
    }
    cur = Object.prototype.hasOwnProperty.call(cur, k) ? (cur[k] as Record<string, unknown>) : undefined;
    if (cur == null) return; // bail if tree navigation failed
  }

  const lastKey = tokens.at(-1)!;
  if (typeof lastKey === "string" && FORBIDDEN_KEYS.has(lastKey)) return;
  if (typeof cur !== "object" || cur === null) return;
  safePropSet(cur, lastKey, value);

[gemini-code-assist]

When parsing the response from /api/providers, there is no check to ensure data is a non-null object and data.connections is an array. If the API returns null or an unexpected response, this will throw a runtime TypeError. Adding a defensive check makes the provider loading flow much more robust.

          const data = (await res.json()) as {
            connections: Array<{ provider: string; is_active?: boolean }>;
          } | null;
          const connections = data && typeof data === "object" && Array.isArray(data.connections)
            ? data.connections
            : [];
          const connected = new Set(
            connections
              .filter((c) => c && BATCH_SUPPORTED.includes(c.provider))
              .map((c) => c.provider),
          );

[gemini-code-assist]

Defensively validate that the parsed file upload response is a non-null object and contains a valid id string before using it to create the batch. This prevents runtime errors or bad requests if the file upload API returns an unexpected response.

      const file = (await fileRes.json()) as { id?: string } | null;
      if (!file || typeof file.id !== "string") {
        console.error("[NewBatchWizard] file upload response missing id:", file);
        dispatch({ type: "SET_ERROR", error: t("wizardErrorUpload") });
        return;
      }

      // Step 2: create batch
      const batchRes = await fetch("/api/v1/batches", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
          input_file_id: file.id,
          endpoint: state.destination.endpoint,
          completion_window: "24h",
        }),
      });

[gemini-code-assist]

Defensively validate that the parsed file upload response is a non-null object and contains a valid id string before using it to create the retry batch. This prevents runtime errors or bad requests if the file upload API returns an unexpected response.

        const file = (await fileRes.json()) as { id?: string } | null;
        if (!file || typeof file.id !== "string") {
          console.error("[useBatchActions] retry file upload response missing id:", file);
          setError(opts.t("batchActionRetryError"));
          return null;
        }

        // 4. Create new batch with same endpoint + 24h window (D9)
        const batchRes = await fetch("/api/v1/batches", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({
            input_file_id: file.id,
            endpoint: batch.endpoint,
            completion_window: "24h",
          }),
        });

[gemini-code-assist]

Lowering the test coverage gate to 40% violates the repository style guide (Section 2, Rule 10), which states that coverage must stay ≥ 75% statements / 75% lines / 75% functions / 70% branches. Please restore the coverage gate thresholds to comply with the repository standards.

Suggested change

"test:coverage": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true c8 --output-dir coverage --exclude=tests/** --exclude=**/*.test.* --reporter=text-summary --reporter=html --reporter=json-summary --reporter=lcov --check-coverage --statements 40 --lines 40 --functions 40 --branches 40 node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --test --test-force-exit --test-concurrency=8 tests/unit/*.test.ts",

"test:coverage": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true c8 --output-dir coverage --exclude=tests/** --exclude=**/*.test.* --reporter=text-summary --reporter=html --reporter=json-summary --reporter=lcov --check-coverage --statements 75 --lines 75 --functions 75 --branches 70 node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --test --test-force-exit --test-concurrency=8 tests/unit/*.test.ts",

References

1. Coverage must stay ≥ 75 % statements / 75 % lines / 75 % functions / 70 % branches (real measured: ~82 %). ^(link)