feat(partitions): scope chunk listing to a single file (fix O(partition) detail view)

[andyne13]

What

Adds an optional file_id (and limit) parameter to GET /partition/{p}/chunks so the document detail view can fetch a single file's chunks with content in one bounded, server-filtered call.

Fixes #514.

Why

The detail view's chunk loading was O(partition): the only content-returning chunk endpoint was partition-wide, and the UI filtered client-side by file_id — so opening one document downloaded every chunk's text in the partition (the endpoint even warns about large payloads). This is the "Loading file…" lag that grows with partition size.

Change

• PartitionService.list_all_chunks(partition, include_embedding=True, file_id=None, limit=None) — when file_id is set it's added to the vector-store filter (query_chunks_by_filter({partition, file_id})), pushing the work down to Milvus; limit caps the result. No file_id → identical behavior to before.
• GET /partition/{p}/chunks exposes file_id and limit query params (passthrough). Permissions unchanged (partition viewer+).

The data path already existed (get_file_chunks filters by file_id server-side but drops text); this exposes it on the content endpoint.

Test

Adds service tests: partition-only filter when no file_id, {partition, file_id} filter when given, and limit capping. Suite: 23 passed, ruff clean.

Frontend follow-up

Separate change on the UI branch: listFileChunks switches to …/chunks?file_id=<id>&include_embedding=false and drops the client-side filter.

Summary by CodeRabbit

• New Features

  • Admin chunk listings now support optional filtering by file ID to scope results to a single file.
  • Optional limit is available to cap returned chunks for improved performance on large partitions.

• Bug Fixes

  • Negative limit values are now rejected for both chunk listing and file chunk retrieval.

• Tests

  • Added unit tests covering file ID filter pushdown, output limiting, and validation behavior for negative limits.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e0df641a-c8ea-43a0-a6e0-bca415f534e8

📥 Commits

Reviewing files that changed from the base of the PR and between b30b1fb and df1abe1.

📒 Files selected for processing (3)

• openrag/api/routers/admin/partitions.py
• openrag/services/orchestrators/partition_service.py
• tests/unit/services/orchestrators/test_partition_service.py

🚧 Files skipped from review as they are similar to previous changes (3)

• tests/unit/services/orchestrators/test_partition_service.py
• openrag/api/routers/admin/partitions.py
• openrag/services/orchestrators/partition_service.py

---

📝 Walkthrough

Walkthrough

Two new optional parameters, file_id and limit, are added to PartitionService.list_all_chunks and the corresponding admin API endpoint. A validation helper rejects negative limits in both services. When file_id is supplied, filtering is pushed down to the vector store; limit truncates results post-fetch. Five unit tests verify filter construction, result capping, and rejection of negative limits.

Changes

Per-file chunk filtering and limit

Layer / File(s)	Summary
Limit validation and PartitionService implementation openrag/services/orchestrators/partition_service.py	New _validate_limit(limit) helper rejects negative limits with ValidationError. list_all_chunks gains file_id: str | None and limit: int | None parameters; vector-store filter includes file_id when present, and rows are sliced to limit before response shaping. get_file_chunks calls _validate_limit before querying.
API endpoint parameters, docs, and validation openrag/api/routers/admin/partitions.py	Module imports Query for parameter validation. get_file endpoint limit uses Query(default=2000, ge=0) to enforce non-negative values. list_all_chunks endpoint description updated to document file_id and limit; function signature gains both optional parameters forwarded to the service call.
Fake vector store instrumentation and test coverage tests/unit/services/orchestrators/test_partition_service.py	FakeVectorStore gains last_chunk_filters field and records filters in query_chunks_by_filter for assertions. Five new async tests verify partition-only filtering without file_id, combined partition+file_id filtering when present, limit-based result truncation, and rejection of negative limit (HTTP 422) for both list_all_chunks and get_file_chunks.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 Hop, hop — no more waiting in the queue,
A file_id filter now narrows the view!
The vector store whispers just what you need,
limit clips the pile with cottontail speed.
One file's chunks, not a partition's whole stew —
This rabbit approves of your O(file) debut! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding file_id scoping to chunk listing to fix O(partition) performance in document detail view.
Linked Issues check	✅ Passed	All objectives from issue #514 are met: optional file_id and limit parameters added to chunks endpoint, filtering pushed to vector store, and partition-only default behavior preserved.
Out of Scope Changes check	✅ Passed	All changes directly support the linked issue: endpoint updates for file_id/limit filtering, service layer validation and filtering logic, and comprehensive unit tests.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/per-file-chunks-content

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openrag/services/orchestrators/partition_service.py`:
- Around line 362-364: The `limit` parameter in the partition service method
accepts negative values, which causes unintended behavior when used in list
slicing operations like `rows[:limit]` where a negative limit returns
all-but-tail rows instead of being rejected. Add validation to ensure the
`limit` parameter is either None or a non-negative integer before it is used in
any slicing operations. Additionally, apply the same constraint at the API layer
in the admin partitions router by using `Query(default=None, ge=0)` to enforce
non-negative limits at the endpoint level.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b9df2c16-348e-4403-9311-8430120ae82a

📥 Commits

Reviewing files that changed from the base of the PR and between 14301a4 and b30b1fb.

📒 Files selected for processing (3)

• openrag/api/routers/admin/partitions.py
• openrag/services/orchestrators/partition_service.py
• tests/unit/services/orchestrators/test_partition_service.py