feat: add document metadata expansion#161

Merged

lsorber merged 19 commits intomainfrom

eb-preprocessing-callbacks

Oct 18, 2025

Commits on Oct 18, 2025

feat: add callbacks for metadata generation

emilradix
authored and
lsorber
committed
fix: input chunk instead of document to chunk_metadata_function

SimonJasansky
authored and
lsorber
committed
feat: add public function to extract metadata using an LLM call

SimonJasansky
authored and
lsorber
committed
fix: remove doctest-formatted docstring

SimonJasansky
authored and
lsorber
committed
feat: add public expand_document_metadata() function

SimonJasansky
authored and
lsorber
committed
feat: copilot suggestions and formatting improvements

SimonJasansky
authored and
lsorber
committed
fix: allow free-text lists, improve docstring, improve logging, improve code documentation

SimonJasansky
authored and
lsorber
committed
refactor: use mapping instead of multiple if/else statements

SimonJasansky
authored and
lsorber
committed
feat: add parameter max_chars to metadata_fields to optionally limit the length of the document text the llm receives

SimonJasansky
authored and
lsorber
committed
style: end code comments with periods

SimonJasansky
authored and
lsorber
committed
feat(expand_document_metadata): add kwargs argument that is passed to litellms batch_completion

SimonJasansky
authored and
lsorber
committed
feat: improve system prompt

SimonJasansky
authored and
lsorber
committed
refactor(expand_document_metadata): remove private helper functions, move field specs to system prompt, add argument content_char_limit

SimonJasansky
authored and
lsorber
committed
docs: fix docstring in expand_document_metadata

SimonJasansky
authored and
lsorber
committed
feat(expand_document_metadata): add MetadataField dataclass

SimonJasansky
authored and
lsorber
committed
feat: use annotated to associate prompts

emilradix
authored and
lsorber
committed
chore: rename to max_context_size

emilradix
authored and
lsorber
committed
feat: address review comments
lsorber
committed
fix: fix linting issues
lsorber
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add document metadata expansion#161

feat: add document metadata expansion#161
lsorber merged 19 commits intomainfrom
eb-preprocessing-callbacks

Commits on Oct 18, 2025

feat: add callbacks for metadata generation

fix: input chunk instead of document to chunk_metadata_function

feat: add public function to extract metadata using an LLM call

fix: remove doctest-formatted docstring

feat: add public expand_document_metadata() function

feat: copilot suggestions and formatting improvements

fix: allow free-text lists, improve docstring, improve logging, improve code documentation

refactor: use mapping instead of multiple if/else statements

feat: add parameter max_chars to metadata_fields to optionally limit the length of the document text the llm receives

style: end code comments with periods

feat(expand_document_metadata): add kwargs argument that is passed to litellms batch_completion

feat: improve system prompt

refactor(expand_document_metadata): remove private helper functions, move field specs to system prompt, add argument content_char_limit

docs: fix docstring in expand_document_metadata

feat(expand_document_metadata): add MetadataField dataclass

feat: use annotated to associate prompts

chore: rename to max_context_size

feat: address review comments

fix: fix linting issues

feat: add document metadata expansion#161lsorber merged 19 commits intomainsuperlinear-ai/raglite:mainfrom eb-preprocessing-callbackssuperlinear-ai/raglite:eb-preprocessing-callbacksCopy head branch name to clipboard

Commits

Commits on Oct 18, 2025

feat: add document metadata expansion#161
lsorber merged 19 commits intomainfrom
eb-preprocessing-callbacks