-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Labels
benchmark/evalRelated to benchmark/evalsRelated to benchmark/evalsdevRelated to general dev tasksRelated to general dev tasksenhancementNew feature or requestNew feature or request
Description
- Support Gemini models: Refactor the
parse_with_gemini
function such that the API call goes throughcreate_response
- Create tests for the functionality. Can create a separate file (e.g.
tests/test_schema_parser.py
) - Verify and fix errors (likely with using
max_tokens
) when using o-series models from OpenAI - Check if there are any benchmarks for a similar functionality. If not, start building a dataset for evaluation.
- Add support for
dataclass
objects, potentially with additional attributes forsample_values
(examples of possible values) andalternate_keys
(examples of alternative names for each attribute to support robustness across unstandardized documents).
We can create separate issues for any of the above if needed.
cc: @Vaishnav2804
Metadata
Metadata
Assignees
Labels
benchmark/evalRelated to benchmark/evalsRelated to benchmark/evalsdevRelated to general dev tasksRelated to general dev tasksenhancementNew feature or requestNew feature or request