better pdf parsing #754
better pdf parsing #754
Conversation
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
❌ Changes requested. Reviewed everything up to 57a8c21 in 1.0 minute and 6.655169999999998 seconds
More details
- Looked at
409lines of code in10files - Skipped
2files when reviewing. - Skipped posting
0drafted comments based on config settings.
Workflow ID: wflow_3Jv7F8oL8WjV6Qg4
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
|
|
||
| print(f"Overriding parser for {parser_override['document_type']} with {parser_override['parser']}") | ||
|
|
||
| for doc_type, parser_infos in self.AVAILABLE_PARSERS.items(): |
There was a problem hiding this comment.
The implementation of ParsingPipe does not correctly handle multiple parsers for a single document type. It always selects the first parser in the list, which might not be the intended behavior if specific parsers are better suited for certain PDFs. Consider implementing a mechanism to select the appropriate parser based on the document's characteristics or user configuration.
Summary:
Introduced new PDF parsers and updated the parsing pipeline to support multiple and overrideable parsers.
Key points:
PDFParserUnstructured,PDFParserLocal,PDFParserVLM, andPDFParserMarkerinr2r/parsers/media/pdf_parser.py.__init__.pyfiles inr2randr2r/parsersto include new PDF parsers.r2r/main/assembly/factory.pyto passoverride_parserstocreate_parsing_pipe.ParsingPipeinr2r/pipes/ingestion/parsing_pipe.pyto support multiple parsers for a document type and handle parser overrides.ParserComponentLayoutExtractionandParserComponentOCRinr2r/parsers/common/layout_extraction.pyfor layout extraction and OCR.IngestionServiceinr2r/main/services/ingestion_service.pyto remove unnecessary blank line.Generated with ❤️ by ellipsis.dev