Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add options for choosing OCR engines #118

Merged
merged 40 commits into from
Oct 8, 2024
Merged

feat: add options for choosing OCR engines #118

merged 40 commits into from
Oct 8, 2024

Conversation

dolfim-ibm
Copy link
Contributor

@dolfim-ibm dolfim-ibm commented Oct 2, 2024 •
edited
Loading

Issue resolved by this Pull Request:
Resolves #119, #130

Checklist:

  • Commit Message Formatting: Commit titles and messages follow guidelines in the
    conventional commits.
  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

@nikos-livathinos nikos-livathinos self-requested a review October 2, 2024 16:30
dolfim-ibm and others added 11 commits October 3, 2024 13:38
@dolfim-ibm
add message for failed easyocr import
81d176c 
Signed-off-by: Michele Dolfi <[email protected]>
@nikos-livathinos
fix(TesserOcrModel): Refactor code to catch exception in case of impo…
1d4517f 
…rt error

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
chore(data_scanned): Simplify the OCR test images. Add GT for easyocr…
7ab3b62 
…, tesserocr, tesseract

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(tests): Extend test_e2e_ocr_conversion to cover all OCR engines (…
e571ab5 
…easyocr, tesserocr, tesseract)

Signed-off-by: Nikos Livathinos <[email protected]>
@dolfim-ibm
add tesseract in CI, improve error messages and allow to specify the …
f57e4b2 
…tesseract cmd

Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
Merge remote-tracking branch 'origin/main' into feat-multiple-ocr-eng…
2784d9c 
…ines
@dolfim-ibm
update test results
b3293ff 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
add missing install
544f298 
Signed-off-by: Michele Dolfi <[email protected]>
@nikos-livathinos
feat(tests): Introduce fuzzy text comparison for OCR tests based on L…
49652ee 
…evenshtein edit distance

Signed-off-by: Nikos Livathinos <[email protected]>
@dolfim-ibm
add tesseract language packages
99dfbf6 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
expose easyocr arguments
9eb3afc 
Signed-off-by: Michele Dolfi <[email protected]>
cau-git
cau-git reviewed Oct 7, 2024
View reviewed changes
docling/cli/main.py Outdated Show resolved Hide resolved
@cau-git
Copy link
Contributor

cau-git commented Oct 7, 2024 •
edited by nikos-livathinos
Loading

Additional steps:

  • Document TESSDATA_PREFIX for OCR
  • Accept to override it on TesserOcrOptions (like with EasyOCR)
  • Inject static ENV var TESSDATA_PREFIX into CI
  • Create simple test sample that never fails on levensthein distance

nikos-livathinos and others added 4 commits October 7, 2024 17:26
@nikos-livathinos
fix(BoundingBox): Fixing the BoundingBox.area() method to work for al…
a9b22a8 
…l values of CoordOrigin

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(OCR): Skip zero area OCR cells for all OCR engines
6faff14 
Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(CI/CD): Add envvar TESSDATA_PREFIX in the checks.yml to ensure th…
bd1837f 
…at tesseract has the proper path for the language models.

Signed-off-by: Nikos Livathinos <[email protected]>
@dolfim-ibm
fix tessdata env
7532ede 
Signed-off-by: Michele Dolfi <[email protected]>
nikos-livathinos and others added 13 commits October 8, 2024 07:07
@nikos-livathinos
fix(tests): Refactor the data_scanned with a very simple document tha…
be6489b 
…t allows all OCR engines to produce the same result.

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(CI/CD): Use the eng language package location to set the TESSDATA…
8ec8c38 
…_PREFIX envvar

Signed-off-by: Nikos Livathinos <[email protected]>
@dolfim-ibm
reorder sections in custom_convert
471daee 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
docs: explain OCR options
73108d5 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
Merge remote-tracking branch 'origin/main' into feat-multiple-ocr-eng…
6774604 
…ines
@dolfim-ibm
remove pydantic warning for model_
ea3f720 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
add docs for TESSDATA_PREFIX
5bd6477 
Signed-off-by: Michele Dolfi <[email protected]>
@nikos-livathinos
fix(test): Update test data for OCR
072aaf6 
Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(test): Introduce parameter in verify_conversion_result() to all…
29e65e9 
…ow skipping the verification of the cells. It is used in case of OCR tests.

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix(TesserOcrModel): Fix cell coordinates
118afee 
Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
feat(OCR): Introduce support for the language path in the pipelines…
074acd7 
… of both Tesseract OCR engines.

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
chore(OCR): Rename class names to use Tesseract for the tesserocr and…
70a8a2c 
… TesseractCLI for the tesseract process

Signed-off-by: Nikos Livathinos <[email protected]>
@nikos-livathinos
fix: Rename the tesseract OCR related classes and filenames
bb8cd0f 
Signed-off-by: Nikos Livathinos <[email protected]>
@cau-git cau-git marked this pull request as ready for review October 8, 2024 16:31
cau-git
cau-git previously approved these changes Oct 8, 2024
View reviewed changes
Copy link
Contributor

@cau-git cau-git left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Great work!

@dolfim-ibm
Copy link
Contributor Author

This PR is also fixing the issue identified in #130.

@cau-git cau-git mentioned this pull request Oct 8, 2024
Closed
@dolfim-ibm
keep only one example
800b16b 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
missing one part of the comment
3d66062 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm dolfim-ibm merged commit f96ea86 into main Oct 8, 2024
4 checks passed
@dolfim-ibm dolfim-ibm deleted the feat-multiple-ocr-engines branch October 8, 2024 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cau-git cau-git cau-git approved these changes

@PeterStaar-IBM PeterStaar-IBM Awaiting requested review from PeterStaar-IBM

@nikos-livathinos nikos-livathinos Awaiting requested review from nikos-livathinos

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add support for Tesseract as OCR
4 participants
@dolfim-ibm @cau-git @nikos-livathinos @PeterStaar-IBM