Information about pypdfium2 functionalities for working with PDF text

[EmanueleGusso]

Dear @adam-huganir and @mara004
First of all I'd like to thank you for your amazing work.
I'm looking for a library which is able to identify the bounding boxes of characters and text lines in a pdf in order to obtain:

1. the coordinates of each character;
2. the coordinates of each text line ordered in reading order;
3. the textual content of the text lines.

This functionalities are provided, for example, by https://github.com/pdfminer/pdfminer.six but I am looking for an alternative and I read about foxit and then I read about you.
Reading the foxit documentation (http://cdn01.foxitsoftware.com/pub/foxit/manual/enu/FoxitPDF_SDK20_Guide.pdf?fbclid=IwAR0jfV9RyMx2pgBzboe8YRnPwoD98Fdp1XNYQBn-BGOXev3K4V-kJEDV7iQ) it seems like the three features I mentioned are provided but it's not clear whether foxit is free or not.
So I'm writing you to ask if pypdfium2 is able to fulfill those 3 requirements.
I really hope you can answer me and I really thank you in advance for your availability.
Emanuele

[mara004]

Dear @EmanueleGusso,

Thanks for your interest in pypdfium2. I haven't used the text functions of PDFium yet, but since Chromium is able to mark and extract text, I would presume that it's possible to get all required information using the low-level ctypes interface of pypdfium2. I think you may be looking for the FPDFText_...() functions?

1. Probably FPDFText_GetCharBox().
2. FPDFText_GetRect(). (However, I belive this is based on heuristics, since PDFs only store the position of each text glpyh, but there's no information on lines or text groups.)
3. FPDFText_GetBoundedText() may be used to obtain text of a certain area on a page.

I recommend that you take a look at pdf_text_page.py of pdfbrain, which implements a support model for working with the PDFium text API.

Anything of this section in the Foxit documentation is freely available in the open-source PDFium. That said, I think pdfminer.six has an excellent reputation for tasks related to PDF texts. Why are you looking for an alternative, if I may ask?

[EmanueleGusso]

Dear @mara004
First of all thank you very much for your answer. You clarified a few things and I'll have a look at pdfbrain before trying to use directly pdfium because I'd rather stick with the Python language.
You're right, pdfminer has an excellent reputation but I am looking for the most reliable library and I found some issues with pdfminer (like this one pdfminer/pdfminer.six#281 ).
So, I'm afraid that there are some pdfs which, for unknown reasons, aren't properly handled by pdfminer.
When I read about PDFium I read that it is now maintained by Google so I immediately thought that PDFium should be the most reliable thing out there. What do you think about this thought?
For sure you are more experienced than me on this subject so I am open to all your suggestions.
Again, thank you very much for your work and your availability.

[mara004]

Dear @EmanueleGusso,

You clarified a few things and I'll have a look at pdfbrain before trying to use directly pdfium because I'd rather stick with the Python language.

Well, you could still use pypdfium2 & Python, but the referenced code of pdfbrain should give you a good idea on how to interface with the mentioned PDFium API functions using ctypes. Let me know if you have any problems with this, and I'll try to help (though my knowledge of ctypes is quite limited yet).

When I read about PDFium I read that it is now maintained by Google so I immediately thought that PDFium should be the most reliable thing out there. What do you think about this thought?

I'd recommend that you try opening some of your sample documents (on which pdfminer.six does not work properly) in Chromium to see if PDFium detects text positions correctly.

[mara004]

I have now added a support model for text extraction with commits bbc2438 and 58396a2.