feature: upload from pdf2md incrementallyu rather than just at the end

[densumesh]

Description

<replace w/ a helpful description of your issue, including steps to replicate when/if relevant>

Target(s)

<replace w/ name of the service(s) which are associated with this issue>

Community channels

Matrix is preferred. Reach out on discord or Matrix for further assistance.

• discord
• matrix

[vtempest]

https://airesearch.js.org/functions/convertPDFToHTML.html
I've checked out the pdf2md but think how will that scale to 10k PDFs
That's why you need a hybrid use my 0$ version which also does reference extraction
Then ocr images tables only if needed by the prompt about that chunk. But not summarize or topicify types of prompts. Otherwise this is a very expensive overkill to use gpt token many PDFs

Nick Khami (@skeptrune) is probably telling you all to ingore me which is violating open source code of conduct for welcoming environment when he passive aggressively looks for bs excuse to block me ignoring my years of research ideas many of which he later adopted like switching to HF vector and pdf2md. This is not the way to handle it hoping I go away instead talk it out apologize in the spirit of the holidays and include qualified developers.