Observing how Detailed Report on local docs happens - and something is not right

[danieldekay]

I have run my Detailed report on my own documents, and have done it with a retriever and without a retriever turned on. Both create usable documents when it comes to the content, but both have sever issues with citing sources - they point to not existing URLs half the time.

Also, the list of references is empty.

In the Debugger

I am currently watching the detailed report happening on my local documents, and wrapping my head around how the detailed report works, actually.

In the first iteration there comes a point where the context becomes garbled.

[danieldekay]

Additionally, I have noticed that it likes to grab the literature appendix from my documents as related content, and not text from the actual article:

[danieldekay]

I have an idea, why the references are not listed: for local documents the URLs are never set to the document file urls, or anything else.

    async def _construct_detailed_report(self, introduction: str, report_body: str) -> str:
        toc = self.gpt_researcher.table_of_contents(report_body)
        conclusion = await self.gpt_researcher.write_report_conclusion(report_body)
        conclusion_with_references = self.gpt_researcher.add_references(
            conclusion, self.gpt_researcher.visited_urls) # the urls are empty for local documents.
        report = f"{introduction}\n\n{toc}\n\n{report_body}\n\n{conclusion_with_references}"
        return report

[ElishaKay]

Green light for the PR 👍