feat: add input chunking to summarize task

[edward-ly]

Closes #150. Lightly tested with gpt-4o, gpt-4o-mini, and gpt-3.5-turbo.

[julien-nc]

👍

The admin setting input could be moved in the "Text generation" section. Wdyt?

[edward-ly]

The admin setting input could be moved in the "Text generation" section. Wdyt?

Maybe the chunk size could also be considered as a usage limit, but since we only use it on the summarize task, I think it would make sense to move it there too.

[marcelklehr]

The code looks really good. The algorithm deviates a little from what llm2 does, though: In llm2 the summaries of the chunks are concatenated and fed through the same algorithm again (ie. chunked and summarized) until there is only one summary left.

[marcelklehr]

Oops, clicked the wrong button.

[edward-ly]

The code looks really good. The algorithm deviates a little from what llm2 does, though: In llm2 the summaries of the chunks are concatenated and fed through the same algorithm again (ie. chunked and summarized) until there is only one summary left.

Now that you mention it, I do see the similar loop in llm2 now, fixed.

[marcelklehr]

I'm about to change the algorithm in llm2 slightly, so we should change it here as well a last time: When the input is shorter than the chunk size it should still go through the llm summarizer once.

[julien-nc]

👍 A few adjustments and good to go!

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

[marcelklehr]

Looks good and works in my tests. It seems we lost multilingual support though. When summarizing a German text that is longer than the chunk size the resulting summary was in english in my test.

[edward-ly]

Looks good and works in my tests. It seems we lost multilingual support though. When summarizing a German text that is longer than the chunk size the resulting summary was in english in my test.

In my testing, summarizing English text produced output in either Spanish or Italian, hence the change in prompt. If you want me to revert that change for now, we can do that.

[marcelklehr]

In my testing, summarizing English text produced output in either Spanish or Italian

Oh, that's not intended of course :/ Ideally we should find a prompt that works for all languages to produce the summary in the same language as the input.

[edward-ly]

Changed the prompt again after some basic prompt engineering and experimenting. Hopefully, this will produce more reliable results.

[marcelklehr]

Thanks for these adjustments, works well for me now!

[edward-ly]

Rebased and adjusted after #167.

[marcelklehr]

Woop woop 🎉
Thanks @edward-ly !