Appears that multiple passes are stepping on each other #45
Comments
|
Thanks for reporting. I will look into it. |
|
Figured a bit more log output may help show the scope just in case. paperless-ai | Fetched page 46, got 1 matching documents. Total so far: 479 |
|
Processing tag: AI-new Refreshing tag cache... Tag cache refreshed. Found 25 tags. Found tag "AI-new" in cache with ID 80 Found tag "AI" in cache with ID 5 Created new correspondent "Palo Alto Networks" with ID 33 Removing unused tags from document 97, keeping tags: [ 80, 5 ] No tags to remove Current tags for document 97: [ 80 ] Adding new tags: [ 80, 5 ] Combined tags: [ 80, 5 ] Updated document 97 with: { tags: [ 80, 5 ], correspondent: 33, title: 'XYZ 01.05.2024' } Document 97 updated in processed_documents Starting scheduled scan at 2025-01-07T13:00:00.471Z Filtering documents for tags: [ 'ai-new' ] Fetched page 1, got 63 matching documents. Total so far: 63 Fetched page 2, got 6 matching documents. Total so far: 69 Finished filtering. Found 69 documents matching the predefined tags. 80 30 Current config TAGS: [ 'AI-new' ] Current config PROMPT_TAGS: [] 80 30 2025-01-07T13:09:52: PM2 log: Stopping app:paperless-ai id:0 2025-01-07T13:09:52: PM2 log: 0 application online, retry = 3 2025-01-07T13:09:52: PM2 log: App name:paperless-ai id:0 disconnected 2025-01-07T13:09:52: PM2 log: App [paperless-ai:0] exited with code [0] via signal [SIGINT] 2025-01-07T13:09:52: PM2 log: pid=33 msg=process killed 2025-01-07T13:09:53: PM2 log: PM2 successfully stopped 2025-01-07T13:10:01: PM2 log: Launching in no daemon mode 2025-01-07T13:10:02: PM2 log: App [paperless-ai:0] starting in -cluster mode- 2025-01-07T13:10:02: PM2 log: App [paperless-ai:0] online Loading .env from: /app/data/.env Loaded environment variables: { PAPERLESS_API_URL: 'http://paperless-ngx:8000/api', PAPERLESS_API_TOKEN: '1fd33f9XXXXXXX9c05fb8XXXXXXd9138' } (node:19) [DEP0040] DeprecationWarning: The (Use Server running on port 3000 [DEBUG] [07.01.25, 13:10] OpenAI request sent Configured scan interval: */15 * * * * Starting initial scan at 2025-01-07T13:10:04.583Z Refreshing tag cache... Tag cache refreshed. Found 25 tags. Filtering documents for tags: [ 'ai-new' ] Error fetching tag text for ID 24: Request failed with status code 500 Fetched page 1, got 63 matching documents. Total so far: 63 Fetched page 2, got 6 matching documents. Total so far: 69 Finished filtering. Found 69 documents matching the predefined tags. Starting scheduled scan at 2025-01-07T13:15:00.562Z Refreshing tag cache... Tag cache refreshed. Found 25 tags. Filtering documents for tags: [ 'ai-new' ] Error fetching tag text for ID 67: Request failed with status code 500 Error fetching tag text for ID 54: Request failed with status code 500 Error fetching tag text for ID 24: Request failed with status code 500 Error fetching tag text for ID 64: Request failed with status code 500 Error fetching tag text for ID 68: Request failed with status code 500 Error fetching tag text for ID 61: Request failed with status code 500 Error fetching tag text for ID 66: Request failed with status code 500 Error fetching tag text for ID 60: Request failed with status code 500 Error fetching tag text for ID 62: Request failed with status code 500 Error fetching tag text for ID 65: Request failed with status code 500 Error fetching tag text for ID 69: Request failed with status code 500 Error fetching tag text for ID 9: Request failed with status code 500 Error fetching tag text for ID 61: Request failed with status code 500 Fetched page 1, got 63 matching documents. Total so far: 63 Fetched page 2, got 6 matching documents. Total so far: 69 Finished filtering. Found 69 documents matching the predefined tags. |
|
Version latest. |
|
I am also seeing the status 500 error codes like @STL2020 shows. |
Addressing Fixes and new Features: Fixes: #66 #61 #58 #55 #53 #45 #59 #52 #49 #31 #37 #52 Added: - Big New Feature: Playground - Try your prompts on your documents and see how they perform. In Playground no data will be updated in Paperless. - Added Code and Markdown interpretation in Chat Mode. - Chat Mode now works with Ollama
|
the processing definitely appears much faster, appears to be pulling in all of the documents and tags, and goes fast enough that it shouldn't step on itself. Very much improved and the 500 errors appear to be gone as well |
Describe the bug
Cron job is set to run every 30 minutes, logs appear that multiple runs are going concurrently. I have over 5k documents so it seems scanning through them for ones tagged to be processed is taking a very long time.
To Reproduce
Steps to reproduce the behavior:
set up default container with 30 minute Cron job and many documents
paperless-ai | Fetched page 23, got 0 matching documents. Total so far: 8
paperless-ai | Fetched page 58, got 0 matching documents. Total so far: 4
Expected behavior
prior run should finish before another pass is started
Desktop (please complete the following information):
The text was updated successfully, but these errors were encountered: