Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error during upsert - Document store document-stores not found #3660

Closed
PylotLight opened this issue Dec 9, 2024 · 4 comments
Closed

Error during upsert - Document store document-stores not found #3660

PylotLight opened this issue Dec 9, 2024 · 4 comments
Labels
question Further information is requested

Comments

@PylotLight
Copy link
Contributor

PylotLight commented Dec 9, 2024

Describe the bug
Running upsert from within a chatflow seems to work..
But doing it through document stores errors.
Jumping onto the pod and viewing the logs, I can see the .pdf files are being sent to different locations:

/data/storage/0e343de5-1c95-42d3-9d89-1504b8b29870/ Works as expected and is the chatflow file upload location.
/data/storage/docustore/6410ed8e-2ef6-4f6f-891c-96415f7e720a/ Does not work and is the document-stores upload location.

I can see it's meant to be doing:

curl 'https://host.com/725e5c87-7165-4b94-8351-4aca6d7f132d/llm-tool-flowise-master-flowise/api/v1/document-store/store/6410ed8e-2ef6-4f6f-891c-96415f7e720a'

But what it's actually calling for some reason:

curl 'https://host.com/725e5c87-7165-4b94-8351-4aca6d7f132d/llm-tool-flowise-master-flowise/api/v1/document-store/store/document-stores'

So for no reason at all, the storeId in the browser is being set to document-stores instead of the actual storeId, no idea why yet tho.

To Reproduce
Steps to reproduce the behavior:

  1. Go to document-stores, create new store, add a file and process it.
  2. Click on upsert
  3. See error
Oh snap!
The following error occured when loading this page.
Status: 500
Error: documentStoreServices.getDocumentStoreById - Error: documentStoreServices.getDocumentStoreById - Document store document-stores not found
Please retry after some time. If the issue persists, reach out to us on our Discord server.
Alternatively, you can raise an issue on Github.

Expected behavior
Upserts document correctly

Screenshots
Error

Flow
N/A

Setup
Helm/K8s/Docker

  • Installation [e.g. docker, npx flowise start, pnpm start]
  • Flowise Version [e.g. 1.2.11]
  • OS: [e.g. macOS, Windows, Linux]
  • Browser [e.g. chrome, safari]

Additional context

Using legacy mode doesn't help. Tested with multiple different pdf files, all with same error

@HenryHengZJ
Copy link
Contributor

Is this via API or from the UI?

@HenryHengZJ HenryHengZJ added the question Further information is requested label Dec 12, 2024
@PylotLight
Copy link
Contributor Author

This is via the UI.
I've done a ton of debugging on this as it makes no sense at all, but I'm also not that familiar with react.
The issue again is basically when clicking on the upsert chunks button on any document loader type (tried multiple), it redirects the call to
https://host.com/725e5c87-7165-4b94-8351-4aca6d7f132d/llm-tool-flowise-master-flowise/api/v1/document-store/store/document-stores

However we can see that the button should be making this call.
In src/packages/ui/src/views/docstore/DocumentStoreDetail.jsx

console.log("ds:", documentStore)
navigate(`/document-stores/vector/${documentStore.id}/${loader.id}`)

The document store when clicking the button is correct in logging, however somewhere during the navigate the URL get's altered.

ds: {id: '3c74933a-dc08-4199-8dd2-49da9c87f08f', name: 'test-store', description: '', loaders: Array(1), whereUsed: Array(0), …}

In src/packages/server/src/controllers/documentstore/index.ts I've added some logging and can see that the Request from express is already incorrect.

const getDocumentStoreById = async (req: Request, res: Response, next: NextFunction) => {
    try {
        if (typeof req.params.id === 'undefined' || req.params.id === '') {
            throw new InternalFlowiseError(
                StatusCodes.PRECONDITION_FAILED,
                `Error: documentStoreController.getDocumentStoreById - id not provided!`
            )
        }
        console.log("updating get: ", req.params)
        const apiResponse = await documentStoreService.getDocumentStoreById(req.params.id)
...

As the logs below show me clicking on the document loader, then on upsert chunks button.

updating get:  { id: '3c74933a-dc08-4199-8dd2-49da9c87f08f' }
store ID 3c74933a-dc08-4199-8dd2-49da9c87f08f
updating get:  { id: 'document-stores' }
store ID document-stores

What happens between these two steps, and where does the Req come from that leads to storeID being set incorrectly?

@PylotLight
Copy link
Contributor Author

FYI we solved this one internally.
In src/packages/ui/src/views/docstore/VectorStoreConfigure.jsx

    const URLpath = document.location.pathname.toString().split('/')
    const storeId = pathSegments[3] || null
    const docId = pathSegments[4] || null

should probably be

    const URLpath = document.location.pathname.toString().split('/')
    const storeId = URLpath[URLpath.length - 2] === 'document-stores' ? '' : URLpath[URLpath.length - 2] || null
    const docId = URLpath[URLpath.length - 1] === 'document-stores' ? '' : URLpath[URLpath.length - 1] || null

Also this URLpath creation is used in a few places and should possibly have an imported util used in all locations.
Perhaps you missed this one, but this fixed it for us.

@HenryHengZJ
Copy link
Contributor

should be fixed with this PR #3761

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants