Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error: Unsupported model type: whisper #539

Closed
lsiem opened this issue Jan 5, 2024 · 4 comments · Fixed by #549
Closed

Error: Unsupported model type: whisper #539

lsiem opened this issue Jan 5, 2024 · 4 comments · Fixed by #549
Labels
bug Something isn't working Docker Issue related to Docker

Comments

@lsiem
Copy link

lsiem commented Jan 5, 2024 •
edited by timothycarambat
Loading

When uploading an mp4 file I get the error: failed fetch.

The docker log output shows the following:

2024-01-05 09:53:44 -- Working The PisaSales user interface.mp4 --
2024-01-05 09:53:44 [Conversion Required] .mp4 file detected - converting to .wav
2024-01-05 09:53:45 [Conversion Processing]: 4KB converted
2024-01-05 09:53:46 [Conversion Processing]: 37376KB converted
2024-01-05 09:53:46 [Conversion Processing]: 54596KB converted
2024-01-05 09:53:46 [Conversion Complete]: File converted to .wav!
2024-01-05 09:58:45 fetch failed
2024-01-05 09:49:08 ┌─────────────────────────────────────────────────────────┐
2024-01-05 09:49:08 │  Update available 5.3.1 -> 5.7.1                        │
2024-01-05 09:49:08 │  Run the following to update                            │
2024-01-05 09:49:08 │    npm i --save-dev prisma@latest                       │
2024-01-05 09:49:08 │    npm i @prisma/client@latest                          │
2024-01-05 09:49:08 └─────────────────────────────────────────────────────────┘
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 getGitVersion Command failed: git rev-parse HEAD
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 
2024-01-05 10:01:27 Failed to load the native whisper model: Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 node:internal/process/promises:288
2024-01-05 10:01:27             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 10:01:27             ^
2024-01-05 10:01:27 
2024-01-05 10:01:27 Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 
2024-01-05 10:01:27 Node.js v18.19.0

I am running anythingllm with my Apple M1 Max running macOS 14.2.1 (23C71)
@timothycarambat timothycarambat added bug Something isn't working Docker Issue related to Docker labels Jan 5, 2024
@timothycarambat
Copy link
Member

Related: huggingface/transformers.js#314

@timothycarambat
Copy link
Member

Can you shell into your container and see if there are any files in /app/server/storage/models/Xenova/whisper-small. The model should only download when needed for the first time and be loaded from the disk on subsequent runs. However, if for whatever reason HuggingFace blocks the download, then the model would fail to download and thus, inference would fail

I do not have an M-series chip so I am unable to replicate this issue currently. We have the "patched" version of transformers running already so that should be resolved.

@shatfield4
Copy link
Collaborator

shatfield4 commented Jan 5, 2024 •
edited by timothycarambat
Loading

I get a similar fetch failed error in my docker logs as well on an M1 Pro running macOS 12.4

2024-01-05 09:45:41 -- Working jfk.wav --
2024-01-05 09:45:41 [INFO] The native whisper model has never been run and will be downloaded right now. Subsequent runs will be faster. (~250MB)
2024-01-05 09:45:41 
2024-01-05 09:45:41 
2024-01-05 09:46:17 Failed to load the native whisper model: TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 node:internal/process/promises:288
2024-01-05 09:46:17             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 09:46:17             ^
2024-01-05 09:46:17 
2024-01-05 09:46:17 TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 
2024-01-05 09:46:17 Node.js v18.19.0

@timothycarambat
Copy link
Member

timothycarambat commented Jan 5, 2024 •
edited
Loading

Okay, here is the root cause.

On AMD machines, Docker runs AnythingLLM in an x86 emulator on the container. This emulator overhead causes ONNX models to run very slowly if at all. This slowness also leads to timeouts on the HTTPS call to HF when downloading the models needed to embed,whisper, etc.

The fix for this is to publish an AMD64 image for AnythingLLM so that AMD machines can download an appropriate image and not run in a compatibility mode.

  • Implement arm64 build and publish for anythingllm docker image

This support causes issues with lanceDB, who require Rust binaries for it to run.

  • We need to bump LanceDB to a version that publishes support for linux_arm64 binaries so that the non-emulated docker container can use LanceDB.
  • puppeteer does not publish an arm86 compatible chromium with it - so we need to patch this as well.

timothycarambat added a commit that referenced this issue Jan 5, 2024
@timothycarambat
Update build process
b1f0ced 
Bump @lancedb/vectordb to 0.1.19 for AMD compatibilty
resolves #539
This was referenced Jan 5, 2024
Closed
Merged
timothycarambat added a commit that referenced this issue Jan 9, 2024
@timothycarambat @shatfield4
Build & Publish AnythingLLM for ARM64 and x86 (#549)
58971e8 
* Update build process to support multi-platform builds
Bump @lancedb/vectordb to 0.1.19 for ARM&AMD compatibility
Patch puppeteer on ARM builds because of broken chromium
resolves #539
resolves #548

---------

Co-authored-by: shatfield4 <[email protected]>
@timothycarambat timothycarambat mentioned this issue Apr 21, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working Docker Issue related to Docker
Projects
None yet
Milestone
No milestone
3 participants
@shatfield4 @timothycarambat @lsiem