Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transformer not working in js #1021

Open
5 tasks
djaffer opened this issue Nov 12, 2024 · 2 comments
Open
5 tasks

Transformer not working in js #1021

djaffer opened this issue Nov 12, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@djaffer
Copy link

djaffer commented Nov 12, 2024

System Info

@xenova/transformers": "^2.17.2
@huggingface/transformers:"^3.0.2"

Environment/Platform

  • Website/web-app
  • Browser extension
  • Server-side (e.g., Node.js, Deno, Bun)
  • Desktop app (e.g., Electron)
  • Other (e.g., VSCode extension)

Description

Unexpected token '<', "<!doctype "... is not valid JSON
SyntaxError: Unexpected token '<', "<!doctype "... is not valid JSON
at JSON.parse ()
at getModelJSON (http://localhost:3025/static/js/vendors-node_modules_xenova_transformers_src_models_js-node_modules_xenova_transformers_src_t-98979f.chunk.js:35589:15)
at async Promise.all (index 0)
at async loadTokenizer (http://localhost:3025/static/js/vendors-node_modules_xenova_transformers_src_models_js-node_modules_xenova_transformers_src_t-98979f.chunk.js:28435:16)
at async AutoTokenizer.from_pretrained (http://localhost:3025/static/js/vendors-node_modules_xenova_transformers_src_models_js-node_modules_xenova_transformers_src_t-98979f.chunk.js:32349:46)
at async Promise.all (index 0)
at async loadItems (http://localhost:3025/static/js/vendors-node_modules_xenova_transformers_src_models_js-node_modules_xenova_transformers_src_t-98979f.chunk.js:26304:3)
at async pipeline (http://localhost:3025/static/js/vendors-node_modules_xenova_transformers_src_models_js-node_modules_xenova_transformers_src_t-98979f.chunk.js:26249:19)
at async testModel (http://localhost:3025/static/js/bundle.js:39226:18)

Reproduction

Tried with @xenova/transformers too and then tried with @huggingface/transformers.

const pipe = await pipeline('sentiment-analysis');
const out = await pipe('I love transformers!');
console.log(out);

const extractor = await pipeline('feature-extraction', 'Xenova/bge-base-en-v1.5');
const texts = ['Hello world.', 'Example sentence.'];
const embeddings = await extractor(texts, { pooling: 'mean', normalize: true });
console.log(embeddings);
@djaffer djaffer added the bug Something isn't working label Nov 12, 2024
@xenova
Copy link
Collaborator

xenova commented Nov 16, 2024

Hi there 👋 This should have been fixed in @huggingface/transformers. Can you provide the error log for that (the ones shown above are for @xenova/transformers)?

Note: You must uninstall @xenova/transformers and replace all imports of @xenova/transformers with @huggingface/transformers

@djaffer
Copy link
Author

djaffer commented Nov 19, 2024

Thanks Xenova. I will try again. The model wasn't working with @huggingface/transformers.

ERROR in ./node_modules/@huggingface/transformers/dist/transformers.js 42679:34-64
Module not found: Error: Can't resolve './' in '/home/code/node_modules/@huggingface/transformers/dist'

webpack compiled with 1 error and 1 warning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants