TypeError: hands.Hands is not a constructor

[mwbouwkamp-mendix]

System information (Please provide as much relevant information as possible)

• React/typescript
• @mediapipe/hands version 0.4.1646424915

Describe the current behavior:
When using the default code example, I get the error TypeError: hands.Hands is not a constructor

code:

        const hands = new mpHands.Hands({
            locateFile: (_file: string) => {
                return `https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/hands.js`;
            }
        });

[kuaashish]

Hi @mwbouwkamp-mendix,
Could you elaborate your query with complete details and steps followed to reproduce the issue from our end. Thank you!

[mwbouwkamp-mendix]

Hi, @kuaashish!

I am trying to use the mediapipe library in a Mendix widget. For this, I did the following:

• Create an empty Mendix widget (https://docs.mendix.com/howto/extensibility/create-a-pluggable-widget-one/)
• npm i @mediapipe/hands
• Create a component for the video and canvas elements
• Create a component where, in a useEffect, I am trying to replicate what is done in the sample code

My useEffect function:

    useEffect(() => {
        const videoElement = document.getElementById("input_video") as HTMLVideoElement;
        const canvasElement = document.getElementById("output_canvas") as HTMLCanvasElement;
        const canvasCtx = canvasElement.getContext("2d");

        if (!canvasCtx) {
            return;
        }

        const onResults = (results: mpHands.Results): void => {
            canvasCtx.save();
            canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
            canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);
            if (results.multiHandLandmarks) {
                for (const landmarks of results.multiHandLandmarks) {
                    drawingUtils.drawConnectors(canvasCtx, landmarks, mpHands.HAND_CONNECTIONS, {
                        color: "#00FF00",
                        lineWidth: 5
                    });
                    drawingUtils.drawLandmarks(canvasCtx, landmarks, { color: "#FF0000", lineWidth: 2 });
                }
            }
            canvasCtx.restore();
        };

        const hands = new mpHands.Hands({
            locateFile: (_file: string) => {
                return `https://cdn.jsdelivr.net/npm/@mediapipe/hands@${mpHands.VERSION}/${file}`;
            }
        });

        hands.setOptions({
            maxNumHands: 1,
            modelComplexity: 1,
            minDetectionConfidence: 0.5,
            minTrackingConfidence: 0.5
        });

        hands.onResults(onResults);

        const camera = new cameraUtils.Camera(videoElement, {
            onFrame: async () => {
                await hands.send({ image: videoElement });
            },
            width: 1280,
            height: 720
        });

        camera.start();

When using this widget in a Mendix app, I get the constructor error.

[ayushgdev]

Hi @mwbouwkamp-mendix
Please try importing mediapipe as

import { useEffect } from 'react';
import { Camera } from '@mediapipe/camera_utils';
import { Hands, HAND_CONNECTIONS, Results } from '@mediapipe/hands';
import { drawConnectors, drawLandmarks } from '@mediapipe/drawing_utils';

and create the hands object like

const hands = new Hands({
      locateFile: (file) => {
        return `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}`;
      },
    });

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[mwbouwkamp-mendix]

Unfortunately, this is not solving my issue. I still get the same error as before that it is not a constructor.

Appology for the late resonse...

[ayushgdev]

Okay. Would it be possible to provide the minimal example code to reproduce the issue at our end, may be create a temporary GitHub repo to clone and test the error?
Also, please provide the steps to reproduce the issue.

[mwbouwkamp-mendix]

To reproduce

• Create an empty Mendix widget (https://docs.mendix.com/howto/extensibility/create-a-pluggable-widget-one/)
• npm i @mediapipe/hands
• npm i @mediapipe/drawing_utils
• npm i @mediapipe/camera_utils
• replace HandTracking.tsx for:

import { ReactElement, createElement, useEffect } from "react";

import { Camera } from "@mediapipe/camera_utils";
import { Hands, HAND_CONNECTIONS } from "@mediapipe/hands";
import { drawConnectors, drawLandmarks } from "@mediapipe/drawing_utils";

export function HandTracking(): ReactElement {
    useEffect(() => {
        const videoElement = document.getElementsByClassName("input_video")[0] as HTMLVideoElement;
        const canvasElement = document.getElementsByClassName("output_canvas")[0] as HTMLCanvasElement;
        const canvasCtx = canvasElement.getContext("2d");

        function onResults(results: { image: CanvasImageSource; multiHandLandmarks: any }): void {
            if (!canvasCtx) {
                return;
            }

            canvasCtx.save();
            canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
            canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);
            if (results.multiHandLandmarks) {
                for (const landmarks of results.multiHandLandmarks) {
                    drawConnectors(canvasCtx, landmarks, HAND_CONNECTIONS, { color: "#00FF00", lineWidth: 5 });
                    drawLandmarks(canvasCtx, landmarks, { color: "#FF0000", lineWidth: 2 });
                }
            }
            canvasCtx.restore();
        }

        const hands = new Hands({
            locateFile: file => {
                return `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}`;
            }
        });
        hands.setOptions({
            maxNumHands: 2,
            modelComplexity: 1,
            minDetectionConfidence: 0.5,
            minTrackingConfidence: 0.5
        });
        hands.onResults(onResults);

        const camera = new Camera(videoElement, {
            onFrame: async () => {
                await hands.send({ image: videoElement });
            },
            width: 1280,
            height: 720
        });
        camera.start();
    }, []);

    return (
        <div className="container">
            <video className="input_video"></video>
            <canvas className="output_canvas" width="1280px" height="720px"></canvas>
        </div>
    );
}

• Create a package with npm run build
• Create a Mendix project
• Add package to the widgets folder
• Open Mendix project
• Add widget to the homepage
• Run the app
• View the app
• See how the widget is not displayed and the error message shows up in the console

Here is the repo https://github.com/mwbouwkamp-mendix/handTrackingWidget

[phongchainos]

I also face this issue but only in production build, it happens with Camera in camera_utils too. For now i get Camera and Hands from cdn link instead of npm package and it work, the versions are same but npm package doesnt work in build

[taha-azzabi]

i confirm have the same problem in production environnement only

[taha-azzabi]

I'm facing the same error on production with a Vite Nuxt project ,i will try to reproduce with a CodePen

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[ayushgdev]

Hello @taha-azzabi @nohr

This seems to be happening due to the Rollup CommonJS plugin, since this plugin can't work with obfuscated packages and the JS source code for MediaPipe is not public. Rollup uses static analysis to determine its exports.

We are working on a solution to unveal the JS code so that this issue does not arise. Meanwhile, can you refer to #2883 (comment) and let us know if this works for you?
PS: The link by the author has changed to this link

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[mwbouwkamp-mendix]

The workaround is not working for me. Could you please provide me with an indication when this may be fixed?

We are working on a solution to unveal the JS code so that this issue does not arise.

[ayushgdev]

@mwbouwkamp-mendix Thanks for letting us know the workaround no longer works. On the fix, we don't have any timeline yet, but I would suggest to expect a wait of around a month.

[schmidt-sebastian]

We are in the process of creating new JS libraries that should not have this issue, as they are built using ESM. While they have not yet reached feature parity, you can find them here: https://www.npmjs.com/package/@mediapipe/tasks-vision, https://www.npmjs.com/package/@mediapipe/tasks-text, https://www.npmjs.com/package/@mediapipe/tasks-audio. The libraries are also open sourced in this repo: https://github.com/google/mediapipe/tree/master/mediapipe/tasks/web

Due to our existing infrastructure, it is difficult for us to release the existing libraries using ESM.

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[spencer17x]

Maybe you can use vite-plugin-mediapipe：https://github.com/Spencer17x/arca/tree/main/packages/vite-plugin/vite-plugin-mediapipe

[MadeWithStone]

per @schmidt-sebastian we used the new apis, @mediapipe/tasks-vision, which works properly in vite locally and in production. Over at Kinetix ML we have it working locally with vite and in production with vercel deploy after facing similar issues with the face landmarks api. You can checkout our implementation here https://github.com/Kinetix-ML/kml-pipe-ts/blob/main/src/operations/faceMeshDetect.ts