[BUG] The input directory "/var/bin" does not exist #41

rajkrajpj · 2023-01-18T14:22:30Z

Environment

chromium Version: 109.0.1
puppeteer / puppeteer-core Version: 19.4.0
Node.js Version: v18.12.0
Lambda / GCF Runtime: nodejs18.x

Expected Behavior

Current Behavior

The input directory \"/var/bin\" does not exist.

Steps to Reproduce

const browser = await puppeteer.launch({
    args: chromium.args,
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath("/var/bin"),
    headless: chromium.headless,
    ignoreHTTPSErrors: true,
  });

Possible Solution

The text was updated successfully, but these errors were encountered:

Sparticuz · 2023-01-18T17:11:51Z

Does /var/bin exist in the function's container? I'll need way more context in order to help

rajkrajpj · 2023-01-18T18:21:13Z

Thanks for the response I appreciate the help @Sparticuz .

I've created the /var/bin directory with the chromium.br file inside in both the root and relative directory of the lambda function.

I've copied the same chromium.br file from the node_modules.

Please let me know if my understanding to your reply is correct 🙏

Sparticuz · 2023-01-18T18:38:33Z

adding it to var/bin inside the function will add it to the functions working directory, not to /var/bin. If you open terminal and navigate to that folder, you can type pwd and it will give you the path you are looking for. Though, if you are uploading the br files in the function, I'm not sure what the benefit is over just using the default.

rajkrajpj · 2023-01-19T13:29:27Z

Thanks for clarifying and for the suggestion @Sparticuz .

I've followed your advise changing it to:

executablePath: await chromium.executablePath('var/bin')

Alongside other different input attempts, it did not work.

I suspected that the chromium binary file (chromium.br?), is not being copied into the source code upon serverless deployment.

I've validated my assumptions by:

copying the chromium.br file to the same directory where my lambda handler is
logging all relative files to check if its there - its not there

Would you know what I need to do to make the chromium.br file (and other binary files needed) be copied when I deploy?

I appreciate your help thank you so much 🙏

rajkrajpj · 2023-01-19T15:40:57Z

Following above's comment -

I verified that the /bin directory was not existing in the presence of the lambda environment.
It is existing in my local node_modules though

rajkrajpj · 2023-01-19T17:22:06Z

I've been able to solve the issue by including the locally installed dependency in the serverless deployment.

include:
    - node_modules/@sparticuz

gbanis · 2023-01-29T02:59:02Z

Not sure if this is a hack or what but here's how I made it work with AWS SAM:

Followed the instructions under AWS Lambda Layer from here.
Added a layer in my function's template pointing to the layer just created in previous step
Added "puppeteer-core": "^19.6.0" and "@sparticuz/chromium": "^110.0.0" as dev dependencies
Pointed the executable path to here executablePath: await chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin')

Hope this helps the next noob like myself trying to make this work.

Sparticuz · 2023-01-30T18:53:11Z

Interesting, you shouldn't have needed to do that. You are specifically pointing to the layer location.

5neaX · 2023-01-31T07:56:03Z

@rajkrajpj , @Sparticuz or anyone else. I have been stuck on trying to run my webscraper on AWS for 2 weeks now trying everything. Ive tried all chrome-aws-lambda, sparticuz aws lambda etc and got to this after reading through the internet. Now this is my setup

after typing 'npm ls' you can see these are my dependencies

C:\Users\glenn
├── @sparticuz/[email protected]
└── [email protected]

Im using Node.js Version: v18.12.0

Lambda / GCF Runtime: nodejs18.x

Memory
2048MB
Ephemeral storage
2048MB

my index.js file is as bellow which runs a function called scraper in scrape.js

`const scraper = require('./scrape.js');

exports.handler = async (event) => {
const data = await scraper();
return {
statusCode: 200,
body: JSON.stringify(data),
};
};
`
I zipped these (package.json, package-lock, node_modules, index.js , scrape.js) and uploaded to s3 and used the s3 url to add the code to the lambda function.

In the scraping code before the bowser launched i have a console.log so i know everything works up until there. However the error occurs straight after and it doesnt seem like the browser is launching when i hit test. I get this error

and this is what ive used to open the browser

await (async () => { const browser = await puppeteer.launch({ args: chromium.args, defaultViewport: chromium.defaultViewport, executablePath: await chromium.executablePath("/var/task/node_modules/@sparticuz/chromium/bin"), headless: chromium.headless, ignoreHTTPSErrors: true, }).catch((error) => {console.log(error); });

please note in the executable path ive tried leaving is empty (), writing var/bin, and what is in there currently is what I saw according your last post here. Im not quite sure what to put here and how to find out how, and if this fixes the problem

Could you or anyone please explain what the solution is and how I can finally overcome this in a simple way possible considering im average at coding? Do i need to copy and paste chromium.br somewhere else ??

will buy you a case of beers if you can solve this XD

thanks

Glenn,

[email protected]

5neaX · 2023-01-31T08:34:41Z

I've been able to solve the issue by including the locally installed dependency in the serverless deployment.
include:
    - node_modules/@sparticuz

Hey, im a little confused about this step specifically as it appears to be what i need to do to solve the problem? how to I include the locally instaled dependency in the serverless deployment ?

JobaDiniz · 2023-02-09T00:19:41Z

Pointed the executable path to here executablePath: await chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin')

I faced the same error: The input directory \"/var/bin\" does not exist..

@gbanis I'm another noob myself, and you saved my day. However, I do know why this is the case. Maybe it should be documented in the Readme.

But I wonder how people deal with layers in SAM. I had to manually create the bucket, and then manually download the artifact from the release, and then manually run the command

bucketName="chromiumUploadBucket" && \
aws s3 cp chromium-v110.0.0-layer.zip "s3://${bucketName}/chromiumLayers/chromium-v110.0.0-layer.zip" && \
aws lambda publish-layer-version --layer-name chromium --description "Chromium v110.0.0" --content "S3Bucket=${bucketName},S3Key=chromiumLayers/chromium-v110.0.0-layer.zip" --compatible-runtimes nodejs --compatible-architectures x86_64

I still didn't figure it out how make that part of SAM or automatic.

buithaibinh · 2023-04-19T03:41:07Z

I encountered a similar issue, and the solution was to exclude '@sparticuz/chromium' from the bundled lambda. Here's a code snippet using CDK:

bundling: {
  externalModules: ['aws-sdk', '@sparticuz/chromium'] // Add any external modules here
},

ktwbc · 2023-06-04T23:12:43Z

Not sure if this is a hack or what but here's how I made it work with AWS SAM:

Followed the instructions under AWS Lambda Layer from here.

Added a layer in my function's template pointing to the layer just created in previous step

Added "puppeteer-core": "^19.6.0" and "@sparticuz/chromium": "^110.0.0" as dev dependencies

Pointed the executable path to here executablePath: await chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin')

Hope this helps the next noob like myself trying to make this work.

I was getting Error: The input directory "/var/task/src/bin" does not exist. at Function.executablePath with an empty await chromium.executablePath() when deployed on a node18 lambda using the lambda layer method and
"@sparticuz/chromium": "^113.0.1",
"puppeteer-core": "20.5.0",

chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin') fixes it, not sure why it was unable to find the path alone, it was resolving as undefined

noudadrichem · 2023-11-29T16:56:44Z

I now also have to force the nodejs path in my lambda code using:

const lambdaLayerChromiumExecPath = '/opt/nodejs/node_modules/@sparticuz/chromium/bin'; // Ref; https://github.com/Sparticuz/chromium/issues/167
const executablePath = process.env.BROWSER_EXECUTABLE_PATH || (await chromium.executablePath(lambdaLayerChromiumExecPath));

Because I'm getting the /var/bin error. I'm creating the layer like so in CDK:

    private createLayerFromZip(name: string, zipName: string, props?) {
        const layer = new LayerVersion(this, name, {
            code: Code.fromAsset(path.join(__dirname, `../layers/${zipName}`)),
            compatibleRuntimes: [Runtime.NODEJS_18_X],
            ...props,
        });

Where the ZIP is the download from the release page.

But now I read from @Sparticuz that that is not the way to handle this. What is the proper/envisioned way to do this?

nicolavs · 2024-06-27T00:36:53Z

Not sure if this is a hack or what but here's how I made it work with AWS SAM:

Followed the instructions under AWS Lambda Layer from here.
Added a layer in my function's template pointing to the layer just created in previous step
Added "puppeteer-core": "^19.6.0" and "@sparticuz/chromium": "^110.0.0" as dev dependencies
Pointed the executable path to here executablePath: await chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin')
Hope this helps the next noob like myself trying to make this work.

i'm following this
with this setup

"@sparticuz/chromium": "123.0.1",
"puppeteer-core": "22.12.1",
Node.js Version: v18.12.0

Lambda Runtime: nodejs18.x

Memory
2048MB
Ephemeral storage
2048MB

I'm not getting Error: The input directory "/var/task/src/bin"
but the task timeout after 30s, usually only takes 2s

RRACI-Blerand · 2024-08-02T22:17:47Z

Does /var/bin exist in the function's container? I'll need way more context in order to help

Hello @Sparticuz please help me with this error :
chromium error { e: error: the input directory "/var/task/node_modules/@sparticuz/chromium/bin" does not exist.

This is happening when my function is uploaded in lambda , I am using Nodejs serverless configuration.

It has been a week that i try to fix it but no luck.

Best regards.

Sparticuz · 2024-08-02T23:00:14Z

Does /var/bin exist in the function's container? I'll need way more context in order to help

Hello @Sparticuz please help me with this error :
chromium error { e: error: the input directory "/var/task/node_modules/@sparticuz/chromium/bin" does not exist.

This is happening when my function is uploaded in lambda , I am using Nodejs serverless configuration.

It has been a week that i try to fix it but no luck.

Best regards.

Make sure you either have this package as a dependency or you have it as a layer

RRACI-Blerand · 2024-08-02T23:12:10Z

/var/task/node_modules/@sparticuz/chromium/bin

Thanks for quick reply @Sparticuz Actually when i downloaded the function from lambda as zip i noticed the bin folder was missing from you library (/node_modules/@sparticuz/chromium/bin) => bin folder missing i have no clue how that happened since locally it is there .
My guess is that when i am deploying to lamda something is happening there ?!

Sparticuz · 2024-08-02T23:13:48Z

@RRACI-Blerand You should be able to just deploy the zip itself as a layer. If you are using a bundler you'll need to include it in the bundle somehow. Each bundler is different

RRACI-Blerand · 2024-08-05T00:17:46Z

@RRACI-Blerand You should be able to just deploy the zip itself as a layer. If you are using a bundler you'll need to include it in the bundle somehow. Each bundler is different

Hi there @Sparticuz i managed to upload the function into s3 bucket then from there upload as zip into my lambda function and there is nothing missing from the libraries now, however i have an error now would you be so kind to assist me please.

ERROR Error: Error: Navigating frame was detached
at #onFrameDetached (/var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:103:47)
at /var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/third_party/mitt/mitt.js:62:7
at Array.map ()
at Object.emit (/var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/third_party/mitt/mitt.js:61:20)
at CdpFrame.emit (/var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/EventEmitter.js:83:23)
at #removeFramesRecursively (/var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/FrameManager.js:450:15)
at #onClientDisconnect (/var/task/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/FrameManager.js:94:42)

npearson72 · 2025-01-10T16:38:58Z

@Sparticuz given the number of issues people have related to this problem, I wonder if recommending the approach @gbanis outlined here #41 (comment) might not be the best way reduce the noise around this topic. It's possibly not as elegant of an approach as you imagined, but it's guaranteed to work in all cases.

If you can get onboard with that, I'd be happy to update the README with a "recommended" or an "if all else fails" section.

kieronjmckenna · 2025-01-14T12:56:40Z

Can confirm that #41 (comment) did the trick for me as well.

Randomly starting getting this error after a deploy, but specifying the chromium path fixed it

bryankocol · 2025-02-18T21:23:39Z

The easiest solution for me was to use a pre-built Lambda Layer, a list which can be found here. Once added, there is no need to change the executablePath in your code. You can include @sparticuz/chromium as a dependency, but exclude it from the sam build. Example yml:

ExampleLambda:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs20.x
      MemorySize: 512
      Timeout: 30
      Policies:
        - AWSLambdaBasicExecutionRole
      Layers:
        - arn:aws:lambda:us-east-1:764866452798:layer:chrome-aws-lambda:50
    Metadata:
      BuildMethod: esbuild
      BuildProperties:
        External:
          - "@sparticuz/chromium"

rajkrajpj added the bug Something isn't working label Jan 18, 2023

rajkrajpj closed this as completed Jan 19, 2023

rajkrajpj mentioned this issue Jan 19, 2023

[BUG] Error: Runtime exited with error: signal: killed Runtime.ExitError #43

Closed

Enzodtz mentioned this issue Feb 22, 2023

Adding example for AWS SAM #64

Closed

Mark-McCracken mentioned this issue May 25, 2023

feat: upgrade node to 18.16.0 guardian/consent-management-platform#747

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] The input directory "/var/bin" does not exist #41

[BUG] The input directory "/var/bin" does not exist #41

rajkrajpj commented Jan 18, 2023

Sparticuz commented Jan 18, 2023

rajkrajpj commented Jan 18, 2023

Sparticuz commented Jan 18, 2023 •

edited

Loading

rajkrajpj commented Jan 19, 2023

rajkrajpj commented Jan 19, 2023

rajkrajpj commented Jan 19, 2023

gbanis commented Jan 29, 2023

Sparticuz commented Jan 30, 2023

5neaX commented Jan 31, 2023

5neaX commented Jan 31, 2023

JobaDiniz commented Feb 9, 2023

buithaibinh commented Apr 19, 2023

ktwbc commented Jun 4, 2023

noudadrichem commented Nov 29, 2023 •

edited

Loading

nicolavs commented Jun 27, 2024

RRACI-Blerand commented Aug 2, 2024

Sparticuz commented Aug 2, 2024

RRACI-Blerand commented Aug 2, 2024 •

edited

Loading

Sparticuz commented Aug 2, 2024

RRACI-Blerand commented Aug 5, 2024

npearson72 commented Jan 10, 2025

kieronjmckenna commented Jan 14, 2025

bryankocol commented Feb 18, 2025 •

edited

Loading

[BUG] The input directory "/var/bin" does not exist #41

[BUG] The input directory "/var/bin" does not exist #41

Comments

rajkrajpj commented Jan 18, 2023

Environment

Expected Behavior

Current Behavior

Steps to Reproduce

Possible Solution

Sparticuz commented Jan 18, 2023

rajkrajpj commented Jan 18, 2023

Sparticuz commented Jan 18, 2023 • edited Loading

rajkrajpj commented Jan 19, 2023

rajkrajpj commented Jan 19, 2023

rajkrajpj commented Jan 19, 2023

gbanis commented Jan 29, 2023

Sparticuz commented Jan 30, 2023

5neaX commented Jan 31, 2023

5neaX commented Jan 31, 2023

JobaDiniz commented Feb 9, 2023

buithaibinh commented Apr 19, 2023

ktwbc commented Jun 4, 2023

noudadrichem commented Nov 29, 2023 • edited Loading

nicolavs commented Jun 27, 2024

RRACI-Blerand commented Aug 2, 2024

Sparticuz commented Aug 2, 2024

RRACI-Blerand commented Aug 2, 2024 • edited Loading

Sparticuz commented Aug 2, 2024

RRACI-Blerand commented Aug 5, 2024

npearson72 commented Jan 10, 2025

kieronjmckenna commented Jan 14, 2025

bryankocol commented Feb 18, 2025 • edited Loading

Sparticuz commented Jan 18, 2023 •

edited

Loading

noudadrichem commented Nov 29, 2023 •

edited

Loading

RRACI-Blerand commented Aug 2, 2024 •

edited

Loading

bryankocol commented Feb 18, 2025 •

edited

Loading