command not found (in image node:12.6-buster-slim, as opposed to GH Actions)

[fwilhe]

Hi,

as documented here GitHub configures passwordless sudo, so I can have sudo in my shell scripts to perform individual commands as root.

Given this minimal workflow:

on: push
name: Test
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - run: sudo id

The result on Actions is uid=0(root) gid=0(root) groups=0(root)

Running this workflow in act yields this result:

$ act
[Test/build] 🚀  Start image=node:12.6-buster-slim
[Test/build]   🐳  docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Test/build]   🐳  docker cp src=/Users/fwilhe/temp/test-actions/. dst=/github/workspace
[Test/build] ⭐  Run sudo id
| /github/workflow/0: line 2: sudo: command not found
[Test/build]   ❌  Failure - sudo id
Error: exit with `FAILURE`: 127

Not sure what the most elegant solution would be without re-building the node:12.6-buster-slim image to install sudo and configure it passwordless. But I think the goal should be to be 1:1 compatible with actions, if possible.

[cplee]

See https://github.com/nektos/act#runners.

You can customize the image used for the runs-on. I've created an image that is a match to GitHub runners, but its like 15GB!

[fwilhe]

Hi @cplee

thanks for the work you put into this.

I know that the image is configurable, and I've locally tested with an image where I installed sudo. So this is kind of a workaround, but the reason I opened this issue is that this limits compatibility with workflows that run on GitHub. My intention was to point out where the documented (and actual) behavior of Actions is unequal the default of act, which makes it less useful compared to what it could be.

As mentioned I'm not sure what solution I would propose, as maintaining separate images seems inappropriate.

[cplee]

Tough problem. The runners that GitHub Actions offers are huge. I didn't feel right using an 18GB image as the default, requiring users of act to download that large of an image to run it.

I chose the node image because it was required to run node actions and still light enough. Not sure how to proceed...not sure what good enough looks like for the default docker image.

Open to ideas 🙏

[aidansteele]

I wonder if there's a way we could make an intermediary image that has 90% of the needed CLI tools for 10% of the footprint of the giant image. The only way we could reasonably do that (beyond intuition) is scanning a corpus of public GH Actions and seeing what works.

Then again, maybe it would give people a false sense of security as these issues would crop up less often - so it would be "stranger" and more likely to result in bug reports. 🤔

[aniongithub]

Maybe act could use a .Dockerfile that the user can customize to make sure whatever tools they need is available in the used image. This could live in ~/.act/ or the current project directory?

[cplee]

see #196 for discussion on creating a slim runner

[christopherhesse]

I think at least the documentation should be updated, it currently states "The environment variables and filesystem are all configured to match what GitHub provides." but should say that that is only the case if you use this non-default image.

I think that docker for mac/windows is actually not very good with large images, but that an 18GB image in general isn't that crazy given the high speed internet and large disks available today.

[mobileka]

I would at least add common tools that are used in CI builds. For example, make is currently missing. I think that adding it doesn't change the size of the image that much but saves time and makes it much more convenient for a big crowd of people.

Or maybe I'm doing something wrong without realizing it and make is actually in the image :)

[fwilhe]

I would at least add common tools that are used in CI builds

I guess there is not enough agreement on what is "common". In the end you would include everything that github installs in the runner, which @cplee already did in that 20 gig image.

I have to say that building my own "runtime" image worked well for me.

The nice thing with that image is that you can also use it to run the whole action in via

jobs:
  build:
    runs-on: ubuntu-latest
    container: myimage

and to not rely on any "setup" actions. That makes it much easier to reproduce the github build on your local machine using act.

Hope that helps.

[mobileka]

I guess there is not enough agreement on what is "common"

Yes, I agree. But the initial argument against using the huge github image seems to be its size. Such tools as make are tiny and are incredibly popular in this context, so it's not clear to me what's the practical motivation of not including it into the default image.

That makes it much easier to reproduce the github build on your local machine using act.

In my opinion, most of the people interested in this project are interested in it because they already use github actions in their projects, not the other way around. Asking them to change their existing CI configuration (especially on a company/organization level) just to be able to test it with a particular tool doesn't seem to be practical.

Just my opinion though.

[fwilhe]

so it's not clear to me what's the practical motivation of not including it into the default image.

I'm not the author of act, so I can only guess that the motivation is to use an off-the-shelf image, and not to build one and then discuss with everyone what should be included or not.

Asking them to change their existing CI configuration (especially on a company level) just to be able to test it with a particular tool doesn't seem to be practical.

No, I'm not asking anyone to do that, I just say what worked for me.

[catthehacker]

@xloem

• What modification?
• https://github.com/nektos/act#use-an-alternative-runner-image

[xloem]

Modifications to remove or install inaccessible commands. I'll ask the first question a different way:

• How do I build a complete docker github workflow image?

I infer the way to run a custom docker image is:

• Make an empty folder
• Make a Dockerfile in the folder based on a simple template (needs link to example)
• Add lines to the Dockerfile to install any missing packages (needs example format)
• Build the Docker file into an image (needs commandline)
• Change your .actrc to reference the name of the image like nektos/act-environments-ubuntu:18.04 in -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04

[catthehacker]

How do I build a complete docker github workflow image?

It's just not possible due to the way how Docker works. See actions/runner-images#2320 (comment)

way to run a custom docker image

I'd say it's not our responsibility to teach how to make Docker images, there is plenty of (much better) sources that go into details on how to do that where official Docker documentation is quite good source on that.
README.md describes:

• It requires Docker
• How to run different Docker image

Eventually there could be link to Docker documentation regarding how to make Docker images in README.md.

[killthekitten]

I was debugging another issue and wanted to check if the large image would solve it. It looks like I'm stuck with the micro image that I've chosen at the very first install. Is there a way to force-reset to the large one?

I've tried a bunch of things:

• Reinstalling act
• Deleting ~/.actrc
• Deleting ~/.cache/act
• Providing -P ... with the command
• Deleting volumes and containers

After reinstalling it fails almost instantly and never attempts to download the large image, and it looks like containers section of the workflow overrides the runs-on.

Logs

DEBU[0000] Loading environment from /Users/user/project/.env 
DEBU[0000] Loading secrets from /Users/user/project/.act.secrets 
DEBU[0000] Loading workflows from '/Users/user/project/.github/workflows' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/ci.yml' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/deploy-production.yaml' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/deploy-staging.yaml' 
DEBU[0000] Planning job: test                           
DEBU[0000] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0000] HEAD matches refs/heads/main                 
DEBU[0000] using github ref: refs/heads/main            
DEBU[0000] context env => map[ACT:true] 
[Python application/test] 🚀  Start image=python:3.8.5
DEBU[0000] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0000] HEAD matches refs/heads/main                 
DEBU[0000] using github ref: refs/heads/main            
[Python application/test]   🐳  docker pull python:3.8.5
DEBU[0000] Image exists? true                           
[Python application/test]   🐳  docker volume rm act-Python-application-test
[Python application/test]   🐳  docker create image=python:3.8.5 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python application/test] Created container name=act-Python-application-test id=47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837 from image python:3.8.5 (platform: linux/amd64)
[Python application/test] ENV ==> [RUNNER_TOOL_CACHE=/opt/hostedtoolcache RUNNER_OS=Linux RUNNER_TEMP=/tmp]
[Python application/test]   🐳  docker run image=python:3.8.5 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python application/test] Starting container: 47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837
[Python application/test] Started container: 47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837
[Python application/test]   🐳  docker cp src=/Users/user/project/. dst=/Users/user/project
[Python application/test] Exec command '[mkdir -p /Users/user/project]'
DEBU[0000] Writing tarball /var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act438170973 from /Users/user/project/. 
DEBU[0000] Stripping prefix:/Users/user/project/ src:/Users/user/project/. 
[Python application/test] Extracting content from '/var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act438170973' to '/Users/user/project'
DEBU[0006] Writing entry to tarball workflow/event.json len:2 
DEBU[0006] Writing entry to tarball workflow/envs.txt len:0 
DEBU[0006] Writing entry to tarball home/.act len:0     
[Python application/test] Extracting content to '/Users/user/project'
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] context env => map[ACT:true] 
DEBU[0006] context env => map[ACT:true] 
[Python application/test] setupEnv => map[ACT:true]
[Python application/test] ⭐  Run actions/checkout@v2
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
[Python application/test] Skipping actions/checkout
[Python application/test]   ✅  Success - actions/checkout@v2
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0007] HEAD matches refs/heads/main                 
DEBU[0007] using github ref: refs/heads/main            
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0007] HEAD matches refs/heads/main                 
DEBU[0007] using github ref: refs/heads/main            
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Evaluating 'secrets['SOME_SECRET_KEY']' instead of 'secrets.SOME_SECRET_KEY' 
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Evaluating 'secrets['SOME_SECRET_KEY']' instead of 'secrets.SOME_SECRET_KEY' 
[Python application/test] setupEnv => map[ACT:true]
[Python application/test] ⭐  Run webfactory/[email protected]
[Python application/test]   ☁  git clone 'https://github.com/webfactory/ssh-agent' # ref=v0.5.2
[Python application/test]   cloning https://github.com/webfactory/ssh-agent to /Users/user/.cache/act/[email protected]
[Python application/test] Cloned https://github.com/webfactory/ssh-agent to /Users/user/.cache/act/[email protected]
[Python application/test] Checked out v0.5.2
DEBU[0008] Read action &{webfactory/ssh-agent  Run `ssh-agent` and load an SSH key to access other private repositories map[ssh-auth-sock:{Where to place the SSH Agent auth socket false } ssh-private-key:{Private SSH key to register in the SSH agent true }] map[] {node12 map[] dist/index.js  [] []} {yellow loader}} from '/Users/user/.cache/act/[email protected]/action.yml' 
DEBU[0008] About to run action &{webfactory/ssh-agent  Run `ssh-agent` and load an SSH key to access other private repositories map[ssh-auth-sock:{Where to place the SSH Agent auth socket false } ssh-private-key:{Private SSH key to register in the SSH agent true }] map[] {node12 map[] dist/index.js  [] []} {yellow loader}} 
DEBU[0008] type=3 actionDir=/Users/user/.cache/act/[email protected] actionPath= Workdir=/Users/user/project ActionCacheDir=/Users/user/.cache/act [email protected] containerActionDir=/actions 
DEBU[0008] Removing /Users/user/.cache/act/[email protected]/.gitignore before docker cp 
[Python application/test]   🐳  docker cp src=/Users/user/.cache/act/[email protected] dst=/actions/
[Python application/test] Exec command '[mkdir -p /actions/]'
DEBU[0008] Writing tarball /var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act810834712 from /Users/user/.cache/act/[email protected] 
DEBU[0008] Stripping prefix:/Users/user/.cache/act/ src:/Users/user/.cache/act/[email protected] 
[Python application/test] Extracting content from '/var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act810834712' to '/actions/'
DEBU[0008] executing remote job container: [node /actions/[email protected]/dist/index.js] 
[Python application/test] Exec command '[node /actions/[email protected]/dist/index.js]'
| OCI runtime exec failed: exec failed: container_linux.go:367: starting container process caused: exec: "node": executable file not found in $PATH: unknown
[Python application/test]   ❌  Failure - webfactory/[email protected]
DEBU[0008] exit with `FAILURE`: 126                     
DEBU[0008] exit with `FAILURE`: 126                     
DEBU[0008] exit with `FAILURE`: 126                     
Error: exit with `FAILURE`: 126

[catthehacker]

@killthekitten
You are using python:3.8.5 according to your log.
To get default image survey, .actrc cannot exist in ~, ~/.config/ and current directory where act is run or you can just replace image yourself, those are described in README

[killthekitten]

@catthehacker thanks for the fast response!

To get default image survey, .actrc cannot exist in ~,

Did you mean it can exist or cannot exist?

I tried replacing the python runtime with no big success:

act -j test --secret-file .act.secrets -v -P python:3.8.5=nektos/act-environments-ubuntu:18.04

Maybe it has to do with runs-on and container defined at the same time? It comes from this section of my workflow:

jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: python:3.8.5

[catthehacker]

container will use the image you defined in container:. -P is used to replace image used in runs-on:.

[jonico]

@catthehacker : First of all I like to show my fullest appreciation for all the work you were doing to provide access to multiple image sizes. I was wondering whether in order to avoid many duplicate issues here - one could create symlinks to tools that have not been included in the smaller images to a file that once executed provides a message explaining that you need a larger image, point to this issue and then error out?

[catthehacker]

@jonico that's an amazing idea, I'll try to work on that.

[jonico]

@jonico that's an amazing idea, I'll try to work on that.

If one were to use an unusual exit code for that script, one may also have special treatment of this error code one the docker container exited - will definitely try out any suggestions you do 😊

[catthehacker]

I would use already existing exit code 127 which stands for command not found but I will use command-not-found or write small Go program to make proper error message and add it to image.

[jonico]

@catthehacker: I will feature nektos/act in GitHub's upcoming summer fest. I figured you are in a similar time zone and might be interested in listening in. Here is the registration link (everybody is welcome - and there will be swag for the first 200 registrants): https://resources.github.com/webcasts/Summerfest-2021-South-Europe/

[catthehacker]

@jonico

[jonico]

@catthehacker: Wonderful 😻 - I would suggest to change the error message to
"Please see #107 for potential reasons and solutions." but this is probably just 🚲 shedding 😊

[RyanMeulenkamp]

Why isn't the gh command installed though?

[Waltari10]

So I tried using the medium instead of slim for git, but noticed medium doesn't have yarn :/

[fwilhe2]

Not sure if this is useful for anyone, but out of curiosity I've tried to build a 'minimal' runner image based on github's packer scripts. The repo is here, the image is published to DockerHub both for x86 and arm.

The idea is to keep the pre-installed amount of tools low because I assume most workflows anyway use setup-* actions.

I know about existing images, but maybe mine fill a gap and are useful for some people?

Let me know if this is useful to you.

[alexeysergeev-cm]

Why isn't the gh command installed though?

Were you able to resolve this? Should I install gh first via brew install?

[RichardJECooke]

Is this the command I'm supposed to use please? It doesn't seem to spend time pulling any bigger image though -

./act -P catthehacker/ubuntu:custom-latest-20240215 -s GITHUB_TOKEN=abIUV8V push

I still get this error

[Test FusionAuth login/run-tests]   ✅  Success - Main Get docker-compose.yaml
[Test FusionAuth login/run-tests] ⭐ Run Main Start FusionAuth
[Test FusionAuth login/run-tests]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/1-composite-1.sh] user= workdir=faDockerComposeFilePath
| /var/run/act/workflow/1-composite-1.sh: line 2: docker: command not found
[Test FusionAuth login/run-tests]   ❌  Failure - Main Start FusionAuth
[Test FusionAuth login/run-tests] exitcode '127': command not found, please refer to https://github.com/nektos/act/issues/107 for more information

same as when I tried running

./act -P ubuntu-22.04=-self-hosted 

[rseymour]

What's interesting is the runner links are broken above, and I would love to know how to use the 18GB runner right now.

[rseymour]

For those looking, the docs on runners have apparently moved here: https://nektosact.com/usage/runners.html also my issue is one of surprise. The first run I was asked how large an env would I like. I chose micro, because I'm just starting. By making that uninformed choice, I spent the last couple hours trying to figure out what issue 107 meant and how to do -P ubuntu-latest=catthehacker/ubuntu:act-latest once I finally found the documentation. Forcing a user on first run to make a size choice with no obvious way to change said choice is a great way to trap folks in this non-intuitive error.

[munapower]

So I tried using the medium instead of slim for git, but noticed medium doesn't have yarn :/

I added these steps and act was able to add yarn with that

      - name: Set Node.js 16.x
        uses: actions/setup-node@v3
        with:
          node-version: 16.x
      - name: Run install
        uses: Borales/[email protected]
        with:
          cmd: install # will run `yarn install` command
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} # if needed
      - name: 🚀　Build bundle
        uses: Borales/[email protected]
        with:
          cmd: run build
          dir: ./

[lewsmith]

Hi,

as documented here GitHub configures passwordless sudo, so I can have sudo in my shell scripts to perform individual commands as root.

Given this minimal workflow:

on: push
name: Test
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - run: sudo id

The result on Actions is uid=0(root) gid=0(root) groups=0(root)

Running this workflow in act yields this result:

$ act
[Test/build] 🚀  Start image=node:12.6-buster-slim
[Test/build]   🐳  docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Test/build]   🐳  docker cp src=/Users/fwilhe/temp/test-actions/. dst=/github/workspace
[Test/build] ⭐  Run sudo id
| /github/workflow/0: line 2: sudo: command not found
[Test/build]   ❌  Failure - sudo id
Error: exit with `FAILURE`: 127

Not sure what the most elegant solution would be without re-building the node:12.6-buster-slim image to install sudo and configure it passwordless. But I think the goal should be to be 1:1 compatible with actions, if possible.

Very old issue, but I'm getting around it by installing sudo if env.ACT is true. That way only 4 extra lines are needed. E.g.

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Install Act dependencies
        if: ${{ env.ACT }}
        run: |
          apt-get update && apt-get install sudo -y

      - name: Install dependencies
        run: |
          sudo apt-get update && sudo apt-get install curl build-essential -y