Permissions issue on EC2 self-hosted runner

[snussik]

Hi!
Tried to test your example workflow https://github.com/snussik/ec2-github-runner
But on master commits nothing happens at all.
And VScode linter shows error:

[machulav]

@snussik have you found the issue?

As I see, you set up the workflow for the master branch. But there is no master branch for this repository. The main branch here is the main. I hope it helps.

[snussik]

@machulav hi! Yep, I've discovered it. And I have a new problem now )) When I create remote runner on aws and try to run npm ci (or npm install), I get:
npm ERR! correctMkdir failed to make directory /home/github/.npm/_locks
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /home/github
npm ERR! errno -13

The commands: whoami shows that bash runs under the github user.

The part of workflow file, is:

# Test WF for GA

name: CI

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  start-runner:
    name: Start self-hosted EC2 runner
    runs-on: ubuntu-latest
    outputs:
      label: ${{ steps.start-ec2-runner.outputs.label }}
      ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
    steps:
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID_RUNNER }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY_RUNNER }}
          aws-region: ${{ secrets.AWS_REGION_RUNNER }}
      - name: Start EC2 runner
        id: start-ec2-runner
        uses: machulav/[email protected]
        with:
          mode: start
          github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
          ec2-image-id: ami-02501629bfc260e9c
          ec2-instance-type: t2.micro
          subnet-id: subnet-04850abda92c62906
          security-group-id: sg-0fda70c804d158ece
  do-the-job:
    name: Do the job on the runner
    runs-on: ${{needs.start-runner.outputs.label}} # run the job on the newly created runner
    needs: start-runner # required to start the main job when the runner is ready
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Use Node.js 12
        uses: actions/setup-node@v1
        with:
          node-version: 12.18.2
      - name: Cache node modules
        uses: actions/cache@v1
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: ${{ runner.os }}-node-
      - name: Build
        run: |
          whoami
          npm ci

Is that related to AWS Credentials? I thought that github user runs ints workflow under needed permissons inside the runner instance?

[snussik]

@machulav found out with npm config list that npm HOME variable is set in /home/github. That is why npm tries to mkdir in /home/github and meets EACCES error in this dir.
I think that it's a problem of self-hosted runners when you create new users in them, as on GitHub hosted runner or even sell-hosted, created in order with official GitHub doc doesn't have such problem.

[snussik]

I think that in aws.js (on line 11) it should be something like:
useradd -m -d /home/actions-runner && cd /home/actions-runner

[machulav]

@snussik thank you for the discovery!

In the meantime, @lielran found that the Docker container actions don't run on the self-hosted runner and created PR #11 for that.

So I will check both your ideas together as the action should be a bit updated in order to run any kind of action on it.

[lielran]

@snussik thank you for the discovery!

In the meantime, @lielran found that the Docker container actions don't run on the self-hosted runner and created PR #11 for that.

So I will check both your ideas together as the action should be a bit updated in order to run any kind of action on it.

yes, I've tried to install docker via user-data and also used ready-made AMI with docker.
but the action wasn't able to access docker-engine due to permissions issues.

[machulav]

Yeah, as both of you pointed, there is a general issue with permissions. So I renamed the issue to correspond to that.

The main reason why I create github user is the following: all the bootstrap scripts from ./src/aws.js entered as user data and run as the root user. At the same time, GitHub's run.sh can not be run as a root.

[snussik]

@machulav yep, it's a common "problem" with this bash script. Dumb working receipt is:export RUNNER_ALLOW_RUNASROOT=1 and then run ./run.sh

[lielran]

@machulav yep, it's a common "problem" with this bash script. Dumb working receipt is:export RUNNER_ALLOW_RUNASROOT=1 and then run ./run.sh

Boom! works like magic.
Thanks for the comment on this "Dumb" workaround that saves me hours of debugging

[machulav]

The RUNNER_ALLOW_RUNASROOT fixed the issue!
Also, I've made a couple of additional improvements. See more details in PR #14. I hope it helps.
Later I will create a new release with these changes.

Please note that to reduce the scope of changes and testing, I limited supported Linux distributive to only Amazon Linux 2.
In the future, I'm going to add support for the other distributive if required.

@lielran @snussik thank you for your support and contributions! It means a lot!
You, folks, really motivate me to continue working on the action! 🙌

[lielran]

The RUNNER_ALLOW_RUNASROOT fixed the issue!
Also, I've made a couple of additional improvements. See more details in PR #14. I hope it helps.
Later I will create a new release with these changes.

Please note that to reduce the scope of changes and testing, I limited supported Linux distributive to only Amazon Linux 2.
In the future, I'm going to add support for the other distributive if required.

@lielran @snussik thank you for your support and contributions! It means a lot!
You, folks, really motivate me to continue working on the action! 🙌

here is the final result, we are already using it in our CD pipeline.

[machulav]

Wow, very nice! 🎉

@lielran May I ask you to describe your use case a bit? What do you use the self-hosted runner for and why is it important in your CD pipeline?

[lielran]

Wow, very nice! 🎉

@lielran May I ask you to describe your use case a bit? What do you use the self-hosted runner for and why is it important in your CD pipeline?

Sure,
we are using flyway to align our DB schema (rds,pg). the db is running in isolated subnet without(direct) internet access. the easist thing to do is run the migration from within the vpc on a public subnet(application subnet).

[machulav]

Cool! Thank you for the information!

[sasrinivasan-exelixis]

@lielran I am slo planning to use this and flyway. Can you please share an example action and your flyway setup?

[lielran]

@lielran I am slo planning to use this and flyway. Can you please share an example action and your flyway setup?

I personally moved to use hosted Github runners on the VPC and also plan to move to GitHub action code build