Docker Support

[SethFalco]

Is your feature request related to a problem? Please describe.
It would be great to be able to run this without having to worry about having the correct version of Python / pip on my environment, and isolated from other system resources.

Describe the solution you'd like
To be able to run the script through Docker. I'm willing to contribute towards anything Docker related myself, however before it'd be easier if the script had:

• a way to have to configure the settings via environment variables.
• a way to authenticate to Twitch non-interactively
  • would an authorization token be viable?
  • making password an environment variable and setting would work for non 2FA users
  • to cover 2FA users, either a setting for TOTP which users will just have to then run quickly after configuring the first time, or to generate the cookies externally and mount the ./cookies/ folder as a volume

When or if that's done, a Dockerfile and docker-compose.yml can be created which can be configured via environment variables.

The following should work for a starting point:

.dockerignore

.github/
assets/
cookies/
logs/
.dockerignore
.gitignore
Dockerfile
example.py
LICENSE
run.py

*.yml
*.yaml
*.md

docker-compose.yml

version: "3.8"
services:
  miner:
    build: ./
    restart: "unless-stopped"
    volumes:
      - ./logs/:/home/logs/
    environment:
      twitch_miner_username: "USERNAME"
      twitch_miner_password: "AUTH"

Dockerfile

FROM python:3.9.1-slim-buster
LABEL maintainer="Tkd-Alex"

WORKDIR /home/
COPY ./ /home/

RUN echo "Adding miner system group and user" && \
    groupadd --system --gid 1000 miner && \
    useradd --system --gid miner --uid 1000 --shell /bin/bash miner && \
    echo "Install pip dependencies" && \
    pip install -r requirements.txt;

USER miner

ENTRYPOINT ["python", "run.py"]

[Tkd-Alex]

Great idea! Docket It's always a good choice for every project.

I don't have too much experience, usually the docker-compose It's useful when we have more than one container to merge or not? For this project a simple Docker It's not enough?

Currently, I've very busy with a lot of things and I want just to complete the current work-in-progress feature. I'll start to work on this asap

But If you have free time and experience you can work on it and submit a PR 🥳

[SethFalco]

I don't have too much experience, usually the docker-compose It's useful when we have more than one container to merge or not? For this project a simple Docker It's not enough?

Ahh yeah, you're right. I personally just prefer to make a docker-compose.yml if I'm intending to continue to run and configure a container as opposed to only running it as a one-off. If you think it's not suitable, I see no problem with ignoring that one; it can just be left to user discretion.

It's not unheard of to do this though, for example looking at the docs for two projects I also run via Docker:

• https://github.com/pi-hole/docker-pi-hole/#quick-start
• https://jellyfin.org/docs/general/administration/installing.html#docker

These both include a docker-compose.yml despite being a single service.

[Tkd-Alex]

I've written this little note about docker-compose only for personal doubt.
For me It's the same use only the Dockerfile or both with docker-compose. Obviously in the future if we need another service we have already the compose file 😃

[Tkd-Alex]

@SethFalco you think It's wrong link the run.py via volumes? So we can easily set up the scripts.
In fact, I don't understand why run.py It's in .dockerignore 🤔

[SethFalco]

@SethFalco you think It's wrong link the run.py via volumes? So we can easily set up the scripts.
In fact, I don't understand why run.py It's in .dockerignore thinking

The run.py is in the .dockerignore because when the base image is build, a default run.py file shouldn't be provided, right? You'd only want to attach that via volumes. (.dockerignore is only excluding files for the base image, it will not ignore files for volumes.)

So basically to build the base image without the run.py, so the user can add their own run.py via a volume.

---

Ideally, I think the run.py shouldn't be required to run this in a Docker container, though. Or at least the configuration of the script shouldn't be in the run.py file. The optimal approach I think would be if as much configuration as possible could be done through environment variables alone, and that a run.py could be added as a volume for users that need the flexibility of it, such as per streamer settings.

This is why I thought it might be nice if settings could be done via environment variables first, before making a Dockerfile.

Example

version: "3.8"
services:
  miner:
    build: ./
    restart: "unless-stopped"
    volumes:
      - ./logs/:/logs/
    environment:
      twitch_miner_username: "USERNAME"
      twitch_miner_password: "PASSWORD"
      twitch_miner_claim_drops_startup: false
      twitch_miner_logger_settings_save: true
      twitch_miner_logger_settings_console_level: "INFO"
      twitch_miner_logger_settings_file_level: "DEBUG"
      twitch_miner_logger_settings_emoji: true
      twitch_miner_logger_settings_less: false
      twitch_miner_streamer_settings_make_predictions: false
      twitch_miner_streamer_settings_follow_raid: true
      twitch_miner_streamer_settings_claim_drops: false
      twitch_miner_streamer_settings_watch_streak: true
      twitch_miner_mine_streamers: "streamer1,streamer2,streamer3,streamer4,streamer5"
      twitch_miner_mine_followers: true

Variable names are just examples based on the object structure, they can be anything though.

[Tkd-Alex]

We obviously could be migrating all the settings via env variable, but we will lose the ability to create custom settings for each streamer 🤔

[SethFalco]

This is subjective, what I've described is just what I think would be a nice Docker setup. You're welcome to disagree if you think there is a better way to go for this project.

That's why I suggested that this could be done in stages, rather than trying to achieve Docker support immediately.

This could be achieved already by making run.py a mountable volume, but it'd certainly be an unconventional way to configure a Docker container, and possibly error-prone relative to the simplicity of key/value pairs.

Weblate is a great example of what I'm referring to, they make common settings that can be represented by key/value pairs as environment variables, and require volumes just for any advanced settings.

Normal settings: https://docs.weblate.org/en/latest/admin/install/docker.html?#docker-environment-variables
Custom configuration file: https://docs.weblate.org/en/latest/admin/install/docker.html?#custom-configuration-files

1. (Optional) Move to a configuration file, i.e. YAML, TOML, Properties, INI, etc
2. Support environment variables that will be used if not specified in the configuration file or run.py.
3. Then Docker support, which will use the configuration file or run.py if it's mounted, else environment variables.

[Tkd-Alex]

Maybe the work done on #75 can help us to parse some settings from ENV to my script. I'll see :)

[kevinlul]

Hello, I can help out with this. Doing it for the current configuration model doesn't seem too bad, but is #75 the plan moving forward?

[CappiSteijns]

Any updates on this? Would love to get this working on Docker!

[eltomato89]

+1 !
Being able to start the whole setup with a one liner (including auto restarts) would be so nice!
Could support on the docker side tho ..

[Rakambda]

Just as info, I personally do run it inside docker with https://github.com/RakSrinaNa/Twitch-Miner .

From the docker-compose it can be run as a docker-compose stack or a service (docker stack deploy). This isn't the ideal setup as there's lot of things mounted and it still depends on a main.py that may change between commits. But well, it's better than nothing, runs by itself and restarts as it needs.

[eltomato89]

Thanks! I'll give it a try!
I'm curious how it will run on kubernetes 🙂