Improve the docker-compose development environment

Signed-off-by: Miquel Sabaté Solà <[email protected]>

Author: mssola

.env.example

@@ -1,3 +1,4 @@
+SECRET_KEY_BASE=8ea53ad3bc6c03923e376c8bdd85059c1885524947a7efe53d5e9c9d4e39861106ffd6a2ece82b803072ed701e6c960bade91644979e679416c5f255007237ae
 EXTERNAL_PORT=8080
 SCC_USERNAME=
 SCC_PASSWORD=

Dockerfile

@@ -1,15 +1,12 @@
-FROM opensuse/leap:15.2
+FROM opensuse/leap:15.3
 
-RUN zypper --non-interactive install --no-recommends timezone wget gcc-c++ libffi-devel git-core zlib-devel \
-        libxml2-devel libxslt-devel cron libmariadb-devel mariadb-client vim ruby2.5 ruby2.5-devel ruby2.5-rubygem-bundler &&\
-    zypper --non-interactive install -t pattern devel_basis
-
-ENV DOCKERIZE_VERSION v0.6.0
-RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
-    && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
-    && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
-
-ENV RAILS_ENV production
+RUN zypper --non-interactive install --no-recommends \
+        timezone wget gcc-c++ libffi-devel git-core zlib-devel \
+        libxml2-devel libxslt-devel cron libmariadb-devel mariadb-client \
+        vim ruby2.5 ruby2.5-devel ruby2.5-rubygem-bundler SUSEConnect && \
+    zypper --non-interactive install -t pattern devel_basis && \
+    update-alternatives --install /usr/bin/bundle bundle /usr/bin/bundle.ruby2.5 5 && \
+    update-alternatives --install /usr/bin/bundler bundler /usr/bin/bundler.ruby2.5 5
 
 WORKDIR /srv/www/rmt/
 
@@ -28,22 +25,6 @@ RUN sed -i 's/#!\/usr\/bin\/env ruby/#!\/usr\/bin\/ruby.ruby2.5/g' /srv/www/rmt/
     chown _rmt /srv/www/rmt/public/repo && \
     chown _rmt /srv/www/rmt/public/suma
 
-RUN printf "database: &database\n\
-  host: <%%= ENV['MYSQL_HOST'] %%>\n\
-  username: <%%= ENV['MYSQL_USER'] %%>\n\
-  password: <%%= ENV['MYSQL_PASSWORD'] %%>\n\
-  database: <%%= ENV['MYSQL_DATABASE'] %%>\n\
-database_development:\n\
-  <<: *database\n\
-  database: <%%= ENV['MYSQL_DATABASE'] %%>\n\
-cli:\n\
-  user: root\n\
-  group: root\n\
-scc:\n\
-  username: <%%= ENV['SCC_USERNAME'] %%>\n\
-  password: <%%= ENV['SCC_PASSWORD'] %%>\n\
-" >> config/rmt.local.yml
-
 EXPOSE 4224
 
-CMD dockerize -wait tcp://$MYSQL_HOST:3306 -timeout 60s true && bundle.ruby2.5 exec rails s -b 0.0.0.0 -p 4224
+CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0", "-p", "4224"]

README.md

@@ -66,16 +66,55 @@ Please view our [guide](docs/installation.md) to assist you in the RMT installat
 In order to run the application locally using docker-compose:
 
 1. Copy the `.env.example` file to `.env`.
-2. Add your organization credentials to `.env` file. Mirroring credentials can be obtained from the [SUSE Customer Center](https://scc.suse.com/organization).
-3. Start `docker-compose` and build the containers:
-    ```
-    docker-compose up --build
-    ```
-4. The web server is accessible at [http://localhost:8080/](http://localhost:8080/). This URL can be used for registering clients.
-5. To start a shell inside the RMT docker container, run the following command:
-    ```
-    docker-compose exec rmt bash
-    ```
+2. Add your organization credentials to `.env` file. Mirroring credentials can
+   be obtained from the [SUSE Customer
+   Center](https://scc.suse.com/organization). At this point you might also want
+   to tweak the `EXTERNAL_PORT` environment variable from this file if you want
+   to expose the main service with a port different than from the default one.
+3. Change the permissions on the `public` folder so anyone can access it (i.e.
+   `chmod -R 0777 public`). This is needed so the docker container can write
+   into this specific directory which is protected by default by the `rmt-cli`
+   tool.
+4. Build the containers needed by `docker-compose`:
+    ```
+    docker-compose build
+    ```
+5. Run everything with `docker-compose up`.
+
+After doing all this, there will be `http://localhost:${EXTERNAL_PORT}` exposed
+to the network of the host, and you will be able to register clients by using
+this url. At this point, though, notice that there are two ways to run clients
+in a dockerized fashion as well.
+
+First of all, you can run a client from a custom container (e.g. generated from
+the `SUSE/connect` repository). With this in mind, be aware that you need to be
+on the same network namespace as the host (or the `docker-compose` setup). You
+can manage this with the `--network` flag of the `docker run` command. One easy
+way to achieve this is to set the network as the one from the host: `docker run
+--network=host <...>`. Moreover, notice that `dmidecode`, which is run by
+`SUSEConnect`, will try to access some privileged devices (e.g. `/dev/mem`). By
+default this will also fail, which is why some users simply pass the
+`--privileged` flag to workaround this. This is certainly a solution, but it's
+cleaner to simply add the needed capabilities and the needed devices. In
+conclusion, for a clean run of a client, you could run the following command:
+
+``` sh
+$ docker run --rm --network=host --cap-add=sys_rawio --device /dev/mem:/dev/mem -ti <your-docker-image> /bin/bash
+> SUSEConnect -r <regcode> --url http://localhost:${EXTERNAL_PORT}
+```
+
+Another option is to simply attach a new session into the running `rmt` service,
+which already has the needed devices and capabilities. Thus, you could do
+something like this:
+
+``` sh
+$ docker-compose exec rmt /bin/bash
+> SUSEConnect -r <regcode> --url http://localhost:${EXTERNAL_PORT}
+```
+
+All in all, the code you might be working on sits as a volume inside of the
+Docker container. Thus, you will be able to code as usual and the Docker
+container will behave as if you were working entirely locally.
 
 ## API documentation
 

bin/compose-init.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+begin
+  ::RMT::Db.setup!
+rescue ::RMT::Db::TimeoutReachedError
+  Rails.logger.error "Database connection timed out!"
+  exit 1
+end
+
+# Create the `/var/lib/rmt/system_uuid` file if it does not exist already. This
+# will ease up the first run of `rmt-cli` inside of this container.
+unless File.exist?('/var/lib/rmt/system_uuid')
+  system('dmidecode -s system-uuid > /var/lib/rmt/system_uuid')
+end
+
+system('bundle exec rails s -b rmt -p 4224')

config/environments/development.rb

@@ -1,3 +1,5 @@
+require_relative '../../lib/rmt/logger'
+
 Rails.application.configure do
   # Settings specified here will take precedence over those in config/application.rb.
 
@@ -34,6 +36,7 @@
   # Highlight code that triggered database queries in logs.
   config.active_record.verbose_query_logs = true
 
+  config.logger = RMT::Logger.new($stdout)
 
   # Raises error for missing translations.
   # config.action_view.raise_on_missing_translations = true

config/rmt.yml

@@ -3,27 +3,26 @@
 # You can create rmt.local.yml to override settings in this file.
 
 database: &database
-  host: localhost
-  database: rmt_development
-  username: rmt
-  password: rmt
+  host: <%= ENV.fetch('MYSQL_HOST') { 'localhost' } %>
+  database: <%= ENV.fetch('MYSQL_DATABASE') { 'rmt_development' } %>
+  username: <%= ENV.fetch('MYSQL_USER') { 'rmt' } %>
+  password: <%= ENV.fetch('MYSQL_PASSWORD') { 'rmt' } %>
   adapter: mysql2
   encoding: utf8
   timeout: 5000
   pool: 5
 
 database_development:
   <<: *database
-  database: rmt_development
 
 database_test:
   <<: *database
   database: rmt_test
 
 scc:
   host: https://scc.suse.com/connect
-  username:
-  password:
+  username: <%= ENV['SCC_USERNAME'] %>
+  password: <%= ENV['SCC_PASSWORD'] %>
   sync_systems: true
 
 mirroring:

docker-compose.yml

@@ -6,28 +6,36 @@ volumes:
 services:
   db:
     image: mariadb:10.2
-    restart: always
+    restart: unless-stopped
     volumes:
-      - db_storage:/var/lib/mysql
+      - mariadb:/var/lib/mysql
     extends:
       service: base
       file: docker-compose-base.yml
 
   rmt:
     build: .
+    restart: unless-stopped
     environment:
       - MYSQL_HOST=db
-      - SECRET_KEY_BASE=8ea53ad3bc6c03923e376c8bdd85059c1885524947a7efe53d5e9c9d4e39861106ffd6a2ece82b803072ed701e6c960bade91644979e679416c5f255007237ae
+      - SECRET_KEY_BASE=${SECRET_KEY_BASE}
       - SCC_USERNAME=${SCC_USERNAME}
       - SCC_PASSWORD=${SCC_PASSWORD}
     volumes:
-      - ./public:/srv/www/repo/public/
+      - .:/srv/www/rmt
+      - public:/srv/www/repo/public/
     depends_on:
       - db
     extends:
       service: base
       file: docker-compose-base.yml
-    command: /bin/bash -c 'bundle.ruby2.5 exec rails db:migrate && bundle.ruby2.5 exec rails s -b rmt'
+    # Needed if you want to run SUSEConnect from inside this container.
+    devices:
+      - "/dev/mem:/dev/mem"
+    # Needed if you want to run SUSEConnect from inside this container.
+    cap_add:
+      - sys_rawio
+    command: bundle exec rails runner /srv/www/rmt/bin/compose-init.rb
 
   nginx:
     image: nginx:1.14
@@ -39,3 +47,7 @@ services:
     depends_on:
       - rmt
     entrypoint: /bin/bash -c 'cat /tmp/default.template | sed "s/\\\$$server_port/$EXTERNAL_PORT/g" > /etc/nginx/conf.d/default.conf && nginx -g "daemon off;"'
+
+volumes:
+  public:
+  mariadb:

lib/rmt/db.rb

@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+module RMT
+  # The DB module has useful methods for DB purposes. This is largely based on
+  # SUSE/Portus.
+  module Db
+    WAIT_TIMEOUT  = 90
+    WAIT_INTERVAL = 5
+
+    # Pings the DB and returns a proper symbol depending on the situation:
+    #   * ready: the database has been created and initialized.
+    #   * empty: the database has been created but has not been initialized.
+    #   * missing: the database has not been created.
+    #   * down: cannot connect to the database.
+    #   * unknown: there has been an unexpected error.
+    def self.ping
+      ::RMT::Db.migrations? ? :ready : :empty
+    rescue ActiveRecord::NoDatabaseError
+      :missing
+    rescue Mysql2::Error
+      :down
+    rescue StandardError
+      :unknown
+    end
+
+    # Returns true if the migrations have been run. The implementation is pretty
+    # trivial, but this gives us a nice way to test this module.
+    def self.migrations?
+      ActiveRecord::Base.connection
+      return unless ActiveRecord::Base.connection.table_exists? 'schema_migrations'
+
+      !ActiveRecord::Base.connection.migration_context.needs_migration?
+    end
+
+    def self.setup!
+      count = 0
+
+      loop do
+        case ::RMT::Db.ping
+        when :down
+          Rails.logger.info 'Database not ready yet. Waiting...'
+          sleep WAIT_INTERVAL
+          count += 5
+        when :empty
+          Rails.logger.info 'Database empty: migrating...'
+          system('bundle exec rake db:migrate')
+        when :missing
+          Rails.logger.info 'Database missing: creating...'
+          system('bundle exec rake db:create')
+        when :ready
+          Rails.logger.info 'Database ready!'
+          break
+        end
+
+        raise ::RMT::Db::TimeoutReachedError if count >= WAIT_TIMEOUT
+      end
+    end
+
+    # Raised if any timeout reached
+    class TimeoutReachedError < RuntimeError; end
+  end
+end