[WIP] staytus, staytus-module: init at 2017-11-08

[ravloony]

Motivation for this change

Add Staytus (staytus.co) package and service.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

---

At the moment, this builds and sets up the database, but does not start the service. I have not yet managed to figure out all the environment stuff needed to get procodile, puma, bundler, rake and rails to work together happily. Any help would be much appreciated!

[ravloony]

Currently, the systemd unit file that is generated looks ok. But when it runs, it says:

staytus-procodile-start[10962]: No processes to start.

procodile is supposed to start two jobs: web, and worker

Running the worker: /nix/store/vs983zxrjb8xwfdx4z2s6lrq6n18ll9c-staytus-rake/bin/staytus-jobs
works,

but running the web job: /nix/store/vs983zxrjb8xwfdx4z2s6lrq6n18ll9c-staytus-rake/bin/staytus-puma
Puma starting in single mode...

• Version 3.9.1 (ruby 2.3.5-p376), codename: Private Caller
• Min threads: 5, max threads: 5
• Environment: production
  ERROR: No application configured, nothing to run

Still trying to figure this out, but not getting very far.

[ravloony]

I finally have it working, with themes!

Config value services.staytus.stateDir allows the user to chose a state dir. This keeps the logs around, and also tracks database state.
There is also a run directory. The service empties the run directory, regenerates configs based on the config values, and sets up a symlink to the log directory in the state directory.

There are two systemd services that are started. staytusjobs runs the job worker, and staytus the puma web server. Nginx handles static file accesses and passes off to puma when necessary.

Setting a theme is a matter of setting
services.staytus.themes = { themename = ./path/to/theme/named/themename.zip };
services.staytus.theme = "themename";

[ravloony]

Should I squash my commits into one?

[Mic92]

Can your remove this?

[ravloony]

Yes.

[Mic92]

How much data is stored here? /run is a tmpfs with a limited size, /var/lib might be a better choice.

[ravloony]

Was not aware of that, will change.

[ravloony]

moved to /var/run/staytus

[Mic92]

This might be dangerous if the application creates files in /tmp naively (without using mktemp or similar). Other users might trick this service into to different locations by creating symlinks. A dedicated home is recommend.

[ravloony]

Ok, makes sense. Maybe it is not needed anyway. I'll see if it can simply be removed.

[ravloony]

Removed

[Mic92]

Is there an important use-case for making this configurable? If we advertise this as an API it is harder to change the location in future without breaking compatibility. What does upstream proposes here as an directory? I would use /var/lib/staytus otherwise.

[ravloony]

I think that the users should be able to decide where their logs go. I'll set the default to /var/lib/staytus.

[ravloony]

Done

[Mic92]

Can you provide an example value here.

[ravloony]

Done

[Mic92]

Are staytus themes always zip archives? The following api might be more general:

services.staytus.themes = [
  (fetchzip {
    url = "https://host/theme.zip";
    sha256 = "...";
  })
  (fetchgit {
    url = "https://host/theme.git"
    rev = "some-release";
    sha256 = "...";
  })
];

[ravloony]

That would be much better, yes. Could you share an example of how to handle that at the receiving end? I just use unpackFile at the moment.

[ravloony]

@Mic92 actually in this api, would it be possible to use local files as well as remote ones? Is there a local version of fetchzip, or is it compatible with relative paths?

Reason is, I am using this in nixops, and have my company theme in the same dir as the server nixops config, so I just use ./theme.zip.

[Mic92]

smtpPassword and secretKeyBase will end world-readable up in the nix store. This might be not desired depending on the deployment. Until NixOS/nix#8 is solved we add the option to store secrets in a dedicated file, that is copied/loaded at runtime by the service. As the configuration is created at build time, substitution with sed can be used to replace a password place holder with a concrete value.

[ravloony]

I'll look into it, thanks.

[ravloony]

How does one handle the password file with NixOps?

[ravloony]

I think it might be possible to use unix socket authentication to reset the password to something random every time the service is started.

So the staytus user would connect to mysql via unix socket (along the lines of this: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/databases/mysql.nix#L368), then add an identification using a password that is auto generated, which is then set in an env var, and picked up by rails.

On the face of it, it seems possible, so I'll have a closer look next week.

[Mic92]

@ravloony you can squash the last two commits into a nixos/staytus: add service commit.

[ravloony]

@Mic92 Thanks for the input. I should have fixed the issues tomorrow.

[florianjacob]

@ravloony regarding declarative mysql users & databases part: I'd consider to use databaseHost = "localhost"; instead of 127.0.0.1 / double-check if 127.0.0.1 really works. piwik uses 127.0.0.1 as default host as well, but socket authentication did not work properly with that, so I patched the default to be localhost. Apart from that I've nothing to add in terms of declarative mysql, looks good!

[FRidh]

Closing due to inactivity.