prometheus-smartctl-exporter: init at unstable-2020-11-14

[mweinelt]

Motivation for this change

Monitoring harddisk healthyness over time with Prometheus.

• Negative NixOS test, since /dev/vda doesn't work for the exporter smartmontools.

  {
    "json_format_version": [
      1,
      0
    ],
    "smartctl": {
      "version": [
        7,
        2
      ],
      "svn_revision": "5155",
      "platform_info": "x86_64-linux-5.10.79",
      "build_info": "(local build)",
      "argv": [
        "smartctl",
        "--json",
        "--xall",
        "/dev/vda"
      ],
      "messages": [
        {
          "string": "/dev/vda: Unable to detect device type",
          "severity": "error"
        }
      ],
      "exit_status": 1
    }
  }

• Not sure if this is the right exporter to package, it has a fair share of problems and seems a bit undermaintained 😞
  • which can be fixed by applying a few patches from open pull requests, hope they get merged soon

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[github-actions]

Successfully created backport PR #148657 for release-21.11.

[contrun]

It seems CAP_SYS_ADMIN and DeviceAllow are not enough for this exporter to run. Below is my generated systemd unit.

# /etc/systemd/system/prometheus-smartctl-exporter.service
[Unit]
After=network.target

[Service]
Environment="LOCALE_ARCHIVE=/nix/store/mmhbi0x5by8gih5z0sxbr2i1gm1ps39v-glibc-locales-2.33-56/lib/locale/locale-archive"
Environment="PATH=/nix/store/l6f4z8mmcnnxba8w004xn28y0vr4gdkf-coreutils-9.0/bin:/nix/store/v3lvq9hqshyldc4i6f5jy0zs0k5psbws-findutils-4.8.0/bin:/nix/store/nkwls56wcfwi1r0jnkqkvwx2zk7w3qrz-gnugrep-3.7/bin:/nix/store/vklvyr82ajbz7jm7g8dbkh62k20b0dpr-gnused-4.8/bin:/nix/store/aqwqh3drn9bf6qrs70wiyrxaawh1wcnz-systemd-249.5/bin:/nix/store/l6f4z8mmcnnxba8w004xn28y0vr4gdkf-coreutils-9.0/sbin:/nix/store/v3lvq9hqshyldc4i6f5jy0zs0k5psbws-findutils-4.8.0/sbin:/nix/store/nkwls56wcfwi1r0jnkqkvwx2zk7w3qrz-gnugrep-3.7/sbin:/nix/store/vklvyr82ajbz7jm7g8dbkh62k20b0dpr-gnused-4.8/sbin:/nix/store/aqwqh3drn9bf6qrs70wiyrxaawh1wcnz-systemd-249.5/sbin"
Environment="TZDIR=/nix/store/7wxhajvlpndhmcwrpc73sl959yb7jzbr-tzdata-2021c/share/zoneinfo"



AmbientCapabilities=CAP_SYS_ADMIN
CapabilityBoundingSet=CAP_SYS_ADMIN
DeviceAllow=/dev/nvme0
DevicePolicy=closed
DynamicUser=true
ExecStart=/nix/store/bp4061ma1h2vyz01ssrww0fz3qdb5rgq-smartctl_exporter-unstable-2020-11-14/bin/smartctl_exporter -config /nix/store/2xliikcza7ag842vrcs12rjalk3876l5-smartctl-exporter.yml

Group=smartctl-exporter
LockPersonality=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=false
PrivateTmp=true
ProcSubset=pid
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=true
Restart=always
RestrictAddressFamilies=AF_INET
RestrictAddressFamilies=AF_INET6
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
SupplementaryGroups=disk
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources
UMask=0077
User=root
WorkingDirectory=/tmp

With User=root, this exporter works correctly.

Dez 18 22:34:01 ssg systemd[1]: Started prometheus-smartctl-exporter.service.
Dez 18 22:34:01 ssg smartctl_exporter[2445730]: [Info] Starting on 127.0.0.1:9633/metrics

Without User=root, it stops working.

Dez 18 22:27:41 ssg smartctl_exporter[2435759]: [Info] Starting on 127.0.0.1:9633/metrics
Dez 18 22:27:45 ssg smartctl_exporter[2435759]: [Warning] S.M.A.R.T. output reading error: exit status 2
Dez 18 22:27:45 ssg smartctl_exporter[2435759]: [Warning] The device error log contains records of errors.
Dez 18 22:27:45 ssg smartctl_exporter[2435759]: [Error] Smartctl open device: /dev/nvme0 failed: Permission denied
Dez 18 22:27:45 ssg smartctl_exporter[2435759]: [Error] smartctl returned bad data for device /dev/nvme0
Dez 18 22:28:45 ssg smartctl_exporter[2435759]: [Warning] S.M.A.R.T. output reading error: exit status 2
Dez 18 22:28:45 ssg smartctl_exporter[2435759]: [Warning] The device error log contains records of errors.
Dez 18 22:28:45 ssg smartctl_exporter[2435759]: [Error] Smartctl open device: /dev/nvme0 failed: Permission denied
Dez 18 22:28:45 ssg smartctl_exporter[2435759]: [Error] smartctl returned bad data for device /dev/nvme0

[mweinelt]

Use /dev/nvme0n1 instead of /dev/nvme0.

crw------- 1 root root 248, 0 17. Dez 15:31 /dev/nvme0
brw-rw---- 1 root disk 259, 0 17. Dez 15:31 /dev/nvme0n1

[contrun]

Oops. That's it. Thank you.