Seafile: 10.0.1 -> 11.0.12

[melvyn2]

Description of changes

Updates seafile-server and seahub to 11.0.9. This requires adding the python package djangosaml2. Finally, cleans up the seafile module a little to ensure the proper, matching packages are used.

This required adding a migration script as the sqlite backend was deprecated in favor of MySQL as of version 11. To make this change easier, the persistent user setup of #290104 was merged in.

Finally, a change to the CSRF setup means some users might need to manually whitelist their domains, like so:

  services.seafile = {
    ...

   seahubExtraConf = ''
     CSRF_TRUSTED_ORIGINS = ["https://example.com"]
   '';
  };

This closes #258719

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[melvyn2]

Tests for seafile are failing on master but worked on nixos-24.05; I'll take a look when I can.

Also, the update to 11.0 requires changing a reverse proxy setting that nixos doesn't allow. From https://cloud.seatable.io/dtable/external-links/7b976c85f504491cbe8e/?tid=0000&vid=0000&row-id=BQhH-2HSQs68Nq2EW91DBA, the new django CSRF protection requires the proxy server to pass the Host header untouched, but this is blocked by https://github.com/yandex/gixy/blob/master/docs/en/plugins/hostspoofing.md. This means that either nginx config validation has to be entirely disabled, or CSRF protection has to be disabled.
Neither are ideal solutions; maybe the validation should be a little more configurable, so that skip flags can be passed to gixy.

[melvyn2]

Migration from 10.0 to 11.0 seem to be working properly now, and tests are passing both on 24.05 and unstable. The reverse proxy issue remains, but this is technically not caused by this module, so I'll undraft. A release note line may be necessary.

[greizgh]

Thank you @melvyn2 for looking into this. I've been put off by the deprecation of sqlite (see upstream changelog). It seems to work, but I wouldn't assume this will be the case for long. Are you aware of a decision change?
I see you setup seahub DB from the sql scripts, I've relied on django migrations because it felt weird to have both mechanisms in place. But I guess this is closer to what upstream does.
Regarding CSRF protection, disabling it is not an option (at least to me), but the CSRF_TRUSTED_ORIGINS approach in conf is quite common (and recommended by upstream). Maybe we should expose this through the module (alternatively, this can already be set through extraConfig)?

[melvyn2]

I've been put off by the deprecation of sqlite (see upstream changelog). It seems to work, but I wouldn't assume this will be the case for long. Are you aware of a decision change?

No, I just missed that in the changelog somehow 🤦
I'll re-draft this and see if I can get mysql working in a decent amount of time.

I see you setup seahub DB from the sql scripts, I've relied on django migrations because it felt weird to have both mechanisms in place. But I guess this is closer to what upstream does.

I actually undid this, I originally thought the SQL script was required because during my first attempted upgrade the migration silently failed. The django migrations do work normally and are simpler to use so I've reverted to only using them.

Regarding CSRF protection, disabling it is not an option (at least to me), but the CSRF_TRUSTED_ORIGINS approach in conf is quite common (and recommended by upstream). Maybe we should expose this through the module (alternatively, this can already be set through extraConfig)?

By "disabling" CSRF I meant simply adding the domain to CSRF_TRUSTED_ORIGINS, because I saw this as "bypassing" the protection for that specific origin - it was just bad phrasing from me. Exposing the setting in the module seems like a good choice considering how often this will be required for users to set.

[melvyn2]

The MariaDB-based seafile module is passing tests, and migrations from sqlite/10.0 are almost working. Last error is https://manual.seafile.com/deploy/migrate_from_sqlite_to_mysql/#encountered-errno-150-foreign-key-constraint-is-incorrectly-formed,
which seems to require manually reordering the output of the sqlite dump...
At worst I can solve this with a nasty find-and-replace but I'm looking for a better solution in the mean time.

[melvyn2]

Of course as soon as I post that, I find https://mariadb.com/kb/en/server-system-variables/#foreign_key_checks ...
Will try it next time

[melvyn2]

MariaDB migration is now working in a test environment, and using the ccnet service URL as a default URL for the CSRF whitelist seems like a decent solution for the CSRF issue.
I'll try the migration on my own setup now and see how it goes!

[melvyn2]

Works on my end, after these little changes!

[melvyn2]

I haven't been able to work on this for a decent period of time, but I think with this rebase and modifications (all passing tests and running on my server), it is ready to be merged.

I've removed the remote-mysql additions because they are incomplete and will take non-negligible effort to get into a working state. This was "extra" functionality anyways that should really be in a separate PR; the old sqlite database only supported running on the same host anyways.
I've attached the last state of the WIP commit of this additions in case someone else feels an urge to add this functionality.
mysql_configurable.patch

[melvyn2]

Seems like the failed PR is an OfBorg/GitHub API issue and not something to do with this PR...

[greizgh]

The migration went fine. Let's merge this since seafile is broken on master.

[h7x4]

I'll merge this as is since its better than having it broken on master, but I don't think enabling mysql on the local host is ideal. Would be great if you could open an issue to track progress so others can find it and see that it's not intended as a long term solution. That being said, thank you for the great effort spanning several months!

[dotlambda]

Suggested change

	python tests/run_tests.py
	${python.interpreter} tests/run_tests.py