Load additional Cuttlefish configuration files

[dumbbell]

By default, in addition to $RABBITMQ_CONFIG_FILE, we also load all matching $RABBITMQ_CONFIG_FILES. It allows to split the configuration into many files for easier management.

Here is the behavior in more details:

• If $RABBITMQ_CONFIG_FILES is a directory, all files directly inside it are considered.

• If $RABBITMQ_CONFIG_FILES is a glob pattern, all files matching the pattern are considered.

• $RABBITMQ_CONFIG_FILES is only relevant when the main configuration is either missing (the file does not exist or is empty) or uses the Cuttlefish format.

The default value of $RABBITMQ_CONFIG_FILES is:

• /etc/rabbitmq/conf.d/*.conf on Unix
• %APPDATA%\RabbitMQ\conf.d\*.conf on Windows

CAUTION: If we want to backport this feature, we have to backport #2180 first! The reason is that to make this possible, we have to use the Cuttlefish library, not the executable.

Depends on rabbitmq/rabbitmq-common#357.

[dumbbell]

Before marking this patch ready for review, I would love to get feedback on the behavior and the variable name.

[gerhard]

Pair-tested in dev, works as expected with:

• globs
• dirs
• dirs with subdirs
• config file
• advanced config file

Great debug logging output 👍

[lukebakken]

I'm sure I've seen this feature requested before. Nice work @dumbbell

[dumbbell]

Thanks!

[dumbbell]

I'm going to add a testcase before we merge it.

[dumbbell]

I can't add a testsuite because our framework extension, rabbitmq-ct-helpers, already uses an Erlang-term-based configuration file to configure nodes and it can't be modified without a heavy redesign of our testsuites. Therefore, the additional configuration files added by a testcase would be ignored anyway because of the main configuration file format.

Let's continue without a testcase for this :-(

[dumbbell]

To make use of RABBITMQ_CONFIG_FILES in the context of make run-broker, RABBITMQ_CONFIG_FILE must be set to a regular empty file (/dev/null will not work because the code checks if it is a regular file). This is required because RabbitMQ assumes that the main configuration file is Erlang-term based, in which case additional config files will not be loaded. We force RabbitMQ to use Cuttlefish with this empty file which is a valid Cuttlefish file, but it is not a valid Erlang-term based config file. This way, $RABBITMQ_CONFIG_FILES are loaded.

To improve this, we can probably always take the Cuttlefish code path: if $RABBITMQ_CONFIG_FILE is an Erlang-term based file, we still use Cuttlefish and pass it as the advanced configuration file.

[michaelklishin]

FTR, this change first shipped in 3.9.0.