Skip to content

Conversation

@ansd
Copy link
Member

@ansd ansd commented Oct 10, 2025

This commit fixes the CI failure which tests the following example: https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/vault-default-user

RabbitMQ crashed at startup as follows:

failed to open log file at '/var/log/rabbitmq/rabbit@vault-default-user-server-0.vault-default-user-nodes.examples.log', reason: read-only file system

The problem was that with Vault configured the mount in

vaultAnnotations["vault.hashicorp.com/secret-volume-path-"+secretName] = "/etc/rabbitmq/conf.d"
seems to hide the config file in
https://github.com/docker-library/rabbitmq/blob/master/conf.d/10-defaults.conf containing the setting

log.console = true

Without any vault, the config directory in the RabbitMQ pod looks as follows:

k exec r1-server-0 -c rabbitmq -- ls /etc/rabbitmq/conf.d
10-defaults.conf
10-operatorDefaults.conf
11-default_user.conf
90-userDefinedConfiguration.conf

With vault configured, the config directory in the RabbitMQ pod looks as follows:

k exec vault-default-user-server-0 -c rabbitmq -- ls /etc/rabbitmq/conf.d
10-operatorDefaults.conf
11-default_user.conf
90-userDefinedConfiguration.conf

This commit fixes the issue by adding

log.console = true

to file 10-operatorDefaults.conf.

There is no harm if this setting is defined twice in different files in the /etc/rabbitmq/conf.d directory. The last one wins.

This commit fixes the CI failure which tests the following example:
https://github.com/rabbitmq/cluster-operator/tree/main/docs/examples/vault-default-user

RabbitMQ crashed at startup as follows:
```
failed to open log file at '/var/log/rabbitmq/rabbit@vault-default-user-server-0.vault-default-user-nodes.examples.log', reason: read-only file system
```

The problem was that with Vault configured the mount in
https://github.com/rabbitmq/cluster-operator/blob/575cfacf74775a569202d08e406e4e6b5d9d1780/internal/resource/statefulset.go#L876
seems to hide the config file in
https://github.com/docker-library/rabbitmq/blob/master/conf.d/10-defaults.conf
containing the setting
```
log.console = true
```

Without any vault, the config in the RabbitMQ pod looks as follows:
```
k exec r1-server-0 -c rabbitmq -- ls /etc/rabbitmq/conf.d
10-defaults.conf
10-operatorDefaults.conf
11-default_user.conf
90-userDefinedConfiguration.conf
```

With vault configured, the config directory in the RabbitMQ pod looks as follows:
```
k exec vault-default-user-server-0 -c rabbitmq -- ls /etc/rabbitmq/conf.d
10-operatorDefaults.conf
11-default_user.conf
90-userDefinedConfiguration.conf
```

This commit fixes the issue by adding
```
log.console = true
```
to file `10-operatorDefaults.conf`.

There is no harm if this setting is defined twice in different files in
the `/etc/rabbitmq/conf.d` directory. The last one wins.
@ansd ansd self-assigned this Oct 10, 2025
@MirahImage
Copy link
Member

Resolves #1970

@ansd ansd merged commit 70777c9 into main Oct 10, 2025
39 checks passed
@ansd ansd deleted the vault-log branch October 10, 2025 12:33
@wangjianweiwei
Copy link

The configuration is as follows

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: rabbitmq-operator
    app.kubernetes.io/name: rabbitmq-cluster-operator
    app.kubernetes.io/part-of: rabbitmq
  name: rabbitmq-cluster-operator
  namespace: rabbitmq-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: rabbitmq-cluster-operator
  template:
    metadata:
      labels:
        app.kubernetes.io/component: rabbitmq-operator
        app.kubernetes.io/name: rabbitmq-cluster-operator
        app.kubernetes.io/part-of: rabbitmq
    spec:
      containers:
      - command:
        - /manager
        env:
        - name: OPERATOR_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: TZ
          value: Asia/Shanghai
        image: rabbitmqoperator/cluster-operator:2.17.0
        name: operator
        ports:
        - containerPort: 9782
          name: metrics
          protocol: TCP
        resources:
          limits:
            cpu: 200m
            memory: 500Mi
          requests:
            cpu: 200m
            memory: 500Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - All
          runAsNonRoot: true
          seccompProfile:
            type: RuntimeDefault
      serviceAccountName: rabbitmq-cluster-operator
      terminationGracePeriodSeconds: 10

This is RabbitmqCluster resource list

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmqcluster
  namespace: lionmed-cloud-middleware
  labels:
    app: rabbitmq
spec:
  image: rabbitmq:3.10-management
  replicas: 3
  service:
    type: ClusterIP
  persistence:
    storageClassName: nfs-storage
    storage: 5Gi
  rabbitmq:
    additionalConfig: |
      log.console = true   # 避免写 /var/log 目录

Error still reported after startup

2025-11-08 10:15:53.110637+00:00 [error] <0.223.0> failed to open log file at '/var/log/rabbitmq/rabbit@rabbitmqcluster-server-0.rabbitmqcluster-nodes.lionmed-cloud-middleware_upgrade.log', reason: read-only file system

Hope to get some suggestions. thanks😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants