[BUG] Password set in OPENSEARCH_INITIAL_ADMIN_PASSWORD actually not used, OpenSearch uses the default password

[svok]

Describe the bug

I use docker-compose file with the following content:

  fluent-bit:
    container_name: fluent-bit
    image: fluent/fluent-bit
    ports:
      - "24224:24224"
      - "24224:24224/udp"
      - "2020:2020"
    environment:
      - opensearch_host=opensearch
      - opensearch_user=admin
      - opensearch_pass=admin # THIS IS WOERKING
#      - opensearch_pass=adm-Password0 # THIS IS NOT WORKING
    volumes:
      - ./volumes/fluent-bit-etc/fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
      - ./volumes/fluent-bit-etc/parsers.conf:/fluent-bit/etc/parsers.conf

  opensearch:
    container_name: opensearch
    image: opensearchproject/opensearch:latest
    environment:
      - discovery.type=single-node
      - http.port=9200
      - bootstrap.memory_lock=true
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=adm-Password0
    ports:
      - '9200:9200'
      - '9600:9600'
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      - opensearch-data:/usr/share/opensearch/data

In this docker-compose I set OPENSEARCH_INITIAL_ADMIN_PASSWORD with 'adm-Password0' for OpenSearch. Fluentbit uses OpenSearch for disposing logs. But it doesn't work with password of OpenSearch. It works actually only with the default password 'admin'.

According to https://opensearch.org/blog/replacing-default-admin-credentials/ I have to use 'adm-Password0' indeed. But the test from this blog is not passing. It is getting passed if I use the default password 'admin'.

curl -ku admin:admin https://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "docker-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 45,
  "active_shards" : 45,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 37,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 54.87804878048781
}

On the other hand, variable OPENSEARCH_INITIAL_ADMIN_PASSWORD is used indeed. If I remove it from the configuration I get the following error:

No custom admin password found. Please provide a password via the environment variable OPENSEARCH_INITIAL_ADMIN_PASSWORD.

Related component

Other

To Reproduce

1. Use above docker-compose to start OpenSearch
2. Use curl -ku admin:adm-Password0 https://localhost:9200/_cluster/health?pretty
3. See error

Expected behavior

OpenSearch should use password that is set via OPENSEARCH_INITIAL_ADMIN_PASSWORD variable. Requests with default password admin should cause 403 error.

Additional Details

Plugins
By default in docker image of OpenSearch

Screenshots
Not applicable

Host/Environment (please complete the following information):

• OS: Linux Ubuntu
• Version 20.04
• OpenSearch docker image 756d24015378
• OpenSearch version: 2.14.0
• Docker Compose version v2.27.0

Additional context
Nothing special

[dblock]

Does your cluster fail to start with an error along the lines of "this password doesn't meet complexity requirements"? Try with a password that we know works, like yourStrongPassword123!?

[gaiksaya]

Hi @svok ,

The volume attached to the docker container, does it already contain the security index? If possible can you destroy the volume or attach a new one to see if it takes effect?

[svok]

@dblock

Does your cluster fail to start with an error along the lines of "this password doesn't meet complexity requirements"? Try with a password that we know works, like yourStrongPassword123!?

1. I have no cluster. In the issue the official docker image is discussed
2. I have no problems with providing password. The issue says that provided password in fact is not used

[svok]

@gaiksaya
Thank you. You are absolutely right. The volume really changes the used password.
In spite of this solves my own problem, there are yet two questions remaining:

1. Is it correct that the password is required in OPENSEARCH_INITIAL_ADMIN_PASSWORD if it is really not used?
2. Is it correct that stored password is not affected by the password policy and no warnings are placed in logs on that?

[fabioDMFerreira]

@svok were you able to resolve the issue? I am changing the env variable, but it assumes the default password.

[svok]

@fabioDMFerreira

@svok were you able to resolve the issue? I am changing the env variable, but it assumes the default password.

Yes. The variable affects only on the first start. All other starts ignore it in spite of this variable remains mandatory.
To change the default password again you need to remove the container's volume or do it with the functionality of OpenSearch as described for instance here: https://code.dblock.org/2023/08/08/changing-the-default-admin-password-in-opensearch.html

[peterzhuamazon]

If the volume is not created from scratch but from an older deployment then the var will not set new password and will keep the old password.

Transfer to security repo as they are the developer to this setup.

Thanks.

[cwperks]

[Triage] Thank you for filing this issue @svok. As others have mentioned, make sure to remove any container volumes before starting the cluster so that the security index initializes from scratch. I'm closing this issue, please re-open if you suspect a bug.