Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helm upgrade path from Opendistro-for-Elasticsearch 1.13 #17

Open
avthart opened this issue Jul 27, 2021 · 5 comments
Open

Helm upgrade path from Opendistro-for-Elasticsearch 1.13 #17

avthart opened this issue Jul 27, 2021 · 5 comments

Comments

@avthart
Copy link

avthart commented Jul 27, 2021

We have a couple of questions regarding the new Opensearch project and 1.0.0 release.

We are currently using Opendistro-for-Elasticearch 1.13.x.

  • What are the plans with the Helm chart for opendistro-for-es? Will this Helm chart be (re)used for Opensearch?
  • When can we expect a release for Opensearch 1.0.0?

Also see opendistro-for-elasticsearch/opendistro-build#780

Thanks.

@peterzhuamazon
Copy link
Member

Hi @avthart we already have a helm OpenSearch chart in the repo https://github.com/opensearch-project/opensearch-devops/tree/main/Helm

The Dashboards one is pending on review: opensearch-project/opensearch-devops#10

As for the upgrading path I am not sure whether the one we write for TAR works as well: https://opensearch.org/docs/upgrade-to/index/ since Helm uses docker.

I think @DandyDeveloper @dblock @TheAlgo can help on more on that.

Thanks.

@peterzhuamazon peterzhuamazon self-assigned this Jul 27, 2021
@DandyDeveloper
Copy link
Collaborator

@avthart Chart wise, they are incompatible but can be structured in the same way as before effectively creating the same resources with the same supporting configs/secrets.

That being said, I haven't thoroughly tested my own current production Opendistro ES to check where pain points are.

My understanding is for upgrading the version of ES, if you do a standard rollout via k8s for example, the Opensearch image can replace the OpenDistro ES image as if you were upgrading that way.

However, chart wise, there's a big difference and compatibility was overall ignored in the initial creation of the chart.

This is something I'll likely be looking at in the next month or so, as I'll be needing to migrate my production cluster :)

@TheAlgo
Copy link
Member

TheAlgo commented Jul 29, 2021

However, chart wise, there's a big difference and compatibility was overall ignored in the initial creation of the chart.
@DandyDeveloper
While creating the initial version of the chart the following considerations were made:

  1. Move away from monolithic chart. Decouple OpenSearch Dashboards from the OpenSearch chart.
  2. Give maximum customisability to the end users by making a generic chart. This way we can accommodate deployment strategies for all.

@avthart We can definitely have one chart which satisfies the upgrade from the opendistro chart in case we feel it is the best way to migrate to OpenSearch.

@DandyDeveloper
Copy link
Collaborator

@TheAlgo You may find we already can support an upgrade path since we're using the same resource types. The only big problem is it'll almost definitely result in downtime because the statefulset manifest is significantly different.

I think it's hard to make a one size fits all for upgrade paths. In my case, I have a ton of special cases that never made it into the official Opendistro chart.

I think the important thing to test is basic upgradeability, which needs testing. I may get time for this soonish.

@TheAlgo
Copy link
Member

TheAlgo commented Aug 5, 2021

@TheAlgo You may find we already can support an upgrade path since we're using the same resource types. The only big problem is it'll almost definitely result in downtime because the statefulset manifest is significantly different.

I think it's hard to make a one size fits all for upgrade paths. In my case, I have a ton of special cases that never made it into the official Opendistro chart.

I think the important thing to test is basic upgradeability, which needs testing. I may get time for this soonish.

I agree on it we can support the upgrade path. I would love to understand about those special cases (I am curious) 😸

@TheAlgo TheAlgo transferred this issue from opensearch-project/opensearch-devops Sep 8, 2021
TheAlgo added a commit to TheAlgo/helm-charts that referenced this issue Sep 11, 2021
* Adding a new folder to host Helm related code

Signed-off-by: Barani <[email protected]>

* Helm Chart for OpenSearch (opensearch-project#4)

* Create basic structure of OpenSearch helm chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Add templates and change values

Signed-off-by: Dhiraj Jain <[email protected]>

* Change statefulset and configmap to resolve indentation issue

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix issues in templates

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix typos in statefulset.yaml

* Add multinode deployment feature

Signed-off-by: Dhiraj Jain <[email protected]>

* Update version to reflect the OpenSearch version

* Add explicit security configuration

* Update values.yaml

* Create placeholder README.md

Signed-off-by: Dhiraj Jain <[email protected]>

* Minimum masters should be 3

* Add YAML support for config. sysctl vm.mem fix.

* Fixing PSP. Adding better sysctl logic.

* Adding ref for systctl

* PSP False by default

* Disable HTTP SSL by default for Demo.

* Fix Chart version to sync with OpenSearch Version

Signed-off-by: Dhiraj Jain <[email protected]>

* Change cluster name and enable SSL by default

Signed-off-by: Dhiraj Jain <[email protected]>

Co-authored-by: Aaron Layfield <[email protected]>

* fix: give networkpolicy objects a unique name (opensearch-project#16)

This fixes the problem of installing this chart multiple times in the
same namespace and having the network policy name conflict.

Signed-off-by: Scott Leggett <[email protected]>

* fix: use the stable chart appVersion as image tag by default (opensearch-project#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.

Signed-off-by: Scott Leggett <[email protected]>

* OpenSearch Dashboards Helm Chart (opensearch-project#10)

* Scaffold OpenSearch Dashboards Helm Chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix error for connection refused

Signed-off-by: Dhiraj Jain <[email protected]>

* Add RBAC functionality

Signed-off-by: Dhiraj Jain <[email protected]>

* Add security configurations in the chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Address issues and comments

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix templates

Signed-off-by: Dhiraj Jain <[email protected]>

* Disable SSL by default

* Address comments for beautification

* Address comments

Signed-off-by: Dhiraj Jain <[email protected]>

* chore: update demo config section (opensearch-project#24)

This snippet doesn't make sense in a kubernetes statefulset.

Signed-off-by: Scott Leggett <[email protected]>

* added secretMounts to values.yaml w/ example config (opensearch-project#29)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* Change persistence config to make it more coherent w/ other helm charts (opensearch-project#33)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* add Volumes and change broken (!) yaml indentation (opensearch-project#31)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* support for current ingress apiVersion (opensearch-project#47)

* Helm Chart Fixes for Env variables and volumes (opensearch-project#35)

* Helm Chart Fixes for Env variables and volumes

The opensearch-dashboards chart failed to render correctly when
utilizing the extraEnvs flag, caused by incorrect indentation.

The opensearch chart failed to render when utlizing the secrets for the
security config, this was due to them being in the env section.

This pull request reqolves both issues, verified via running helm
template with the minumal values files included here:

```yaml
envFrom:
  - secretRef:
      name: kibana-secrets
extraEnvs:
  - name: TENANT_ID
    valueFrom:
      secretKeyRef:
        name: kibana-secrets
        key: tenantID
```

```yaml
securityConfig:
  enabled: true
  configSecret: "security-config"
  internalUsersSecret: "internal-users-config"
  rolesMappingSecret: "roles-mapping-config"
  rolesSecret: "roles-config"
```

Signed-off-by: Harrison Goscenski <[email protected]>

* Updating paths in sts to be dynamic

Updating the paths specified in the sts for opensearch to utilize
.Values.opensearchHome to allow for dynamic paths, with a default of
`/usr/share/opensearch` which should be sufficient for most users.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing config path in opensearch-dashboards (opensearch-project#38)

* Fixing config path in opensearch-dashboards

The manifests rendered by the Helm chart place the user provided config
into the incorrect directory. This simply updates that location to the
correct path and updates the values.yaml file to use the correct default
config file so that the user provided setting override the defaults.

Signed-off-by: Harrison Goscenski <[email protected]>

* Updating cert paths to opensearch-dashboards

Cert paths also need to utilize new filesystem location for
opensearch-dashboards config.

Signed-off-by: Harrison Goscenski <[email protected]>

* Resolves issue with securityConfig path (opensearch-project#41)

* Resolves issue with securityConfig path

Issue opensearch-project#39

This updates the securityConfig path in values to use the correct value
for opensearch.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing bad auto formatting

Removing unneeded indentation/newlines.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing missed auto formatting errors

Signed-off-by: Harrison Goscenski <[email protected]>

* resolve issue about .Values.opensearchHome (opensearch-project#52)

refer to this:
opensearch-project/opensearch-devops@fe831db#commitcomment-55395428

Error Msg: nil pointer evaluating interface {}.opensearchHome

* Fix helm chart can not be deployed without ssl (opensearch-project#56)

* Fixing issue exposed by changes in opensearch-project#38

After switching the name of the config file, and removing the shadowing
between the default (from the docker container opensearch-dashbaords.yaml) and the default from the helm chart (dashboards.yaml) there is an issue with the certs that are attempting to be used.

In order for this to work with the defaults, disabled TLS verification
will be needed, and then disabling TLS to remain in line with the
defaults.

I added a commented out section showing what could potentially be used
as TLS config if the user chooses to enable it.

Signed-off-by: Harrison Goscenski <[email protected]>

* Using conventional yaml formatting for ssl config

Moving comments around to follow relevant code and utilizing nested yaml
format rather than dot format.

Signed-off-by: Harrison Goscenski <[email protected]>

* Changing Folder name to Charts

* Change deafult configuration for dashboards

Signed-off-by: TheAlgo <[email protected]>

* Update securityconfig.yaml to remove extra spaces

Signed-off-by: TheAlgo <[email protected]>

Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>
bshifter pushed a commit to banzaicloud/helm-charts-1 that referenced this issue Sep 28, 2021
…rch-project#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.

Signed-off-by: Scott Leggett <[email protected]>
bshifter pushed a commit to banzaicloud/helm-charts-1 that referenced this issue Sep 28, 2021
* Adding a new folder to host Helm related code

Signed-off-by: Barani <[email protected]>

* Helm Chart for OpenSearch (opensearch-project#4)

* Create basic structure of OpenSearch helm chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Add templates and change values

Signed-off-by: Dhiraj Jain <[email protected]>

* Change statefulset and configmap to resolve indentation issue

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix issues in templates

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix typos in statefulset.yaml

* Add multinode deployment feature

Signed-off-by: Dhiraj Jain <[email protected]>

* Update version to reflect the OpenSearch version

* Add explicit security configuration

* Update values.yaml

* Create placeholder README.md

Signed-off-by: Dhiraj Jain <[email protected]>

* Minimum masters should be 3

* Add YAML support for config. sysctl vm.mem fix.

* Fixing PSP. Adding better sysctl logic.

* Adding ref for systctl

* PSP False by default

* Disable HTTP SSL by default for Demo.

* Fix Chart version to sync with OpenSearch Version

Signed-off-by: Dhiraj Jain <[email protected]>

* Change cluster name and enable SSL by default

Signed-off-by: Dhiraj Jain <[email protected]>

Co-authored-by: Aaron Layfield <[email protected]>

* fix: give networkpolicy objects a unique name (opensearch-project#16)

This fixes the problem of installing this chart multiple times in the
same namespace and having the network policy name conflict.

Signed-off-by: Scott Leggett <[email protected]>

* fix: use the stable chart appVersion as image tag by default (opensearch-project#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.

Signed-off-by: Scott Leggett <[email protected]>

* OpenSearch Dashboards Helm Chart (opensearch-project#10)

* Scaffold OpenSearch Dashboards Helm Chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix error for connection refused

Signed-off-by: Dhiraj Jain <[email protected]>

* Add RBAC functionality

Signed-off-by: Dhiraj Jain <[email protected]>

* Add security configurations in the chart

Signed-off-by: Dhiraj Jain <[email protected]>

* Address issues and comments

Signed-off-by: Dhiraj Jain <[email protected]>

* Fix templates

Signed-off-by: Dhiraj Jain <[email protected]>

* Disable SSL by default

* Address comments for beautification

* Address comments

Signed-off-by: Dhiraj Jain <[email protected]>

* chore: update demo config section (opensearch-project#24)

This snippet doesn't make sense in a kubernetes statefulset.

Signed-off-by: Scott Leggett <[email protected]>

* added secretMounts to values.yaml w/ example config (opensearch-project#29)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* Change persistence config to make it more coherent w/ other helm charts (opensearch-project#33)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* add Volumes and change broken (!) yaml indentation (opensearch-project#31)

Signed-off-by: johannes.reppin <[email protected]>

Co-authored-by: johannes.reppin <[email protected]>

* support for current ingress apiVersion (opensearch-project#47)

* Helm Chart Fixes for Env variables and volumes (opensearch-project#35)

* Helm Chart Fixes for Env variables and volumes

The opensearch-dashboards chart failed to render correctly when
utilizing the extraEnvs flag, caused by incorrect indentation.

The opensearch chart failed to render when utlizing the secrets for the
security config, this was due to them being in the env section.

This pull request reqolves both issues, verified via running helm
template with the minumal values files included here:

```yaml
envFrom:
  - secretRef:
      name: kibana-secrets
extraEnvs:
  - name: TENANT_ID
    valueFrom:
      secretKeyRef:
        name: kibana-secrets
        key: tenantID
```

```yaml
securityConfig:
  enabled: true
  configSecret: "security-config"
  internalUsersSecret: "internal-users-config"
  rolesMappingSecret: "roles-mapping-config"
  rolesSecret: "roles-config"
```

Signed-off-by: Harrison Goscenski <[email protected]>

* Updating paths in sts to be dynamic

Updating the paths specified in the sts for opensearch to utilize
.Values.opensearchHome to allow for dynamic paths, with a default of
`/usr/share/opensearch` which should be sufficient for most users.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing config path in opensearch-dashboards (opensearch-project#38)

* Fixing config path in opensearch-dashboards

The manifests rendered by the Helm chart place the user provided config
into the incorrect directory. This simply updates that location to the
correct path and updates the values.yaml file to use the correct default
config file so that the user provided setting override the defaults.

Signed-off-by: Harrison Goscenski <[email protected]>

* Updating cert paths to opensearch-dashboards

Cert paths also need to utilize new filesystem location for
opensearch-dashboards config.

Signed-off-by: Harrison Goscenski <[email protected]>

* Resolves issue with securityConfig path (opensearch-project#41)

* Resolves issue with securityConfig path

Issue opensearch-project#39

This updates the securityConfig path in values to use the correct value
for opensearch.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing bad auto formatting

Removing unneeded indentation/newlines.

Signed-off-by: Harrison Goscenski <[email protected]>

* Fixing missed auto formatting errors

Signed-off-by: Harrison Goscenski <[email protected]>

* resolve issue about .Values.opensearchHome (opensearch-project#52)

refer to this:
opensearch-project/opensearch-devops@fe831db#commitcomment-55395428

Error Msg: nil pointer evaluating interface {}.opensearchHome

* Fix helm chart can not be deployed without ssl (opensearch-project#56)

* Fixing issue exposed by changes in opensearch-project#38

After switching the name of the config file, and removing the shadowing
between the default (from the docker container opensearch-dashbaords.yaml) and the default from the helm chart (dashboards.yaml) there is an issue with the certs that are attempting to be used.

In order for this to work with the defaults, disabled TLS verification
will be needed, and then disabling TLS to remain in line with the
defaults.

I added a commented out section showing what could potentially be used
as TLS config if the user chooses to enable it.

Signed-off-by: Harrison Goscenski <[email protected]>

* Using conventional yaml formatting for ssl config

Moving comments around to follow relevant code and utilizing nested yaml
format rather than dot format.

Signed-off-by: Harrison Goscenski <[email protected]>

* Changing Folder name to Charts

* Change deafult configuration for dashboards

Signed-off-by: TheAlgo <[email protected]>

* Update securityconfig.yaml to remove extra spaces

Signed-off-by: TheAlgo <[email protected]>

Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>
NybbleHub pushed a commit to NybbleHub/helm-charts that referenced this issue Dec 22, 2021
* Adding a new folder to host Helm related code


* Helm Chart for OpenSearch (opensearch-project#4)

* Create basic structure of OpenSearch helm chart


* Add templates and change values


* Change statefulset and configmap to resolve indentation issue


* Fix issues in templates


* Fix typos in statefulset.yaml

* Add multinode deployment feature


* Update version to reflect the OpenSearch version

* Add explicit security configuration

* Update values.yaml

* Create placeholder README.md


* Minimum masters should be 3

* Add YAML support for config. sysctl vm.mem fix.

* Fixing PSP. Adding better sysctl logic.

* Adding ref for systctl

* PSP False by default

* Disable HTTP SSL by default for Demo.

* Fix Chart version to sync with OpenSearch Version


* Change cluster name and enable SSL by default


Co-authored-by: Aaron Layfield <[email protected]>

* fix: give networkpolicy objects a unique name (opensearch-project#16)

This fixes the problem of installing this chart multiple times in the
same namespace and having the network policy name conflict.


* fix: use the stable chart appVersion as image tag by default (opensearch-project#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.


* OpenSearch Dashboards Helm Chart (opensearch-project#10)

* Scaffold OpenSearch Dashboards Helm Chart


* Fix error for connection refused


* Add RBAC functionality


* Add security configurations in the chart


* Address issues and comments


* Fix templates


* Disable SSL by default

* Address comments for beautification

* Address comments


* chore: update demo config section (opensearch-project#24)

This snippet doesn't make sense in a kubernetes statefulset.


* added secretMounts to values.yaml w/ example config (opensearch-project#29)


Co-authored-by: johannes.reppin <[email protected]>

* Change persistence config to make it more coherent w/ other helm charts (opensearch-project#33)


Co-authored-by: johannes.reppin <[email protected]>

* add Volumes and change broken (!) yaml indentation (opensearch-project#31)


Co-authored-by: johannes.reppin <[email protected]>

* support for current ingress apiVersion (opensearch-project#47)

* Helm Chart Fixes for Env variables and volumes (opensearch-project#35)

* Helm Chart Fixes for Env variables and volumes

The opensearch-dashboards chart failed to render correctly when
utilizing the extraEnvs flag, caused by incorrect indentation.

The opensearch chart failed to render when utlizing the secrets for the
security config, this was due to them being in the env section.

This pull request reqolves both issues, verified via running helm
template with the minumal values files included here:

```yaml
envFrom:
  - secretRef:
      name: kibana-secrets
extraEnvs:
  - name: TENANT_ID
    valueFrom:
      secretKeyRef:
        name: kibana-secrets
        key: tenantID
```

```yaml
securityConfig:
  enabled: true
  configSecret: "security-config"
  internalUsersSecret: "internal-users-config"
  rolesMappingSecret: "roles-mapping-config"
  rolesSecret: "roles-config"
```


* Updating paths in sts to be dynamic

Updating the paths specified in the sts for opensearch to utilize
.Values.opensearchHome to allow for dynamic paths, with a default of
`/usr/share/opensearch` which should be sufficient for most users.


* Fixing config path in opensearch-dashboards (opensearch-project#38)

* Fixing config path in opensearch-dashboards

The manifests rendered by the Helm chart place the user provided config
into the incorrect directory. This simply updates that location to the
correct path and updates the values.yaml file to use the correct default
config file so that the user provided setting override the defaults.


* Updating cert paths to opensearch-dashboards

Cert paths also need to utilize new filesystem location for
opensearch-dashboards config.


* Resolves issue with securityConfig path (opensearch-project#41)

* Resolves issue with securityConfig path

Issue opensearch-project#39

This updates the securityConfig path in values to use the correct value
for opensearch.


* Fixing bad auto formatting

Removing unneeded indentation/newlines.


* Fixing missed auto formatting errors


* resolve issue about .Values.opensearchHome (opensearch-project#52)

refer to this:
opensearch-project/opensearch-devops@fe831db#commitcomment-55395428

Error Msg: nil pointer evaluating interface {}.opensearchHome

* Fix helm chart can not be deployed without ssl (opensearch-project#56)

* Fixing issue exposed by changes in opensearch-project#38

After switching the name of the config file, and removing the shadowing
between the default (from the docker container opensearch-dashbaords.yaml) and the default from the helm chart (dashboards.yaml) there is an issue with the certs that are attempting to be used.

In order for this to work with the defaults, disabled TLS verification
will be needed, and then disabling TLS to remain in line with the
defaults.

I added a commented out section showing what could potentially be used
as TLS config if the user chooses to enable it.


* Using conventional yaml formatting for ssl config

Moving comments around to follow relevant code and utilizing nested yaml
format rather than dot format.


* Changing Folder name to Charts

* Change deafult configuration for dashboards


* Update securityconfig.yaml to remove extra spaces


Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>
peterzhuamazon added a commit that referenced this issue Jan 7, 2022
* Add new documentations to helm-charts repo (#1)

* Add new documentations to helm-charts repo


* Replace devops with helm charts keywords


* Grammar improvements


* Update README

* Add issue templates and fix readme typos (#3)

* Add issue templates and fix readme typos


* Replace component name with chart name


* Replace OS/Version to the Helm/Kube versions


* Replace OS/Version to the Helm/Kube versions

* Migrate helm charts from opensearch-devops repo (#7)

* Adding a new folder to host Helm related code


* Helm Chart for OpenSearch (#4)

* Create basic structure of OpenSearch helm chart


* Add templates and change values


* Change statefulset and configmap to resolve indentation issue


* Fix issues in templates


* Fix typos in statefulset.yaml

* Add multinode deployment feature


* Update version to reflect the OpenSearch version

* Add explicit security configuration

* Update values.yaml

* Create placeholder README.md


* Minimum masters should be 3

* Add YAML support for config. sysctl vm.mem fix.

* Fixing PSP. Adding better sysctl logic.

* Adding ref for systctl

* PSP False by default

* Disable HTTP SSL by default for Demo.

* Fix Chart version to sync with OpenSearch Version


* Change cluster name and enable SSL by default


Co-authored-by: Aaron Layfield <[email protected]>

* fix: give networkpolicy objects a unique name (#16)

This fixes the problem of installing this chart multiple times in the
same namespace and having the network policy name conflict.


* fix: use the stable chart appVersion as image tag by default (#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.


* OpenSearch Dashboards Helm Chart (#10)

* Scaffold OpenSearch Dashboards Helm Chart


* Fix error for connection refused


* Add RBAC functionality


* Add security configurations in the chart


* Address issues and comments


* Fix templates


* Disable SSL by default

* Address comments for beautification

* Address comments


* chore: update demo config section (#24)

This snippet doesn't make sense in a kubernetes statefulset.


* added secretMounts to values.yaml w/ example config (#29)


Co-authored-by: johannes.reppin <[email protected]>

* Change persistence config to make it more coherent w/ other helm charts (#33)


Co-authored-by: johannes.reppin <[email protected]>

* add Volumes and change broken (!) yaml indentation (#31)


Co-authored-by: johannes.reppin <[email protected]>

* support for current ingress apiVersion (#47)

* Helm Chart Fixes for Env variables and volumes (#35)

* Helm Chart Fixes for Env variables and volumes

The opensearch-dashboards chart failed to render correctly when
utilizing the extraEnvs flag, caused by incorrect indentation.

The opensearch chart failed to render when utlizing the secrets for the
security config, this was due to them being in the env section.

This pull request reqolves both issues, verified via running helm
template with the minumal values files included here:

```yaml
envFrom:
  - secretRef:
      name: kibana-secrets
extraEnvs:
  - name: TENANT_ID
    valueFrom:
      secretKeyRef:
        name: kibana-secrets
        key: tenantID
```

```yaml
securityConfig:
  enabled: true
  configSecret: "security-config"
  internalUsersSecret: "internal-users-config"
  rolesMappingSecret: "roles-mapping-config"
  rolesSecret: "roles-config"
```


* Updating paths in sts to be dynamic

Updating the paths specified in the sts for opensearch to utilize
.Values.opensearchHome to allow for dynamic paths, with a default of
`/usr/share/opensearch` which should be sufficient for most users.


* Fixing config path in opensearch-dashboards (#38)

* Fixing config path in opensearch-dashboards

The manifests rendered by the Helm chart place the user provided config
into the incorrect directory. This simply updates that location to the
correct path and updates the values.yaml file to use the correct default
config file so that the user provided setting override the defaults.


* Updating cert paths to opensearch-dashboards

Cert paths also need to utilize new filesystem location for
opensearch-dashboards config.


* Resolves issue with securityConfig path (#41)

* Resolves issue with securityConfig path

Issue #39

This updates the securityConfig path in values to use the correct value
for opensearch.


* Fixing bad auto formatting

Removing unneeded indentation/newlines.


* Fixing missed auto formatting errors


* resolve issue about .Values.opensearchHome (#52)

refer to this:
opensearch-project/opensearch-devops@fe831db#commitcomment-55395428

Error Msg: nil pointer evaluating interface {}.opensearchHome

* Fix helm chart can not be deployed without ssl (#56)

* Fixing issue exposed by changes in #38

After switching the name of the config file, and removing the shadowing
between the default (from the docker container opensearch-dashbaords.yaml) and the default from the helm chart (dashboards.yaml) there is an issue with the certs that are attempting to be used.

In order for this to work with the defaults, disabled TLS verification
will be needed, and then disabling TLS to remain in line with the
defaults.

I added a commented out section showing what could potentially be used
as TLS config if the user chooses to enable it.


* Using conventional yaml formatting for ssl config

Moving comments around to follow relevant code and utilizing nested yaml
format rather than dot format.


* Changing Folder name to Charts

* Change deafult configuration for dashboards


* Update securityconfig.yaml to remove extra spaces


Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>

* chore: remove redundant line from yaml (#18)

* fix: remove buggy labels template (#20)

The opensearch-dashboards.standard did not properly escape chart
version, and anyway we should be using the same set of standard labels
as all the other templates.

* fix: use absolute path to opensearch-keystore binary (#27)

It is not in $PATH.

* chore: use consistent indentation in opensearch templates (#24)

* Fix typo in comment (#10)

* fix: make secretMount parameters required (#22)

This fixes the case where a parameter on one of the items is silently
missing.

* fix: avoid line containing only spaces in rendered template (#23)

* fix!: update name of JAVA_OPTS variable (#39)

ES_JAVA_OPTS has been renamed in Opensearch to OPENSEARCH_JAVA_OPTS.

* chore: use consistent indentation in opensearch-dashboards templates (#25)

* Add TheAlgo and DandyDeveloper as the new maintainers of the repo (#47)

* Add DandyDeveloper as the new maintainer of the repo


* Add TheAlgo as part of the maintainer list

* Modify majorVersion fallback logic (#21)

* feat: modify majorVersion fallback logic

* Look in both .Values.imageTag and .Chart.AppVersion before falling
back to a default value.
* Use the built-in semver parsing function.
* Don't ignore the version for non-opensearch images.


* fix: use fallback major version 1 instead of 7

Opensearch is currently version 1.x. 7 seems to be a remnant of
Elasticsearch.

* fix securityConfigSecrets.config.data secrets mount plus permissions (#9)

Fix securityConfigSecrets.config.data secrets mount plus permissions

* Add README for OpenSearch (#48)

* Add README for OpenSearch


* Address comments

* Add support for Helm chart linting and releasing. (#46)

* - Added support for the Helm chart testing action.
- Added support for the Helm chart releaser action.
- Fixed minor lint issues in Helm chart values files.


* Added support for testing in addition to linting.


* - Relaxed event triggers on GitHub actions workflow for lint and test.
- Now using `ubuntu-latest` for GitHub runner references.
- Added `maintainers` to all charts.
- Incremented patch version for each chart.


* - Added title for Installation


* - Added missing helm update step in installation.

* fix: use consistent k8s API semver comparison logic (#19)

This is required to work around bugs in the version string returned by
kubernetes distros such as EKS and GKE, where they have invalid Semantic
Version strings. See helm/helm#3810.

* Fix README.md (#60)

* Enable Helm chart release (#61)

* - Added change logs for the opensearch and opensearch-dashboards Helm
  charts.
- Amended README files to reflect the intended installation and usage.
- Incremented the version numbers to 1.0.2 for both Helm charts in
  adherence to linting rules and Semver 2.


* - Modified OpenSearch chart description


* - Reverted to previous chart installation instructions until we can
  verify the new method succeeds.

* Helm Chart Releaser Trigger Fix (#73)

* - Incremented Helm charts to ensure the releaser workflow triggers a
  difference.


* - Added the `workflow_dispatch` option for manually pushing action
  workflows.

* Remove stale README (#57)

* Incorrect indentation for `extraVolumeMounts`, `extraEnvs`, `envFrom` in `statefulset.yaml`. (#80)

* Changes

- Fixes incorrect indentation for `extraVolumeMounts`, `extraEnvs`, and
  `envFrom`.


* Changes:

- Increment version of the opensearch dashboards chart until PR #75 is
  merged.


* - Amended CHANGELOGs

* enable setting docker registry for all images (#70)

* Added basic support for plugins on nodes (#71)

* Adding support for plugins & Prometheus support.

* Updated annotations

* Add support for plugin installation

* Bumping chart patch.

* Bumping again post merge with origin

* Linting fixes.

* Adding to CI. Updating changelog.

* Possibly fixing linting issues.

* Updating plugin

* Increment chart again

* Bumping chart patch.

* CHANGELOG Updates

* Use the correct master configuration for majorVersion 1 (#69)

* fix: use the correct master configuration for majorversion 1


* chore: bump opensearch chart version

* Amended installation instructions (#81)

* Amended installation instruction and relaxed linting and testing workflow triggers.


* Minor typographic error.


* - Reverted linting and testing trigger globbing.


* - Added path globbing.


* - Removed path globbing.


* Revert "- Amended CHANGELOGs"

This reverts commit e0ab178.

* - Bumped chart versions.
- Amended CHANGELOGs.


* Incremented opensearcn chart version to 1.0.8


* - Added specific references to the underlying charts folder from the
  root-level README.
- Addressed clarifications from @TheAlgo.


* - Modified change log for the OpenSearch Helm chart.

* Use a per-install name for securityconfig secret (#41)

* feat: per-install name for securityconfig secret

Give the securityconfig secret an autogenerated unique name to
facilitate installing the chart multiple times in the same namespace.

This helps with the common case of sharing the securityconfig between
multiple instantiations of this chart to construct an Opensearch
cluster.


* feat: update logic to handle externally defined secrets

See the comments describing how this is intended to work.


* chore: bump opensearch chart version


* chore: add securityConfig to README

* Rework labels in Opensearch chart to match standard recommendations (#37)

* feat: rework labels to match standard recommendations

https://helm.sh/docs/chart_best_practices/labels/#standard-labels
https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/


* chore: bump opensearch chart version

* Add missing helm install commands in README (#90)

* Adding a DCO Check related workflow (#101)

* add missing labels key into roles.yaml (#99)

* add missing labels key into roles.yaml


* Apply suggestions from code review

Co-authored-by: Oliver Hartl <[email protected]>

Co-authored-by: Oliver Hartl <[email protected]>

* fix: fix env and envFrom indentation when using keystore value. (#103)

* fix: fix env and envFrom indentation when using keystore value.


* fix: Chart version bump needed by CI

* FIX: Issue 105 - RBAC enabled (#106)

* - Added missing `labels:` stanza delimeter to role.yaml to address the
  failure when RBAC is enabled.


* - Renamed CI values file for testing RBAC enabled.


* - Indented template line to asthetically match.


* - Incremented OpenSearch chart version to 1.2.2 to accommodate another
  PR.


* - Amended CHANGELOG as per review.

* Add option to disable initContainer chown update (#102)

* Add option to disable initContainer chown update


* True default, not false.


* Remove trailing spaces


* Updating CHANGELOG and README

* Change appVersion of OpenSearch and Dashboards chart (#114)

* Updating Latest API Versions for Ingress and Pod Policies (#94)

* Updating Latest API Versions for Ingress and Pod Policies


* chart version bump


* 1.21 for Policy APIs


* Attempting to use kind + GHA matrix for testing various k8s versions


Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Dhiraj Kumar Jain <[email protected]>

* Fix deprication warnings about node.roles. Now roles described as a list (#124)

* add values for fsgroup-volume image (#127)

* add values for fsgroup-volume image


* Increment the Chart version and update the Changelog


* Add version 1.3.1 to CHANGELOG.md

* fix: Handle log4j2 not being yaml (#110) and chart bump. (#123)

* fix: Handle log4j2 not being yaml (#110) and chart bump.


* Including tpl changes


* Adding log4j example.


* Adding some documentation AND updated per comment.s


* Use project name and clarify from/to.


* Explicitly document that config must be YAML multiline strings.


* Cast as string for use with tpl.


* Because this would be really annoying.


* fix: Handle log4j2 not being yaml (#110) and chart bump to 1.4.0.


Co-authored-by: Aaron Layfield <[email protected]>

* [Dashboards] Add extraVolumes and extraVolumeMounts (#128)

* Remove whitespace in DN (#130)

* Update Chart.yaml


* Remove whitespace in dn.


* update changelog.


* update changelog and chart version.

* Updating the copyright header to reflect the apache-2.0 license (#134)

* Updating the copyright header to reflect the apache-2.0 license


* Update opensearch dashboards version and changelogs


Co-authored-by: Peter Zhu <[email protected]>

* Fix node.roles environment variable (#137)

* Fix node.roles environment variable


* forgotten version bump

* Fix url to values.yaml in README.md in opensearch chart (#139)

* Fix url to values.yaml in README.md in opensearch chart


* Make URL to values.yaml in README.md more consistent (with reference section)


* Increment the Chart version and update the Changelog


* Update version of opensearch chart after resolving merge conflict


Co-authored-by: Dmytro Gorbunov <[email protected]>

* FEATURE: Add support for IngressClassName (#149)

* Added support for the `ingressClassName` field. The
`kubernetes.io/ingress.class` annotation was deprecated in Kubernetes
1.18.


* - Fixed trailing spaces as per chart lint rules.

* docs: fix typo (#152)

* docs: fix typo


* Bump version


* Add changelog


* Add changelog


Co-authored-by: Peter Zhu <[email protected]>

* Removed root-level CHANGELOG.md since each chart maintains their own (#165)

changelog.

* Change helm notes as the pod label key has changed (#148)

* Change helm notes as the pod label key has changed


* bump version


* update CHANGELOG.md


* resolve conflicts


* bump version & update changelog

* fix: deprecated api migration versions (#162)

build: add changelog & bump version

* Updated OpenSearch appVersion to 1.2.1 (#164)

* Updated OpenSearch appVersion to 1.2.0


* Fixed CHANGELOG.MD


* Updated to OpenSearch 1.2.1


* Fixed version


Co-authored-by: Derek Diaz <[email protected]>

* prefer .Chart.AppVersion by default (#175)

Do not specify `imageTag` in the default `values.yaml` to use .Chart.AppVersion by default
Fixes #177

* Add notes about default install in README



Signed-off-by: Sébastien Lehuédé <[email protected]>

* Add notes about default install in README



Signed-off-by: Sébastien Lehuédé <[email protected]>

* Change version number

* Change version number

Signed-off-by: Peter Zhu <[email protected]>

* Remove additional files

Signed-off-by: Peter Zhu <[email protected]>

* Remove additional files

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Peter Zhu <[email protected]>
Co-authored-by: Dhiraj Kumar Jain <[email protected]>
Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>
Co-authored-by: Avery Khoo <[email protected]>
Co-authored-by: alborotogarcia <[email protected]>
Co-authored-by: Michael Primeaux <[email protected]>
Co-authored-by: Kenan Erdogan <[email protected]>
Co-authored-by: Oliver Hartl <[email protected]>
Co-authored-by: Paul LESUR <[email protected]>
Co-authored-by: Hayden Fuss <[email protected]>
Co-authored-by: Sebor <[email protected]>
Co-authored-by: Kersten Schlosser <[email protected]>
Co-authored-by: sastorsl <[email protected]>
Co-authored-by: Rémi BUTET <[email protected]>
Co-authored-by: sebas-intellegens <[email protected]>
Co-authored-by: Barani <[email protected]>
Co-authored-by: Tomas Odehnal <[email protected]>
Co-authored-by: Dmytro Gorbunov <[email protected]>
Co-authored-by: Dmytro Gorbunov <[email protected]>
Co-authored-by: Michael Kriese <[email protected]>
Co-authored-by: davidshtian <[email protected]>
Co-authored-by: Michael Rödel <[email protected]>
Co-authored-by: Derek Diaz Correa <[email protected]>
Co-authored-by: Derek Diaz <[email protected]>
Co-authored-by: K3A <[email protected]>
Co-authored-by: Peter Zhu <[email protected]>
@getsaurabh02 getsaurabh02 moved this from 🆕 New to Later (6 months plus) in Engineering Effectiveness Board Jul 18, 2024
@peterzhuamazon peterzhuamazon removed their assignment Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

No branches or pull requests

4 participants