Helm chart #5512

bikash119 · 2024-09-18T13:32:31Z

Description

Use helm chart to manage k8s resources
Added pre-commit hook to use helm lint to find and fix linting errors in chart templates.

Closes #<issue_number>

Type of change

Improvement (change adding some improvement to an existing functionality)
Documentation update

How Has This Been Tested

on dev machine

install minikube
install helm
helm install <chart_folder_location> . In this case helm install argilla-dev examples/deployment/k8s/argilla-cart
This should install and start elasticsearch and argilla-server to the k8s cluster
kubectl port-forward service/argilla-dev-argilla-chart 6900:6900.
http://localhost:6900 ->should open the Argilla UI login screen.
USERNAME: argilla , PASSWORD: 12345678, will authenticate the user and take user to http://localhost:6900/datasets.

Checklist

I added relevant documentation [TODO]
I followed the style guidelines of this project
I did a self-review of my code
I made corresponding changes to the documentation [TODO]
I confirm My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)[ TODO ]

for more information, see https://pre-commit.ci

…evelop

for more information, see https://pre-commit.ci

bikash119 · 2024-09-18T13:47:07Z

@davidberenstein1957 : Should we remove the k8s resources files inside examples/deployment/k8s folder once we have helm charts for Elasticsearch and Argilla? We will have hard time keeping the k8s resource yamls and chart templates in sync if we decide to keep them both. Please share your feedback.
cc: @gabrielmbmb

…g external elasticsearch

…d as external service

davidberenstein1957 · 2024-09-20T06:17:05Z

@frascuchon

…ment

- Removed k8s resource yamls

- Elasticsearch "built in" is deployed as operator. - Redis, Elasticsearch, Argilla user PV to persist data beyond restarts. - Argilla owner username and password is passed from values.yaml. - Align HPA config with “conventions” – I.e. enable via flag and have static replicaCount variable if not activated. - Allow for injecting some configuration via configmaps / secrets etc. If external ES is used, the entries to /etc/hosts are pushed using cm. - Ingress class is made configurable and can be set in values.yaml. - TODO: Add a readme.md for developers to use it. - TODO: Add unit tests to check persistence storage usage.

bikash119 · 2024-09-21T03:55:18Z

hi @davidberenstein1957 , @frascuchon. Request you to share your feedback on this work as I continue to work on creating readme for this change. I have been able to resolve all the review comments for the original PR 4065.
In a nutshell:

Argilla can be installed using helm chart
Redis is added as dependency chart.
ES is installed using Elastic Operator
ES installation is optional based on value in values.yaml.
All of the services use PV to persist data.
Default Owner user can be set from values.yaml.
modified .pre-commit-config.yml to use helm lint for k8s resource yamls.

- Updated CHANGELOG.md to include the PR for helm chart based installation.

frascuchon

Thanks @bikash119 for this amazing PR.

I let some comments/questions. I will try the installation later.

examples/deployments/k8s/argilla-chart/README.md

argilla-server/CHANGELOG.md

examples/deployments/k8s/argilla-chart/README.md

examples/deployments/k8s/argilla-chart/templates/NOTES.txt

examples/deployments/k8s/argilla-chart/values.yaml

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

@frascuchon

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

- Added steps to get started in minikube cluster for local development

- Updated README.md to include few missing steps

jfcalvo · 2024-09-26T10:22:39Z

@bikash119 additionally to the addition of Redis as dependency we need a change to run the Argilla server workers to process the background jobs enqueues into Redis.

You can take a look to https://github.com/argilla-io/argilla/blob/develop/examples/deployments/docker/docker-compose.yaml#L36-L47 to check how we are running the workers in the Argilla Docker compose example, but the idea is that the instance running the workers should have access to the same config that the server is using and then execute the following:

python -m argilla_server worker

Thanks for the PR btw ❤️

- added a resource yaml for argilla-worker - fixed the ports used for readiness and liveliness probes

- Added labels and selector-labels for worker

…d run integration tests

…e successfully before running integration test

bikash119 · 2024-09-27T06:49:01Z

@jfcalvo Thank you for the pointer. I have included it as a different resource to the helm chart.
Additionally, I have updated the README.md to test the deployment in a clean minikube cluster. Was able to execute the integration test using pytest tests/integration
cc: @davidberenstein1957 , @frascuchon

frascuchon

Thanks for the contribution @bikash119!!!

codecov · 2024-10-01T07:25:02Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.21%. Comparing base (12f5109) to head (e41d372).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5512      +/-   ##
===========================================
- Coverage    91.23%   91.21%   -0.02%     
===========================================
  Files          145      145              
  Lines         5964     5964              
===========================================
- Hits          5441     5440       -1     
- Misses         523      524       +1

Flag	Coverage Δ
argilla-server	`91.21% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pre-commit-ci bot and others added 9 commits October 27, 2023 08:11

[pre-commit.ci] auto fixes from pre-commit.com hooks

8494dda

for more information, see https://pre-commit.ci

added helm chart

a3bcc4a

fixed indentation in Chart.yaml

cc31bb2

fixes helm chart

5ecad78

Merge branch 'develop' of https://github.com/Arvind644/argilla into d…

984d753

…evelop

fixed helm chart readme

68b39aa

[pre-commit.ci] auto fixes from pre-commit.com hooks

387e480

for more information, see https://pre-commit.ci

Merge remote-tracking branch 'upstream/develop' into helm_chart

94133bf

First working commit towards helm chart based k8s resource creation.

36ef0f1

bikash119 added 4 commits September 20, 2024 10:18

Working copy of making elasticsearch either a depedency chart or usin…

6a80313

…g external elasticsearch

Fixed linting issues with k8s resource yamls

ff1591c

Merge remote-tracking branch 'upstream/develop' into helm_chart

b583242

Made configmap optional based on if elasticsearch is installed or use…

f6eb3e1

…d as external service

bikash119 added 6 commits September 20, 2024 18:31

added redis as a dependency for Argilla chart

949dbe1

Elasticsearch operator used for deployment of elasticsearch

edd2693

removed un-necessary file which was used to test elasticsearch deploy…

001a014

…ment

- Introduced PVC to store argilla data

968df11

- Removed k8s resource yamls

Removed k8s folder in the under root argilla folder

fca556a

bikash119 marked this pull request as ready for review September 21, 2024 03:58

bikash119 added 7 commits September 21, 2024 11:29

A getting started guide

a5abe5f

Modified the output of Notes.txt to increase readability.

ba59362

First commit : helm unit test [ WIP ]

be9aa36

All the unit testcases are passing

aaf1fee

- Updated README.md to add instruction to execute helm unittest.

8ea7613

- Updated CHANGELOG.md to include the PR for helm chart based installation.

fixed typo error

ca7d20c

fixed typo error

cc84f54

frascuchon reviewed Sep 24, 2024

View reviewed changes

bikash119 and others added 10 commits September 25, 2024 03:21

Update examples/deployments/k8s/argilla-chart/README.md

021c35d

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

Update argilla-server/CHANGELOG.md

bc20af4

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

Update examples/deployments/k8s/argilla-chart/README.md

9c5af60

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

Update examples/deployments/k8s/argilla-chart/README.md

3783035

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

Update examples/deployments/k8s/argilla-chart/README.md

9fbd738

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

Update examples/deployments/k8s/argilla-chart/templates/NOTES.txt

8389a11

Accept change suggested by @frascuchon Co-authored-by: Paco Aranda <[email protected]>

- Added steps to install ECK Operator.

c238ff7

- Added steps to get started in minikube cluster for local development

- Added readiness and liveliness probe to elasticsearch

b6c0c26

- Updated README.md to include few missing steps

Merge remote-tracking branch 'upstream/develop' into helm_chart

cb5d3bb

Merge branch 'develop' into helm_chart

eb032ce

bikash119 added 7 commits September 27, 2024 09:06

- Updated README.md

2ede26c

- added a resource yaml for argilla-worker - fixed the ports used for readiness and liveliness probes

- Added api-key to enable integration test

351210c

- Added labels and selector-labels for worker

Updated the README to include steps to check status of deployement an…

859964c

…d run integration tests

- fixed typo

9a16369

- fixed typo

fa4d688

- few more instruction to check if all the intermediate steps complet…

8b89e5a

…e successfully before running integration test

Merge branch 'develop' into helm_chart

7d5a1f5

bikash119 added 3 commits September 27, 2024 16:16

Merge branch 'develop' into helm_chart

a25d5ed

Merge branch 'develop' into helm_chart

0d3ea85

Merge branch 'develop' into helm_chart

10aa543

frascuchon self-requested a review September 30, 2024 09:03

frascuchon approved these changes Sep 30, 2024

View reviewed changes

frascuchon added this to the v2.3.0 milestone Sep 30, 2024

Merge branch 'develop' into helm_chart

e41d372

frascuchon merged commit 022b414 into argilla-io:develop Oct 1, 2024
1 of 2 checks passed

davidberenstein1957 mentioned this pull request Oct 17, 2024

feat: added helm chart #4065

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Helm chart #5512

Helm chart #5512

bikash119 commented Sep 18, 2024 •

edited

Loading

bikash119 commented Sep 18, 2024 •

edited

Loading

davidberenstein1957 commented Sep 20, 2024 •

edited

Loading

bikash119 commented Sep 21, 2024 •

edited

Loading

frascuchon left a comment

jfcalvo commented Sep 26, 2024

bikash119 commented Sep 27, 2024

frascuchon left a comment

codecov bot commented Oct 1, 2024

Helm chart #5512

Helm chart #5512

Conversation

bikash119 commented Sep 18, 2024 • edited Loading

Description

bikash119 commented Sep 18, 2024 • edited Loading

davidberenstein1957 commented Sep 20, 2024 • edited Loading

bikash119 commented Sep 21, 2024 • edited Loading

frascuchon left a comment

Choose a reason for hiding this comment

jfcalvo commented Sep 26, 2024

bikash119 commented Sep 27, 2024

frascuchon left a comment

Choose a reason for hiding this comment

codecov bot commented Oct 1, 2024

Codecov Report

bikash119 commented Sep 18, 2024 •

edited

Loading

bikash119 commented Sep 18, 2024 •

edited

Loading

davidberenstein1957 commented Sep 20, 2024 •

edited

Loading

bikash119 commented Sep 21, 2024 •

edited

Loading