Add env_file + ConfigMaps feature to Kompose #799

cdrage · 2017-08-25T19:52:52Z

When using env_file with Docker Compose, a ConfigMap will be generated.

For example:

▶ ./kompose convert -f
script/test/fixtures/configmaps/docker-compose.yml
INFO Kubernetes file "redis-service.yaml" created
INFO Kubernetes file "redis-deployment.yaml" created
INFO Kubernetes file "foo-env-configmap.yaml" created
INFO Kubernetes file "bar-env-configmap.yaml" created

File:

version: '3'

services:
  redis:
    image: 'bitnami/redis:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=no
    # Env file will override environment / warn!
    env_file:
      - "foo.env"
      - bar.env
    labels:
      kompose.service.type: nodeport
    ports:
      - '6379:6379'

To:

apiVersion: v1
data:
  ALLOW_EMPTY_PASSWORD: "yes"
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: foo-env

...
      - env:
        - name: ALLOW_EMPTY_PASSWORD
          valueFrom:
            configMapKeyRef:
              key: ALLOW_EMPTY_PASSWORD
              name: foo-env

kompose-bot · 2017-08-25T19:52:55Z

@cdrage, thank you for the pull request! We'll request some people to review your PR. @containscafeine and @procrypt, please review this.

cdrage · 2017-08-25T19:53:12Z

TODO:

Add tests
Remove conflicts (kobject.go)

cdrage · 2017-08-28T15:22:34Z

Ready for review @ashetty1 @containscafeine @kadel @surajnarwade @surajssd

concaf

Docker Compose file -

version: '3'
  
services:
  redis:
    image: 'bitnami/redis:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=no
    # Env file will override environment / warn!
    env_file:
      - ./bar.env
    labels:
      kompose.service.type: nodeport
    ports:
      - '6379:6379'

Generated ConfigMap -

- apiVersion: v1
  data:
    BAR: FOO
    FOO: BAR
  kind: ConfigMap
  metadata:
    creationTimestamp: null
    name: -/bar-env
kind: List
metadata: {}

Upon doing kompose up -

$ kompose up -f script/test/fixtures/configmap/docker-compose.yaml 
INFO We are going to create Kubernetes Deployments, Services and PersistentVolumeClaims for your Dockerized application. If you need different kind of resources, use the 'kompose convert' and 'kubectl create -f' commands instead. 
 
INFO Deploying application in "default" namespace 
INFO Successfully created Service: redis          
FATA Error while deploying application: Deployment.apps "redis" is invalid: [spec.template.spec.containers[0].env[0].valueFrom.configMapKeyRef.name: Invalid value: "-/bar-env": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.template.spec.containers[0].env[1].valueFrom.configMapKeyRef.name: Invalid value: "-/bar-env": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]

surajnarwade · 2017-08-30T06:26:39Z

pkg/kobject/kobject.go

 	ContainerName   string
 	Image           string              `compose:"image"`
 	Environment     []EnvVar            `compose:"environment"`
+	EnvFile         []string            `compose:""`


we can add compose:"env_file" as it is valid key

surajnarwade · 2017-08-30T06:30:17Z

version 2 docker -compose file,

version: '3'
  
services:
  redis:
    image: 'bitnami/redis:latest'
    environment:
      - ALLOW_EMPTY_PASSWORD=no
    # Env file will override environment / warn!
    env_file:
      - ./bar.env
    labels:
      kompose.service.type: nodeport
    ports:
      - '6379:6379'

doesn't create configmaps,

$ kompose convert 
INFO Kubernetes file "redis-service.yaml" created 
INFO Kubernetes file "redis-deployment.yaml" created

cdrage · 2017-08-30T13:45:06Z

Your example shows Version 3 @surajnarwade

Yeah, at the moment it doesn't support Version 2. I will have to add that. Baby steps! 👍

Can you do a review on the basis Version 3?

surajnarwade · 2017-08-31T13:45:04Z

@cdrage , my bad about version

👍 for baby step

cdrage · 2017-09-05T15:58:16Z

@containscafeine @kadel @surajnarwade @surajssd ready for review 👍

edit: next sprint I will add v2 support, i have updated the conversion.md matrix accordingly.

cdrage · 2017-09-06T19:09:18Z

Reason why this is not in v2 is because of this: docker/libcompose#488

concaf · 2017-09-07T04:08:12Z

@cdrage still getting the error in #799 (review) :(

cdrage · 2017-09-07T12:51:15Z

@containscafeine should work now, i've created a new function in order to format the new env name's correctly, feel free to do another review!

github.com/kubernetes/kompose  add-configmap ✗                                                                                                                                                                                                                          19m ⚑ ◒  
▶ ./kompose up
INFO We are going to create Kubernetes Deployments, Services and PersistentVolumeClaims for your Dockerized application. If you need different kind of resources, use the 'kompose convert' and 'kubectl create -f' commands instead. 
 
INFO Deploying application in "default" namespace 
INFO Successfully created Service: redis          
INFO Successfully created Deployment: redis       
INFO Successfully created Config Map: bar-env     

Your application has been deployed to Kubernetes. You can run 'kubectl get deployment,svc,pods,pvc' for details.

github.com/kubernetes/kompose  add-configmap ✗                                                                                                                                                                                                                          19m ⚑ ◒  
▶ kubectl get cm
NAME      DATA      AGE
bar-env   2         5s

cdrage · 2017-09-26T19:25:31Z

ignoring vendoring updates, this PR needs review

edit: vendoring updates blocked on #835 due to running into errors with logrus + gojsonschema

@kadel @containscafeine @surajnarwade @surajssd please review

surajssd · 2017-10-05T10:15:09Z

pkg/transformer/kubernetes/kubernetes.go

+			APIVersion: "v1",
+		},
+		ObjectMeta: api.ObjectMeta{
+			Name: envName,


Can this be named like other objects kompose creates? appName - maifest type.yaml

Here also we can use same format, just add appName before envName ?

so appName-configmap-env?

@cdrage this sounds good!

surajnarwade · 2017-10-06T13:14:13Z

works for me 👍

When using env_file with Docker Compose, a ConfigMap will be generated For example: ```sh ▶ ./kompose convert -f script/test/fixtures/configmaps/docker-compose.yml INFO Kubernetes file "redis-service.yaml" created INFO Kubernetes file "redis-deployment.yaml" created INFO Kubernetes file "foo-env-configmap.yaml" created INFO Kubernetes file "bar-env-configmap.yaml" created ``` File: ```yaml version: '3' services: redis: image: 'bitnami/redis:latest' environment: - ALLOW_EMPTY_PASSWORD=no # Env file will override environment / warn! env_file: - "foo.env" - bar.env labels: kompose.service.type: nodeport ports: - '6379:6379' ``` To: ```yaml apiVersion: v1 data: ALLOW_EMPTY_PASSWORD: "yes" kind: ConfigMap metadata: creationTimestamp: null name: foo-env ``` ```yaml ... - env: - name: ALLOW_EMPTY_PASSWORD valueFrom: configMapKeyRef: key: ALLOW_EMPTY_PASSWORD name: foo-env ```

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 25, 2017

kompose-bot requested review from concaf and procrypt August 25, 2017 19:52

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 25, 2017

cdrage force-pushed the add-configmap branch from db3d240 to 4a3df1a Compare August 28, 2017 14:50

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 28, 2017

cdrage force-pushed the add-configmap branch 2 times, most recently from d070dde to 15b2de9 Compare August 28, 2017 15:22

cdrage force-pushed the add-configmap branch from 15b2de9 to 7450cdf Compare August 29, 2017 13:22

concaf suggested changes Aug 30, 2017

View reviewed changes

surajnarwade reviewed Aug 30, 2017

View reviewed changes

cdrage force-pushed the add-configmap branch from 7450cdf to f8dccab Compare August 31, 2017 14:16

cdrage force-pushed the add-configmap branch 2 times, most recently from 7fc8fc8 to b0f2b00 Compare September 7, 2017 12:50

cdrage force-pushed the add-configmap branch 4 times, most recently from 9b8e8d1 to 27de17c Compare September 29, 2017 14:54

cdrage force-pushed the add-configmap branch 3 times, most recently from 1bc04b5 to 51bece6 Compare September 29, 2017 16:02

surajssd reviewed Oct 5, 2017

View reviewed changes

surajnarwade approved these changes Oct 6, 2017

View reviewed changes

surajnarwade added the rebase needed label Oct 9, 2017

concaf approved these changes Oct 10, 2017

View reviewed changes

cdrage force-pushed the add-configmap branch from 51bece6 to 973fc49 Compare October 10, 2017 16:37

cdrage force-pushed the add-configmap branch from 973fc49 to f4bfe1f Compare October 10, 2017 16:40

Update vendoring

28601fa

cdrage force-pushed the add-configmap branch from 720366b to 28601fa Compare October 10, 2017 16:56

cdrage merged commit 28c8d78 into kubernetes:master Oct 10, 2017

cdrage mentioned this pull request Oct 10, 2017

Convert docker-compose env files into configmaps #308

Closed

darKoram mentioned this pull request Jan 14, 2018

extended env_file capability to in-docker-compose.yml substitutions #909

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add env_file + ConfigMaps feature to Kompose #799

Add env_file + ConfigMaps feature to Kompose #799

cdrage commented Aug 25, 2017

kompose-bot commented Aug 25, 2017

cdrage commented Aug 25, 2017 •

edited

Loading

cdrage commented Aug 28, 2017

concaf left a comment

surajnarwade Aug 30, 2017

surajnarwade commented Aug 30, 2017

cdrage commented Aug 30, 2017

surajnarwade commented Aug 31, 2017

cdrage commented Sep 5, 2017 •

edited

Loading

cdrage commented Sep 6, 2017

concaf commented Sep 7, 2017

cdrage commented Sep 7, 2017 •

edited

Loading

cdrage commented Sep 26, 2017 •

edited

Loading

surajssd Oct 5, 2017

cdrage Oct 5, 2017

surajssd Oct 10, 2017

surajnarwade commented Oct 6, 2017

Add env_file + ConfigMaps feature to Kompose #799

Add env_file + ConfigMaps feature to Kompose #799

Conversation

cdrage commented Aug 25, 2017

kompose-bot commented Aug 25, 2017

cdrage commented Aug 25, 2017 • edited Loading

cdrage commented Aug 28, 2017

concaf left a comment

Choose a reason for hiding this comment

surajnarwade Aug 30, 2017

Choose a reason for hiding this comment

surajnarwade commented Aug 30, 2017

cdrage commented Aug 30, 2017

surajnarwade commented Aug 31, 2017

cdrage commented Sep 5, 2017 • edited Loading

cdrage commented Sep 6, 2017

concaf commented Sep 7, 2017

cdrage commented Sep 7, 2017 • edited Loading

cdrage commented Sep 26, 2017 • edited Loading

surajssd Oct 5, 2017

Choose a reason for hiding this comment

cdrage Oct 5, 2017

Choose a reason for hiding this comment

surajssd Oct 10, 2017

Choose a reason for hiding this comment

surajnarwade commented Oct 6, 2017

cdrage commented Aug 25, 2017 •

edited

Loading

cdrage commented Sep 5, 2017 •

edited

Loading

cdrage commented Sep 7, 2017 •

edited

Loading

cdrage commented Sep 26, 2017 •

edited

Loading