Skip to content
This repository has been archived by the owner on Aug 17, 2023. It is now read-only.

KfAwsPlugin "managedRelationDatabase" does not work in KF 1.2.0 #463

Open
karlschriek opened this issue Dec 1, 2020 · 4 comments
Open

Comments

@karlschriek
Copy link

I spent some time today rolling out KF 1.2.0 on EKS using exclusively RDS databases instead of an on-cluster MySQL deployment.

The latest kfctl has a "managedRelationDatabase" field that can be set in the kfdef, such as here:

  plugins:
    - kind: KfAwsPlugin
      metadata:
        name: aws
      spec:
        auth:
          cognito:
            certArn: REPLACE_WITH_CERT_ARN
            cognitoAppClientId: REPLACE_WITH_COGNITO_APP_CLIENT_ID
            cognitoUserPoolArn: REPLACE_WITH_COGNITO_USER_POOL
            cognitoUserPoolDomain: REPLACE_WITH_COGNITO_DOMAIN
        region: REPLACE_WITH_AWS_REGION
        enablePodIamPolicy: true
        roles:
        - REPLACE_WITH_ROLE_NAME
        managedRelationDatabase:
          host: REPLACE_WITH_METADATA_RDS_ENDPOINT
          database: REPLACE_WITH_METADATA_RDS_DATABASE
          username: REPLACE_WITH_METADATA_DB_USERNAME
          password: REPLACE_WITH_METADATA_DB_PASSWORD
          port: REPLACE_WITH_METADATA_DB_PORT

This appears to be meant as a way to easily set the required overlays and ensure that the "external-mysql" overlays are rolled out instead of the default "db" ones. However, this did not work as it seems to update configmaps "metadata-config" and "api-service-config" that do not get rolled out at all and are not referenced by any of the points.

So I went about it manually. The following manually changes fixed it for me:

kfctl build -V -f kfctl_aws_cognito.1.2.0.yaml
sh ./configure_rds_overlays.sh
kfctl apply -V -f kfctl_aws_cognito.1.2.0.yaml

where configure_rds_overlays.sh has the following contents:

cat <<EoF > ${KF_CONFIGS_DIR}/.cache/manifests/manifests-1.2.0/pipeline/installs/generic/params.env
bucketName=mlpipeline
dbHost=${RDS_ENDPOINT}
dbPort=${RDS_PORT}
mlmdDb=${METADATA_DB_NAME}
cacheDb=${CACHE_DB_NAME}
pipelineDb=${PIPELINES_DB_NAME}
EoF

cat <<EoF > ${KF_CONFIGS_DIR}/.cache/manifests/manifests-1.2.0/pipeline/installs/generic/params-db-secret.env
username=${RDS_USERNAME}
password=${RDS_PASSWORD}
EoF

cat <<EoF > ${KF_CONFIGS_DIR}/.cache/manifests/manifests-1.2.0/metadata/overlays/external-mysql/params.env
MYSQL_HOST=${RDS_ENDPOINT}
MYSQL_DATABASE=${METADATA_DB_NAME}
MYSQL_PORT=${RDS_PORT}
MYSQL_ALLOW_EMPTY_PASSWORD=true
EoF

cat <<EoF > ${KF_CONFIGS_DIR}/.cache/manifests/manifests-1.2.0/metadata/overlays/external-mysql/secrets.env
MYSQL_USERNAME=${RDS_USERNAME}
MYSQL_PASSWORD=${RDS_PASSWORD}
EoF

cat <<EoF > ${KF_CONFIGS_DIR}/.cache/manifests/manifests-1.2.0/metadata/v3/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../overlays/external-mysql
- ../overlays/istio/virtual-service-metadata-grpc.yaml
configurations:
- params.yaml
EoF

It seems that the above (or at least something similar) is the behaviour that setting managedRelationDatabase is intended to invoke.

@karlschriek
Copy link
Author

@PatrickXYS, @Jeffwan

@PatrickXYS
Copy link
Member

        managedRelationDatabase:
          host: REPLACE_WITH_METADATA_RDS_ENDPOINT
          database: REPLACE_WITH_METADATA_RDS_DATABASE
          username: REPLACE_WITH_METADATA_DB_USERNAME
          password: REPLACE_WITH_METADATA_DB_PASSWORD
          port: REPLACE_WITH_METADATA_DB_PORT

I don't really think this will work, because those are interfaces we pre-defined, but not yet implement

@karlschriek
Copy link
Author

Yes, that is the conclusion I came to as well. Doesn't look like it would be too hard to implement though? Essentially instead of creating the config maps as it currently does it does needs to adjust the overlays as described above. That would be quite useful since one then only needs to update the kfdef

@PatrickXYS
Copy link
Member

All the other parts of implementation are finished, except how to pass password to replace K8s secret, it's fairly easy in the k8s onfigmap, however not in k8s secret.

It's about kfctl implementation, we generate all the manifests (including secrets and configmap) at first, then place them with our KfAwsPlugin parameter.

I touched it previously, might need to spend some time figuring it out.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants