Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using core env vars instead of hardcoded #14

Merged
merged 2 commits into from
Jan 3, 2023
Merged

Conversation

jhsinger-klotho
Copy link
Contributor

• Does any part of it require special attention?
• Does it relate to or fix any issue? makes progress towards #601. It doesnt pull back all of the environment variables, but removes the need of hardcoding for redis, orm, helm env vars.

A lot of this is unit tests, but heres what isnt

python and js persist - we use the new method generateEnvVar (for orm and redis respectively) and then attach that to the exec unit.

Kubernetes - We attach the core.Environment Variable to the value so that our infra understands what it will need to inject. This prevents more hardcoding and creation if the same env vars as before without the _

DeployLib and index.tmpl - We always pass the unit.EnvironmentVariables to whichever function is creating the exec unit

Standard checks

  • Unit tests: Any special considerations? Added around code which was touched and running https://github.com/klothoplatform/klotho/actions/runs/3753900452 at the moment, so would like to see that pass. Also running helm testing manually
  • Docs: Do we need to update any docs, internal or public? No, all internal
  • Backwards compatibility: Will this break existing apps? If so, what would be the extra work required to keep them working? Yes, just changing the method in which we get env vars. We now validate if there are multiple resources for the same capabillity so using a single env var for redis cluster and node will be fine since they are both persist.

@jhsinger-klotho
Copy link
Contributor Author

ran go generate locally and theres more changes, will need to include but leaving out for now

EnvironmentVariableDirective = "environment_variables"
EnvironmentVariableDirective = "environment_variables"
ORM_ENV_VAR_NAME_SUFFIX = "_PERSIST_ORM_CONNECTION"
REDIS_PORT_ENV_VAR_NAME_SUFFIX = "_PERSIST_REDIS_PORT"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOuble checking we don't have a need to differentiate between redis cluster vs node

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#12 should ensure theres never 2 persist resources with the same id, so we should never run into conflicts.

Kind: deployment.Kind,
Type: string(EnvironmentVariableTransformation),
Key: v,
EnvironmentVariable: envVar,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm this field feels very specific to env vars, do we need to have the envVar value here? Isn't the rest of the fields the same and the key would be just be the envVar.name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well the reason im including all of it is because our pulumi plugin knows how to process our core.Environment struct as a dependency environment variable and thats the value we need here. The key remainder is for helm to know how to set the values in the install directory, but it doesnt know how to get an rds instances connection string, etc

@@ -193,6 +193,13 @@ func (chart *KlothoHelmChart) handleExecutionUnit(unit *HelmExecUnit, eu *core.E
return nil, err
}
values = append(values, upstreamValues...)

persistValues, err := unit.handlePersistForExecUnit(deps)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeh, this seems like a better place for this 👍

envVars = append(envVars, core.GenerateOrmConnStringEnvVar(target.Name, target.Kind))
case string(core.PersistRedisNodeKind):
envVars = append(
envVars,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we don't need to distinguish between node/cluster then we can collapse these into

case string(core.PersistRedisNodeKind):
case string(core.PersistRedisClusterKind):
     ... logic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, good catch, will update

Namespace: "default",
},
deploymentYaml: `apiVersion: apps/v1
kind: Deployment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note - do you have an easy way of working w/ these charts in string form? When I was working on the tests like 80% of the pain was getting the output to correctly match w/ the spacing vs tabbing and stuff my editor was doing by default (like setting a tab when I hit enter)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah you always have to use spaces, its a bit annoying, but would be really hard to test all of these cases end to end with integ tests so im just trying to get things in place here. We maybe could write some k8s testing suite to check our compiled output instead of unit tests though

@ewucc
Copy link
Contributor

ewucc commented Dec 22, 2022

No blocking comments

@github-actions
Copy link

Package Line Rate Health
github.com/klothoplatform/klotho/pkg/analytics 2%
github.com/klothoplatform/klotho/pkg/annotation 24%
github.com/klothoplatform/klotho/pkg/core 21%
github.com/klothoplatform/klotho/pkg/env_var 82%
github.com/klothoplatform/klotho/pkg/exec_unit 41%
github.com/klothoplatform/klotho/pkg/infra/kubernetes 58%
github.com/klothoplatform/klotho/pkg/infra/kubernetes/helm 52%
github.com/klothoplatform/klotho/pkg/input 63%
github.com/klothoplatform/klotho/pkg/lang 37%
github.com/klothoplatform/klotho/pkg/lang/dockerfile 0%
github.com/klothoplatform/klotho/pkg/lang/golang 9%
github.com/klothoplatform/klotho/pkg/lang/javascript 46%
github.com/klothoplatform/klotho/pkg/lang/python 60%
github.com/klothoplatform/klotho/pkg/lang/yaml 0%
github.com/klothoplatform/klotho/pkg/logging 7%
github.com/klothoplatform/klotho/pkg/multierr 95%
github.com/klothoplatform/klotho/pkg/provider/aws 54%
github.com/klothoplatform/klotho/pkg/runtime 75%
github.com/klothoplatform/klotho/pkg/static_unit 22%
github.com/klothoplatform/klotho/pkg/validation 70%
Summary 41% (3507 / 8482)

@jhsinger-klotho jhsinger-klotho merged commit bd9c537 into main Jan 3, 2023
@jhsinger-klotho jhsinger-klotho deleted the provider_env_vars branch January 3, 2023 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants