Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic in k8s integration #200

Closed
cgimenes opened this issue Jan 6, 2023 · 8 comments
Closed

Panic in k8s integration #200

cgimenes opened this issue Jan 6, 2023 · 8 comments
Assignees
Labels
🐞 bug Something isn't working

Comments

@cgimenes
Copy link

cgimenes commented Jan 6, 2023

Log from infisical-operator-controller-manager:

panic: runtime error: index out of range [1] with length 1 [recovered]
	panic: runtime error: index out of range [1] with length 1

goroutine 229 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:118 +0x1f4
panic({0x15d71a0, 0xc0005a6738})
	/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/Infisical/infisical/k8-operator/packages/api.GetSecretsFromAPIUsingInfisicalToken({0xc000282e40?, 0xc000521770?}, {0xc0006a8110, 0x4}, {0xc0005a6618, 0x18}, {0xc0005a83e0, 0x1d})
	/workspace/packages/api/api.go:32 +0xcd9
github.com/Infisical/infisical/k8-operator/packages/api.GetAllEnvironmentVariables({0xc0005a6618?, 0xc0003e2500?}, {0xc0006a8110?, 0xc0006a8118?}, {0xc000282e40?, 0xc0006a8120?}, {0xc0005a83e0?, 0x0?})
	/workspace/packages/api/api.go:17 +0x45
github.com/Infisical/infisical/k8-operator/controllers.(*InfisicalSecretReconciler).ReconcileInfisicalSecret(_, {_, _}, {{{0x13f2a9f, 0xf}, {0xc000120e80, 0x1e}}, {{0xc000510f30, 0x10}, {0x0, ...}, ...}, ...})
	/workspace/controllers/infisicalsecret_helper.go:105 +0x2b1
github.com/Infisical/infisical/k8-operator/controllers.(*InfisicalSecretReconciler).Reconcile(0xc000010570, {0x19097f8, 0xc00037dad0}, {{{0xc000510f40?, 0x10?}, {0xc000510f30?, 0x40dae7?}}})
	/workspace/controllers/infisicalsecret_controller.go:58 +0x170
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1909750?, {0x19097f8?, 0xc00037dad0?}, {{{0xc000510f40?, 0x1600280?}, {0xc000510f30?, 0x4045d4?}}})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121 +0xc8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000001540, {0x1909750, 0xc00003fd40}, {0x154c5a0?, 0xc0004308a0?})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320 +0x33c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000001540, {0x1909750, 0xc00003fd40})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:230 +0x333

Checked the code from:

splitToken := strings.Split(infisicalToken, ",")

And noticed that my service token doesn't have any comma.
Did the token format change?

@vmatsiiako
Copy link
Collaborator

vmatsiiako commented Jan 6, 2023

HI @cgimenes! Really sorry about this. Yes, the service token format has changed. it is now separated with dots instead of commas. Plus, it also embeds a projectId inside, so you don't need to specify it yourself. You can generate it in the settings of your project.

@maidul98 would be able to look into this issue in a bit

@maidul98
Copy link
Collaborator

maidul98 commented Jan 7, 2023

@cgimenes Yes, we have recently updated our service tokens to be more rubust. Because of this change we also need to update the K8 operator to accept the new token. I will provide an update when this update is complete here!

@maidul98 maidul98 self-assigned this Jan 7, 2023
@maidul98 maidul98 added the 🐞 bug Something isn't working label Jan 7, 2023
@bartosz-maciaszek
Copy link

bartosz-maciaszek commented Jan 14, 2023

Same issue here. I have a new token (st.aaaa.bbbb.cccc) that seems to be working correctly (INFISICAL_TOKEN=st.aaaa.bbbb.cccc infisical export returns the secrets nicely), but I'm not able to fetch the secrets from within my kubernetes cluster. kubectl describe infisicalSecret infisicalsecret-foobar returns:

(...)
Spec:
  Environment:  Production
  Host API:     https://infisical.com/api
  Managed Secret Reference:
    Secret Name:       managed-secret
    Secret Namespace:  yyyyyyyy
  Project Id:          xxxxxxxxxxxxxxxxxxxxx
  Token Secret Reference:
    Secret Name:       infisical-token
    Secret Namespace:  default
Status:
  Conditions:
    Last Transition Time:  2023-01-14T19:00:00Z
    Message:               Infisical controller has located the Infisical token in provided Kubernetes secret
    Reason:                OK
    Status:                True
    Type:                  secrets.infisical.com/LoadedInfisicalToken
Events:                    <none>

Looks like the token is found, but managed-secret never gets created. The infisical-operator-controller-manager pod keeps restarting and in the log I can see the same error that @cgimenes mentioned:

panic: runtime error: index out of range [1] with length 1 [recovered]
	panic: runtime error: index out of range [1] with length 1

@maidul98
Copy link
Collaborator

@bartosz-maciaszek This is due to the new format for service token which the old operator does not understand. I'm currently working on updating the operator to resolve this issue. We will also have auto redeployment when secrets update, stay tuned!

@bartosz-maciaszek
Copy link

@maidul98 Thanks for the quick reply! Is there a workaround, like an ability to generate a service token compatible with the current version of the operator?

@maidul98
Copy link
Collaborator

The current UI only produces the new service tokens. I'd say give me a few more hours as i'm currently testing out the improved operator. Once done, i'll release it so that you can start utilizing it

@maidul98
Copy link
Collaborator

This issue has been fixed along with other improvoments. Please have a look at https://infisical.com/docs/integrations/platforms/kubernetes

@bartosz-maciaszek
Copy link

bartosz-maciaszek commented Jan 17, 2023

Thank you, @maidul98. I upgraded the secrets-operator to v0.1.2 and it's working now. I also had to change the hostAPI in the InfisicalSecret resource to https://app.infisical.com/api (previously: https://infisical.com/api).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants