Custom actions ignores container volumes

[Isakdl]

When I run the custom action my container settings in my job manifest are overridden by the container specification in the custom actions. When I use the overrides keyword it does not seem to apply these overrides to the final yaml file.

Expected behavior

That the container mounts the volumes.

Actual behavior

The volume mounts are overridden and ignored.

Information

Similar issue: #8974

• Skaffold version: 2.11.1
• Operating system: MacOS Sonoma 14.4.1
• Installed via: Homebrew
• Contents of skaffold.yaml:

apiVersion: skaffold/v4beta7
kind: Config
metadata:
  name: gke-automation
manifests:
  rawYaml:
    - deployment.yaml
    - service.yaml
    - http-route.yaml
    - secret.yaml
    - service-account.yaml
deploy:
  kubectl: {}
customActions:
  - name: custom-job
    executionMode:
      kubernetesCluster:
        overrides: '{"spec":{"template":{"spec":{"containers":[{"volumeMounts":[{"name":"config","mountPath":"/config/config.yaml","subPath":"config.yaml"},{"name":"secrets","mountPath":"/config/secrets.yaml","subPath":"secrets.yaml"}]}]}}}}'
        jobManifestPath: ./job.yaml
    containers:
      - name: migration
        image: image

Job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: migration
  namespace: app-space
  labels:
    app: migration
spec:
  backoffLimit: 0
  template:
    metadata:
      annotations:
        gke-gcsfuse/volumes: "true"
      labels:
        app: migration
    spec:
      serviceAccountName: service-account
      image: image
      containers:
        - name: migration
          volumeMounts:
            - name: config
              mountPath: /config/config.yaml
              subPath: config.yaml
            - name: secrets
              mountPath: /config/secrets.yaml
              subPath: secrets.yaml
      volumes:
        - name: config
          csi:
            driver: gcsfuse.csi.storage.gke.io
            readOnly: true
            volumeAttributes:
              bucketName: config-bucket
              mountOptions: "implicit-dirs"
              gcsfuseLoggingSeverity: warning
        - name: secrets
          csi:
            driver: secrets-store-gke.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: secrets

Steps to reproduce the behavior

Setup a volume mount for the container in a custom actions.

Run the custom action - no volume is mounted.

[jdonzallaz]

We have the same problem. Any news from the team?

[edencybellum]

Same here

[timorickli]

I have the same issue. Is there an easy and clean fix?

[gemoya]

Same here.
I'm trying to pass an envFrom list to the job but skaffold is ignoring the overrides content

[cmanzi-bestow]

Same issue here. Why is the containers property required for using custom job manifests when it will overwrite other properties that we already have defined in our custom Job manifest? Doesn't make any sense.

FYI @cglenn36

Edit: Right now we have no workaround for this...I am going to try to see if I can debug what's going on here. From what we've seen, it looks like it does parse and validate the overrides, but they just never actually get overlayed in the resulting manifest for some reason.

[cmanzi-bestow]

In case anyone is watching, I have a PR open with a proposed fix, just working on tests and awaiting feedback. But in case anyone can't wait the fix is pretty simple, and it eliminates the need for using the overrides.