Jenkins vault plugin cannot read secrets that aren't KV if engine version 2 is used

[ahartma1]

Jenkins and plugins versions report

Environment

Jenkins: 2.462.2
OS: Linux - 5.15.153.1-microsoft-standard-WSL2
Java: 17.0.12 - Ubuntu (OpenJDK 64-Bit Server VM)
---
amazon-ecr:1.136.v914ea_5948634
ant:511.v0a_a_1a_334f41b_
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.3.1-117.v4d95117cd34f
asm-api:9.7-33.v4d23ef79fcc8
authentication-tokens:1.119.v50285141b_7e1
aws-credentials:231.v08a_59f17d742
aws-java-sdk-ec2:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-ecr:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-minimal:1.12.767-467.vb_e93f0c614b_6
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_
branch-api:2.1178.v969d9eb_c728e
build-timeout:1.33
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloud-stats:336.v788e4055508b_
cloudbees-folder:6.951.v5f91d88d76b_b_
command-launcher:115.vd8b_301cc15d0
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.12.0-129.v99a_50df237f7
credentials:1371.vfee6b_095f0a_3
credentials-binding:681.vf91669a_32e45
dark-theme:479.v661b_1b_911c01
display-url-api:2.204.vf6fddd8a_8b_e9
docker-commons:443.v921729d5611d
docker-java-api:3.3.6-90.ve7c5c7535ddd
docker-plugin:1.6.2
docker-workflow:580.vc0c340686b_54
durable-task:568.v8fb_5c57e8417
echarts-api:5.5.1-1
eddsa-api:0.3.0-4.v84c6f0f4969e
email-ext:1814.v404722f34263
font-awesome-api:6.6.0-2
git:5.4.1
git-client:5.0.0
github:1.40.0
github-api:1.321-468.v6a_9f5f2d5a_7e
github-branch-source:1797.v86fdb_4d57d43
gradle:2.12.1
gson-api:2.11.0-41.v019fcf6125dc
hashicorp-vault-pipeline:1.4
hashicorp-vault-plugin:370.v946b_53544a_30
instance-identity:185.v303dc7c645f9
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:80.v8a_dee33ed6f0
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-58.v62e3e85b_a_655
junit:1300.v03d9d8a_cf1fb_
ldap:725.v3cb_b_711b_1a_ef
mailer:472.vf7c289a_4b_420
matrix-auth:3.2.2
matrix-project:832.va_66e270d2946
metrics:4.2.21-451.vd51df8df52ec
mina-sshd-api-common:2.13.2-125.v200281b_61d59
mina-sshd-api-core:2.13.2-125.v200281b_61d59
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.11
pipeline-aggregator-view:104.v94a_e5f6cdb_c3
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-graph-view:340.v28cecee8b_25f
pipeline-groovy-lib:730.ve57b_34648c63
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83
pipeline-stage-view:2.34
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:4.1.0
resource-disposer:0.23
scm-api:696.v778d637b_a_762
script-security:1361.v913100720139
snakeyaml-api:2.3-123.v13484c65210a_
ssh-credentials:343.v884f71d78167
ssh-slaves:2.973.v0fa_8c0dea_f9f
sshd:3.330.vc866a_8389b_58
structs:338.v848422169819
terraform:1.0.10
theme-manager:262.vc57ee4a_eda_5d
timestamper:1.27
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1336.vee415d95c521
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3961.ve48ee2c44a_b_3
workflow-durable-task-step:1371.vb_7cec8f3b_95e
workflow-job:1436.vfa_244484591f
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:678.v3ee58b_469476
workflow-support:920.v59f71ce16f04
ws-cleanup:0.46

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu 24.04 LTS on WSL2

Reproduction steps

1. Install the Vault plugin
2. Set Approle creds as global in credentials
3. Use a Jenkinsfile from a github repository like this:

pipeline {
    agent any
    environment {
        VAULT_ADDR = 'http://localhost:8200' // Replace with your Vault server URL
        VAULT_CREDENTIAL_ID = 'vault-credentials'
        VAULT_AWS_PATH = 'aws/creds/aws-eks' // Path to the AWS credentials in Vault

        
    }
        stage('Fetch AWS Credentials') {
            agent any
            steps {
                script {                
                    def configuration = [vaultUrl: env.VAULT_ADDR,
                        vaultCredentialId: env.VAULT_CREDENTIAL_ID,
                        engineVersion: 2]
                    // Authenticate with Vault using AppRole and fetch AWS credentials
                    withVault([vaultSecrets: [[path: env.VAULT_AWS_PATH, secretValues: [
                        [envVar: 'AWS_ACCESS_KEY_ID', vaultKey: 'access_key'],
                        [envVar: 'AWS_SECRET_ACCESS_KEY', vaultKey: 'secret_key']
                    ]]]]) {
                    }
                }
            }
        }
     #...other stages
}

Expected Results

withVault authenticates, then retrieves the credentials and sets the env vars for use with the next stage (in my case, a terraform init and apply.

Actual Results

{
  "auth": {
    "client_token": "hmac-sha256:bbbfbc9b7825659441257739fe804b2c75776ada3f345cca9deb0800f7541d53",
    "display_name": "approle",
    "entity_id": "d5b665f7-bcc5-da15-3e73-d658f23ddb75",
    "metadata": {
      "role_name": "aws"
    },
    "policies": [
      "aws-eks",
      "default"
    ],
    "policy_results": {
      "allowed": false
    },
    "token_policies": [
      "aws-eks",
      "default"
    ],
    "token_issue_time": "2024-09-13T14:16:45-05:00",
    "token_ttl": 2764800,
    "token_type": "batch"
  },
  "error": "1 error occurred:\n\t* permission denied\n\n",
  "request": {
    "client_id": "d5b665f7-bcc5-da15-3e73-d658f23ddb75",
    "client_token": "hmac-sha256:bbbfbc9b7825659441257739fe804b2c75776ada3f345cca9deb0800f7541d53",
    "id": "85f7dcd3-fcf9-5f57-2def-9f72fe6dd09d",
    "mount_class": "secret",
    "mount_point": "aws/",
    "mount_running_version": "v1.17.5+builtin.vault",
    "mount_type": "aws",
    "namespace": {
      "id": "root"
    },
    "operation": "read",
    "path": "aws/data/creds/aws-eks",
    "remote_address": "127.0.0.1",
    "remote_port": 60746
  },
  "time": "2024-09-13T20:49:09.385094867Z",
  "type": "request"
}
{
  "auth": {
    "client_token": "hmac-sha256:bbbfbc9b7825659441257739fe804b2c75776ada3f345cca9deb0800f7541d53",
    "display_name": "approle",
    "entity_id": "d5b665f7-bcc5-da15-3e73-d658f23ddb75",
    "metadata": {
      "role_name": "aws"
    },
    "policies": [
      "aws-eks",
      "default"
    ],
    "policy_results": {
      "allowed": false
    },
    "token_policies": [
      "aws-eks",
      "default"
    ],
    "token_issue_time": "2024-09-13T14:16:45-05:00",
    "token_ttl": 2764800,
    "token_type": "batch"
  },
  "error": "1 error occurred:\n\t* permission denied\n\n",
  "request": {
    "client_id": "d5b665f7-bcc5-da15-3e73-d658f23ddb75",
    "client_token": "hmac-sha256:bbbfbc9b7825659441257739fe804b2c75776ada3f345cca9deb0800f7541d53",
    "id": "85f7dcd3-fcf9-5f57-2def-9f72fe6dd09d",
    "mount_class": "secret",
    "mount_point": "aws/",
    "mount_running_version": "v1.17.5+builtin.vault",
    "mount_type": "aws",
    "namespace": {
      "id": "root"
    },
    "operation": "read",
    "path": "aws/data/creds/aws-eks",
    "remote_address": "127.0.0.1",
    "remote_port": 60746
  },
  "response": {
    "data": {
      "error": "hmac-sha256:0a1bbbc12169b811ce7cc4b5edddd50adb4e88e9cafb4f217e68f16795f516e9"
    },
    "mount_class": "secret",
    "mount_point": "aws/",
    "mount_running_plugin_version": "v1.17.5+builtin.vault",
    "mount_type": "aws"
  },
  "time": "2024-09-13T20:49:09.385482505Z",
  "type": "response"
}

As you can see, path is "aws/data/creds/aws-eks" instead of "aws/creds/aws-eks"

This leads to permission denied error:

Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: afe1173f-14a2-474d-a400-50e539b60623
com.datapipe.jenkins.vault.exception.VaultPluginException: Access denied to Vault path 'aws/creds/aws-eks'
	at PluginClassLoader for hashicorp-vault-plugin//com.datapipe.jenkins.vault.VaultAccessor.responseHasErrors(VaultAccessor.java:273)
	at PluginClassLoader for hashicorp-vault-plugin//com.datapipe.jenkins.vault.VaultAccessor.retrieveVaultSecrets(VaultAccessor.java:212)
	at PluginClassLoader for hashicorp-vault-plugin//com.datapipe.jenkins.vault.VaultBindingStep$Execution.doStart(VaultBindingStep.java:115)
	at PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Finished: FAILURE

This leads me to believe that this plugin is only developed for the K/V engine, but that is not specified anywhere in the documentation. This plugin needs to either be renamed or be coded to handle different types of secrets individually. There are many secrets that one can retrieve from vault, and most do not have to have /data/ interjected into the path.

Anything else?

No response

Are you interested in contributing a fix?

I don't write Java and I couldn't find where the class was actually transforming the path, but it is doing so and shouldn't in the case of a non K/V2 secrets engine. Perhaps the logic could just be changed slightly to first check at secret_mount/data/secret_path, then if it doesn't find it there, try the original path specified. This would make it work for all the other secrets engines.