Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jobs HTTP API cannot unmarshal string into Go value of type api.X #24159

Open
sjiveson opened this issue Oct 9, 2024 · 0 comments
Open

Jobs HTTP API cannot unmarshal string into Go value of type api.X #24159

sjiveson opened this issue Oct 9, 2024 · 0 comments
Labels
type/bug

Comments

@sjiveson
Copy link

sjiveson commented Oct 9, 2024 •
edited
Loading

Nomad version

v1.7.7+ent

Operating system and Environment details

RHEL 8 on AWS Cloud

Issue

I'm attempting to use the Jobs HTTP API to manage jobs via a Python v3 (using requests) based Lambda. All requests fail with an error similar to this (depending on the endpoint):

json: cannot unmarshal string into Go value of type api.JobRegisterRequest

However, requests made using curl to the same endpoints, using the same jobspec succeed.

Reproduction steps

Convert HCL2 jobspec to JSON:

nomad job run -output job.hcl > job.json

Curl endpoint:

curl --data @job.json --header "X-Nomad-Token: token_value" https://my_fqdn:4646/v1/job/job_name?namespace=my_namespace

Use Python requests - the request_spec is the content of the job.json used with the curl:

                    log.debug('Connecting to Nomad')
                    response = requests.post(
                        url,
                        json = request_spec,
                        headers = {
                            "X-Nomad-Token": nomad_token
                        },
                        verify=False,
                        timeout=10)
                    code = response.status_code

                    if code != 200:
                        log.critical('Request rejected with a HTTP status code of: %s returned', code)
                        raise ConnectionError(response.content)

Expected Result

With curl:

{"Annotations":{"DesiredTGUpdates":{"job_name":{"Ignore":0,"Place":1,"Migrate":0,"Stop":0,"InPlaceUpdate":0,"DestructiveUpdate":0,"Canary":0,"Preemptions":0}},"PreemptedAllocs":null},"FailedTGAllocs":null,"JobModifyIndex":35994111,"CreatedEvals":null,"Diff":{"Type":"Edited","ID":"job_name","Fields":[{"Type":"Edited","Name":"Meta[run_uuid]","Old":"a2f6412a-7ac9-48ae-8214-ed605e1b8a02","New":"b496a872-6c7a-4d44-b5ae-c5c9ab0a7eea","Annotations":null}],"Objects":null,"TaskGroups":[{"Type":"None","Name":"something-group","Fields":null,"Objects":null,"Tasks":[{"Type":"None","Name":"something-task","Fields":null,"Objects":null,"Annotations":null}],"Updates":{"create":1}}]},"NextPeriodicLaunch":null,"Warnings":"","Index":35994111}

Actual Result

With Python:

json: cannot unmarshal string into Go value of type api.JobRegisterRequest

or

json: cannot unmarshal string into Go value of type api.JobPlanRequest

Job file (if appropriate)

{
    "Job": {
        "Region": "europe",
        "Namespace": "namespace_name",
        "ID": "job_name",
        "Name": "job_name",
        "Type": "service",
        "Priority": null,
        "AllAtOnce": null,
        "Datacenters": [
            "eu-west-2"
        ],
        "Constraints": [
            {
                "LTarget": "${attr.kernel.name}",
                "RTarget": "linux",
                "Operand": "="
            },
            {
                "LTarget": "${node.class}",
                "RTarget": "some_class",
                "Operand": "="
            }
        ],
        "Affinities": null,
        "TaskGroups": [
            {
                "Name": "some-group",
                "Count": 1,
                "Constraints": null,
                "Affinities": null,
                "Tasks": [
                    {
                        "Name": "some-task",
                        "Driver": "docker",
                        "User": "",
                        "Lifecycle": null,
                        "Config": {
                            "force_pull": true,
                            "image": "some_registery/some:110c8967b",
                            "labels": [
                                {
                                    "namespace": "blah",
                                    "owner": "blah",
                                    "service": "some"
                                }
                            ],
                            "logging": [
                                {
                                    "config": [
                                        {
                                            "labels": "namespace,service,owner",
                                            "tag": "{{.Name}}/{{.ID}}"
                                        }
                                    ],
                                    "type": "journald"
                                }
                            ],
                            "mount": [
                                {
                                    "readonly": false,
                                    "target": "/tmp",
                                    "tmpfs_options": [
                                        {
                                            "size": 10000000
                                        }
                                    ],
                                    "type": "tmpfs"
                                }
                            ],
                            "ports": [
                                "some-http"
                            ],
                            "readonly_rootfs": false,
                            "volumes": [
                                "/etc/ssl:/etc/ssl"
                            ]
                        },
                        "Constraints": null,
                        "Affinities": null,
                        "Env": {
                            "SPRING_PROFILES_ACTIVE": "sandbox",
                            "VAULT_ADDR": "https://some_vault:8200"
                        },
                        "Services": [
                            {
                                "Name": "some-name-sandbox",
                                "Tags": [
                                    "service=some-name-sandbox",
                                    "traefik.enable=true",
                                    "traefik.http.routers.some.entrypoints=https",
                                    "traefik.http.routers.some.tls=true",
                                    "traefik.http.routers.some.rule=Host(`some.name.some_domain`)"
                                ],
                                "CanaryTags": [
                                    "traefik.consulcatalog.canary=true",
                                    "service=some-name-sandbox",
                                    "traefik.enable=true",
                                    "traefik.http.routers.some-canary.entrypoints=https",
                                    "traefik.http.routers.some-canary.tls=true",
                                    "traefik.http.routers.some-canary.rule=Host(`canary`)"
                                ],
                                "EnableTagOverride": false,
                                "PortLabel": "some-http",
                                "AddressMode": "",
                                "Address": "",
                                "Checks": [
                                    {
                                        "Name": "",
                                        "Type": "http",
                                        "Command": "",
                                        "Args": null,
                                        "Path": "/health",
                                        "Protocol": "http",
                                        "PortLabel": "some-http",
                                        "Expose": false,
                                        "AddressMode": "",
                                        "Advertise": "",
                                        "Interval": 15000000000,
                                        "Timeout": 5000000000,
                                        "InitialStatus": "",
                                        "TLSSkipVerify": false,
                                        "Header": null,
                                        "Method": "",
                                        "CheckRestart": null,
                                        "GRPCService": "",
                                        "GRPCUseTLS": false,
                                        "TaskName": "",
                                        "SuccessBeforePassing": 0,
                                        "FailuresBeforeCritical": 0,
                                        "Body": "",
                                        "OnUpdate": ""
                                    }
                                ],
                                "CheckRestart": null,
                                "Connect": null,
                                "Meta": null,
                                "CanaryMeta": null,
                                "TaggedAddresses": null,
                                "TaskName": "",
                                "OnUpdate": "",
                                "Provider": ""
                            }
                        ],
                        "Resources": {
                            "CPU": 100,
                            "Cores": null,
                            "MemoryMB": 512,
                            "MemoryMaxMB": null,
                            "DiskMB": null,
                            "Networks": null,
                            "Devices": null,
                            "IOPS": null
                        },
                        "RestartPolicy": null,
                        "Meta": null,
                        "KillTimeout": null,
                        "LogConfig": null,
                        "Artifacts": null,
                        "Vault": {
                            "Policies": [
                                "policy1",
                                "policy2"
                            ],
                            "Namespace": null,
                            "Env": true,
                            "ChangeMode": "signal",
                            "ChangeSignal": "SIGUSR1"
                        },
                        "Templates": [
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/appConfig.env",
                                "EmbeddedTmpl": "template_data\n",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": true,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/client.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/client.key",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/server.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/server.key",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/ca_dir/vault-ca.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/ca_dir/AmazonRootCA1.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/ca_dir/AmazonRootCA2.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/ca_dir/AmazonRootCA3.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/ca_dir/AmazonRootCA4.crt",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            },
                            {
                                "SourcePath": null,
                                "DestPath": "${NOMAD_SECRETS_DIR}/something.key",
                                "EmbeddedTmpl": "template_data",
                                "ChangeMode": "restart",
                                "ChangeScript": null,
                                "ChangeSignal": null,
                                "Splay": 5000000000,
                                "Perms": "0644",
                                "Uid": null,
                                "Gid": null,
                                "LeftDelim": null,
                                "RightDelim": null,
                                "Envvars": null,
                                "VaultGrace": null,
                                "Wait": null,
                                "ErrMissingKey": false
                            }
                        ],
                        "DispatchPayload": null,
                        "VolumeMounts": null,
                        "Leader": false,
                        "ShutdownDelay": 0,
                        "KillSignal": "",
                        "Kind": "",
                        "ScalingPolicies": null,
                        "Identity": null
                    }
                ],
                "Spreads": null,
                "Volumes": null,
                "RestartPolicy": {
                    "Interval": 1800000000000,
                    "Attempts": 2,
                    "Delay": 15000000000,
                    "Mode": "fail"
                },
                "ReschedulePolicy": {
                    "Attempts": null,
                    "Interval": null,
                    "Delay": 30000000000,
                    "DelayFunction": "exponential",
                    "MaxDelay": 120000000000,
                    "Unlimited": true
                },
                "EphemeralDisk": null,
                "Update": null,
                "Migrate": null,
                "Networks": [
                    {
                        "Mode": "",
                        "Device": "",
                        "CIDR": "",
                        "IP": "",
                        "DNS": null,
                        "ReservedPorts": null,
                        "DynamicPorts": [
                            {
                                "Label": "some-http",
                                "Value": 0,
                                "To": 9004,
                                "HostNetwork": ""
                            }
                        ],
                        "Hostname": "",
                        "MBits": null
                    }
                ],
                "Meta": null,
                "Services": null,
                "ShutdownDelay": null,
                "StopAfterClientDisconnect": null,
                "MaxClientDisconnect": null,
                "Scaling": null,
                "Consul": null
            }
        ],
        "Update": {
            "Stagger": null,
            "MaxParallel": 1,
            "HealthCheck": "checks",
            "MinHealthyTime": 10000000000,
            "HealthyDeadline": 900000000000,
            "ProgressDeadline": 1200000000000,
            "Canary": 1,
            "AutoRevert": true,
            "AutoPromote": true
        },
        "Multiregion": null,
        "Spreads": [
            {
                "Attribute": "${attr.platform.aws.placement.availability-zone}",
                "Weight": null,
                "SpreadTarget": null
            }
        ],
        "Periodic": null,
        "ParameterizedJob": null,
        "Reschedule": null,
        "Migrate": {
            "MaxParallel": 1,
            "HealthCheck": "checks",
            "MinHealthyTime": 10000000000,
            "HealthyDeadline": 900000000000
        },
        "Meta": {
            "run_uuid": "b496a872-6c7a-4d44-b5ae-c5c9ab0a7eea"
        },
        "ConsulToken": null,
        "VaultToken": "token",
        "Stop": null,
        "ParentID": null,
        "Dispatched": false,
        "DispatchIdempotencyToken": null,
        "Payload": null,
        "ConsulNamespace": null,
        "VaultNamespace": null,
        "NomadTokenID": null,
        "Status": null,
        "StatusDescription": null,
        "Stable": null,
        "Version": null,
        "SubmitTime": null,
        "CreateIndex": null,
        "ModifyIndex": null,
        "JobModifyIndex": null
    },
    "Diff": true,
    "PolicyOverride": false
}

Nomad Server logs (if appropriate)

N/A

Nomad Client logs (if appropriate)

N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug
Projects
Nomad - Community Issues Triage
Status: Needs Triage
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sjiveson