(ZWEAS123E) Invalid token type in response from Authentication service

[dkelosky]

Describe the bug
I'm installing a new instance of Zowe V3. When I try to log into API Catalog I get:

Steps to Reproduce

My zowe.yaml is:

################################################################################
# This program and the accompanying materials are made available under the terms of the
# Eclipse Public License v2.0 which accompanies this distribution, and is available at
# https://www.eclipse.org/legal/epl-v20.html
#
# SPDX-License-Identifier: EPL-2.0
#
# Copyright Contributors to the Zowe Project.
################################################################################

#===============================================================================
# This is a YAML configuration file for a Zowe instance.
#
# YAML is a human-friendly data serialization language for all programming languages.
# To learn more about YAML specifications, please check https://yaml.org/.
#
# To learn more details about the content of this file, please check https://docs.zowe.org/.
#
# For first time users, or for the most common use cases, please pay more
# attention to the entries marked with "**COMMONLY_CUSTOMIZED**". A "directory" refers
# to a directory in z/OS Unix.
#
# If you modify any settings listed in "zwe init --help" command, you may need to
# re-run the "zwe init" command to make them take effect.
#===============================================================================

#-------------------------------------------------------------------------------
# Zowe global configurations
#
# This section includes Zowe setup information used by `zwe install` and
# `zwe init` command, as well as default configurations for Zowe runtime.
#-------------------------------------------------------------------------------
zowe:

  #-------------------------------------------------------------------------------
  # These configurations are used by "zwe install" or "zwe init" commands.
  #-------------------------------------------------------------------------------
  setup:
    # MVS data set related configurations
    dataset:
      # **COMMONLY_CUSTOMIZED**
      # where Zowe MVS data sets will be installed
      prefix: DKELOSKY.PSWI.AZWE003.T
      # **COMMONLY_CUSTOMIZED**
      # PROCLIB where Zowe STCs will be copied over
      proclib: product.PROCLIB
      # **COMMONLY_CUSTOMIZED**
      # Zowe PARMLIB
      parmlib: dkelosky.PSWI.AZWE003.T.CUST.PARMLIB
      # Holds Zowe PARMLIB members for plugins
      parmlibMembers:
        # For ZIS plugins
        zis: ZWESIP00
      # **COMMONLY_CUSTOMIZED**
      # JCL library where Zowe will store temporary JCLs during initialization
      jcllib: dkelosky.PSWI.AZWE003.T.CUST.JCLLIB
      # Utilities for use by Zowe and extensions
      loadlib: dkelosky.PSWI.AZWE003.T.SZWELOAD
      # APF authorized LOADLIB for Zowe
      authLoadlib: dkelosky.PSWI.AZWE003.T.SZWEAUTH
      # **COMMONLY_CUSTOMIZED**
      # APF authorized LOADLIB for Zowe ZIS Plugins
      authPluginLib: dkelosky.PSWI.AZWE003.T.CUST.ZWESAPL

    # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    # Security related configurations. This setup is optional.
    security:
      # security product name. Can be RACF, ACF2 or TSS
      product: RACF
      # security group name
      groups:
        # Zowe admin user group
        admin: ZWEADMIN
        # Zowe STC group
        stc: ZWEADMIN
        # Zowe SysProg group
        sysProg: ZWEADMIN
      # security user name
      users:
        # Zowe runtime user name of main service
        zowe: ZWESVUSR
        # Zowe runtime user name of ZIS
        zis: ZWESIUSR
      # STC names
      stcs:
        # STC name of Zowe main service
        zowe: ZWESLSTC
        # STC name of Zowe ZIS
        zis: ZWESISTC
        # STC name of Zowe ZIS Auxiliary Server
        aux: ZWESASTC

    # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    # Certificate related configurations
    #
    # There are 5 configurations cases. Please choose one from below.

    # >>>> Certificate setup scenario 1
    # PKCS12 (keystore) with Zowe generate certificates.
    # certificate:
      # Type of certificate storage. Valid values are: PKCS12, JCEKS, JCECCAKS, JCERACFKS, JCECCARACFKS, or JCEHYBRIDRACFKS
      # type:
      # pkcs12:
        # **COMMONLY_CUSTOMIZED**
        # Keystore directory
        # directory:
        # # Lock the keystore directory to only accessible by Zowe runtime user and group.
        # lock: true
        # **COMMONLY_CUSTOMIZED**
        # # Certificate alias name. Optional, default value is localhost.
        # # Note: please use all lower cases as alias.
        # name: localhost
        # **COMMONLY_CUSTOMIZED**
        # # Keystore password. Optional, default value is password.
        # password: password
        # **COMMONLY_CUSTOMIZED**
        # # Alias name of self-signed certificate authority. Optional, default value is local_ca.
        # # Note: please use all lower cases as alias.
        # caAlias: local_ca
        # **COMMONLY_CUSTOMIZED**
        # # Password of keystore stored self-signed certificate authority. Optional, default value is local_ca_password.
        # caPassword: local_ca_password
      # # Distinguished name for Zowe generated certificates. All optional.
      # dname:
      #   caCommonName: ""
      #   commonName: ""
      #   orgUnit: ""
      #   org: ""
      #   locality: ""
      #   state: ""
      #   country: ""
      # # Validity days for Zowe generated certificates
      # validity: 3650
      # # Domain names and IPs should be added into certificate SAN
      # # If this field is not defined, `zwe init` command will use
      # # `zowe.externalDomains`.
      # san:
      #   # sample domain name
      #   - dvipa.my-company.com
      #   # sample IP address
      #   - 12.34.56.78

    # # >>>> Certificate setup scenario 2
    # # PKCS12 (keystore) with importing certificate generated by other CA.
    # certificate:
    #   # Type of certificate storage. Valid values are: PKCS12, JCERACFKS. APIML additionally supports: JCEKS, JCECCAKS, JCECCARACFKS, or JCEHYBRIDRACFKS
    #   type: PKCS12
    #   pkcs12:
    #     # **COMMONLY_CUSTOMIZED**
    #     # Keystore directory
    #     directory: /var/zowe/keystore
    #     # Lock the keystore directory to only accessible by Zowe runtime user and group.
    #     lock: true
    #     # # Certificate alias name. Optional, default value is localhost.
    #     # # Note: please use all lower cases as alias.
    #     # name: localhost
    #     # # Keystore password. Optional, default value is password.
    #     # password: password
    #     import:
    #       # **COMMONLY_CUSTOMIZED**
    #       # Existing PKCS12 keystore which holds the certificate issued by external CA.
    #       keystore: ""
    #       # **COMMONLY_CUSTOMIZED**
    #       # Password of the above keystore
    #       password: ""
    #       # **COMMONLY_CUSTOMIZED**
    #       # Certificate alias will be imported
    #       # Note: please use all lower cases as alias.
    #       alias: ""
    #   # **COMMONLY_CUSTOMIZED**
    #   # PEM format certificate authorities will also be imported and trusted.
    #   importCertificateAuthorities:
    #     # Path to the certificate authority signed the certificate will be imported.
    #     - ""

    # # >>>> Certificate setup scenario 3
    # # Zowe generated z/OS Keyring with Zowe generated certificates.
    # certificate:
    #   # Type of certificate storage. Valid values are: PKCS12, JCERACFKS. APIML additionally supports: JCEKS, JCECCAKS, JCECCARACFKS, or JCEHYBRIDRACFKS
    #   type: JCERACFKS
    #   keyring:
    #     # **COMMONLY_CUSTOMIZED**
    #     # keyring name
    #     name: ZoweKeyring
    #     # **COMMONLY_CUSTOMIZED**
    #     # # Label of Zowe certificate. Optional, default value is localhost.
    #     # label: localhost
    #     # **COMMONLY_CUSTOMIZED**
    #     # # label of Zowe CA certificate. Optional, default value is localca.
    #     # caLabel: localca
    #   # # Distinguished name for Zowe generated certificates. All optional.
    #   # dname:
    #   #   caCommonName: ""
    #   #   commonName: ""
    #   #   orgUnit: ""
    #   #   org: ""
    #   #   locality: ""
    #   #   state: ""
    #   #   country: ""
    #   # # Validity days for Zowe generated certificates
    #   # validity: 3650
    #   # # Domain names and IPs should be added into certificate SAN
    #   # # If this field is not defined, `zwe init` command will use
    #   # # `zowe.externalDomains`.
    #   # # **NOTE**: due to the limitation of RACDCERT command, this field should
    #   # #           contain exactly 2 entries with the domain name and IP address.
    #   # san:
    #   #   - dvipa.my-company.com
    #   #   - 12.34.56.78

    # >>>> Certificate setup scenario 4
    # Zowe generated z/OS Keyring and connect to existing certificate
    certificate:
      # Type of certificate storage. Valid values are: PKCS12, JCERACFKS. APIML additionally supports: JCEKS, JCECCAKS, JCECCARACFKS, or JCEHYBRIDRACFKS
      type: JCERACFKS
      keyring:
        # **COMMONLY_CUSTOMIZED**
        # keyring name
        name: ZoweKeyring
        connect:
          # **COMMONLY_CUSTOMIZED**
          # Current owner of the existing certificate, can be SITE or an user ID.
          user: SITE
          # **COMMONLY_CUSTOMIZED**
          # Label of the existing certificate will be connected to Zowe keyring.
          label: "ROOTSTAR"
      # **COMMONLY_CUSTOMIZED**
      # If you have other certificate authorities want to be trusted in Zowe keyring,
      # list the certificate labels here.
      # **NOTE**, due to the limitation of RACDCERT command, this field should
      #           contain maximum 2 entries.
      importCertificateAuthorities:
        - "DigiCert CA"
        - "DigiCert Global Root CA"

    # # >>>> Certificate setup scenario 5
    # # Zowe generated z/OS Keyring with importing certificate stored in data set
    # certificate:
    #   # Type of certificate storage. Valid values are: PKCS12, JCERACFKS. APIML additionally supports: JCEKS, JCECCAKS, JCECCARACFKS, or JCEHYBRIDRACFKS
    #   type: JCERACFKS
    #   keyring:
    #     # **COMMONLY_CUSTOMIZED**
    #     # keyring name
    #     name: ZoweKeyring
    #     # **COMMONLY_CUSTOMIZED**
    #     # # Label of Zowe certificate. Optional, default value is localhost.
    #     # label: localhost
    #     import:
    #       # **COMMONLY_CUSTOMIZED**
    #       # Name of the data set holds the certificate issued by other CA.
    #       # This data set should be in PKCS12 format and contain private key.
    #       dsName: ""
    #       # **COMMONLY_CUSTOMIZED**
    #       # Password for the PKCS12 data set.
    #       password: ""

    # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    # VSAM configurations if you are using VSAM as Caching Service storage
    vsam:
      # VSAM data set with Record-Level-Sharing enabled or not
      # Valid values could be: NONRLS or RLS.
      mode: NONRLS
      # Volume name if you are using VSAM in NONRLS mode
      volume: "USER01"
      # Storage class name if you are using VSAM in RLS mode
      storageClass: ""

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # **COMMONLY_CUSTOMIZED**
  # Zowe runtime (root) directory
  #
  # **NOTE**:
  # 1. if it is not specified and you passed "--update-config" argument
  # when you ran "zwe init" command, this value will be updated with the Zowe
  # runtime where the "zwe" command is located.
  #
  # 2. the runtime directory is designed to be "read only". This is different from the
  # log, workspace, and extension directories. Setting those three to the same parent folder
  # as the runtime directory is unsupported & may cause issues
  #
  # This value is required by ZWESLSTC to know where is Zowe runtime.
  runtimeDirectory: "/u/users/dkelosky/zowe-smpe"

  # **COMMONLY_CUSTOMIZED**
  # Where to store runtime logs
  logDirectory: /u/users/dkelosky/zowe-smpe/logs

  # **COMMONLY_CUSTOMIZED**
  # Zowe runtime workspace directory
  workspaceDirectory: /u/users/dkelosky/zowe-smpe/workspace

  # **COMMONLY_CUSTOMIZED**
  # Where extensions are installed
  extensionDirectory: /u/users/dkelosky/zowe-smpe/extensions

  configmgr:
    # STRICT=quit on any error, including missing schema
    # COMPONENT-COMPAT=if component missing schema, skip it with warning instead of quit
    validation: "STRICT"

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # runtime z/OS job name
  job:
    # Zowe JES job name
    name: ZWE1SV
    # Prefix of component address space
    prefix: ZWE1

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # This is an ID you use to separate multiple Zowe installs when determining
  # resource names used in RBAC authorization checks such as dataservices with RBAC
  # expects this ID in SAF resources
  rbacProfileIdentifier: "1"
  # rbacProfileIdentifier: "profile1"

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # This is an ID that can be used by servers that distinguish their cookies from unrelated Zowe installs,
  # for purposes such as to allow multiple copies of Zowe to be used within the same client
  cookieIdentifier: "1"
  # cookieIdentifier: "cookie1"

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # **COMMONLY_CUSTOMIZED**
  # You can list your external domains on how you want to access Zowe.
  # This should be the domain list you would like to put into your web browser's
  # address bar.
  externalDomains:
    # this should be the domain name to access Zowe APIML Gateway
    - <removed>

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # This is the port you use to access Zowe Gateway from your web browser.
  #
  # In many use cases, this should be same as `components.gateway.port`. But in
  # some use cases, like containerization, this port could be different.
  externalPort: 7554

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  #
  # In this section, behavior such as which TLS levels, ciphers should be used, and if native TLS versus AT-TLS should be used.
  # See the schema for options.
  #
  # This section applies to all components that support it.
  # So far: "zss" and "app-server"
  #
  # This section can be overridden per-component by placing it
  # Under a "zowe" subsection of a component, as in "components.zss.zowe.network"
  network:
    server:
      tls:
        attls: false
        # TLS settings only apply when attls=false
        # Else you must use AT-TLS configuration for TLS customization.
        minTls: "TLSv1.2"
        maxTls: "TLSv1.3"
    client:
      tls:
        attls: false

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # You can un-comment and define any extra environment variables as key/value
  # pairs here.
  # environments:
  #   # Example of a global environment variable for all components
  #   MY_ENV_VAR: my_env_val

  #   # Another example to customize SSH port for VT Terminal Desktop app
  #   ZWED_SSH_PORT: 22
  #   ZWED_TN3270_PORT: 23

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # You can define any Zowe message portions to be checked for and the message added to the
  # system log upon its logging, truncated to 126 characters.
  sysMessages:
  #   # Zowe starting
    - "ZWEL0021I"
  #   # Zowe started
    - "ZWEL0018I"
    - "ZWEL0006I"
  #   # Zowe ready to use
    - "ZWES1601I"
  #   # Zowe stopping
    - "ZWEL0008I"
  #   # Zowe stopped
    - "ZWEL0022I"
  #   # Zowe components starting
    - "ZWEL0001I"
  #   # Zowe components stopped
    - "ZWEL0002I"
  #   # API ML components ready
    - "ZWEAM000I"
  #   # App server ready
    - "ZWED0031I"
  #   # ZSS ready
    - "ZWES1013I"

  #   # Not limited to Zowe message ID's, you can specify your own string for example:
  #   - "ERROR"

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # Enable debug mode for Zowe launch scripts
  launchScript:
    # Set to "debug" or "trace" to display extra debug information
    logLevel: "info"
    # Set to "exit" if you'd like startup to exit if any component has an error in the configure stage, otherwise zwe will warn but continue.
    onComponentConfigureFail: "warn"

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # Default Zowe certificate
  #
  # **NOTE**: these fields can be updated automatically if you pass
  # `--update-config` to `zwe init` command. The generated value will base on
  # your setup in `zowe.setup.certificate` section.
  certificate:
    keystore:
      type: JCERACFKS
      # if keyrings, the format is safkeyring:////stcusername/KeyName
      file: safkeyring://ZWESVUSR/ZoweKeyring
      # "password" should either be the value you set when PKCS12, or literally "password" for keyrings.
      password: password
      # alias is the name of your key/cert. When using keyrings, get the Case Sensitive, Space Sensitive value in a TSO list ring.
      alias: ROOTSTAR
    truststore:
      # truststore usually has same values as keystore (minus alias), but can be different if desired.
      type: JCERACFKS
      file: safkeyring://ZWESVUSR/ZoweKeyring
      password: password
    pem:
      key: ""
      certificate: ""
      #  if keyrings, the format is "safkeyring:////stcusername/KeyName&ca name"
      certificateAuthorities: safkeyring://ZWESVUSR/ZoweKeyring&DigiCert CA,safkeyring://ZWESVUSR/ZoweKeyring&DigiCert Global Root CA

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # How we want to verify SSL certificates of services. Valid values are:
  # - STRICT:    will validate if the certificate is trusted in our trust store and
  #              if the certificate Command Name and Subject Alternative Name (SAN)
  #              is validate. This is recommended for the best security.
  # - NONSTRICT: will validate if the certificate is trusted in our trust store.
  #              This mode does not validate certificate Common Name and Subject
  #              Alternative Name (SAN).
  # - DISABLED:  disable certificate validation. This is NOT recommended for
  #              security.
  verifyCertificates: STRICT


#-------------------------------------------------------------------------------
# Java configuration
#
# Some Zowe components requires Java. Define the path where you have your Java
# is installed.
#
# **NOTE**: this field can be updated automatically if you pass `--update-config`
# to `zwe init` command.
#-------------------------------------------------------------------------------
java:
  # **COMMONLY_CUSTOMIZED**
  # Path to your Java home directory
  home: "/usr/lpp/java/J17.0_64"


#-------------------------------------------------------------------------------
# node.js configuration
#
# Some Zowe components requires node.js. Define the path where you have your
# node.js is installed.
#
# **NOTE**: this field can be updated automatically if you pass `--update-config`
# to `zwe init` command.
#-------------------------------------------------------------------------------
node:
  # **COMMONLY_CUSTOMIZED**
  # Path to your node.js home directory
  home: "/usr/lpp/IBM/cnj/v18r0/IBM/node-latest-os390-s390x"


#-------------------------------------------------------------------------------
# z/OSMF configuration
#
# If your Zowe instance is configured to use z/OSMF for authentication or other
# features. You need to define how to access your z/OSMF instance.
#-------------------------------------------------------------------------------
zOSMF:
  # **COMMONLY_CUSTOMIZED**
  # Host name of your z/OSMF instance
  host: <removed>
  # **COMMONLY_CUSTOMIZED**
  port: 1443
  applId: IZUDFLT


#-------------------------------------------------------------------------------
# Zowe components default configurations
#
# This section includes default configurations for all Zowe components installed
# on the Zowe instance.
#
# Every component should define their own section under `components` with their
# component ID.
#
# For each component, they can always have "enabled" property and "certificate"
# property. More configurations for each component can be found in component
# manifest file.
#-------------------------------------------------------------------------------
components:

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  gateway:
    enabled: true
    port: 7554
    debug: false
    apiml:
      security:
        auth:
          provider: zosmf
          zosmf:
            jwtAutoconfiguration: jwt
            serviceId: ibmzosmf
        authorization:
          endpoint:
            enabled: true
          provider: "native"
        x509:
          enabled: true

    # If we customize this to use different external certificate, than should also
    # define "server.internal.ssl.certificate" and enable "server.internal.ssl.enabled".
    # certificate:
    #   keystore:
    #     alias: ""

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  zaas:
    enabled: true
    port: 7558
    debug: false
  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  api-catalog:
    enabled: true
    port: 7552
    debug: false

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  discovery:
    enabled: true
    port: 7553
    debug: false
    # Define this value to match your number of Discovery StatefulSet if you are running containerized Zowe
    # replicas: 1

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  caching-service:
    enabled: true
    port: 7555
    debug: false

    storage:
      evictionStrategy: removeOldest
      # can be infinispan, inMemory, redis or VSAM(deprecated)
      mode: VSAM
      vsam:
        # your VSAM data set created by "zwe init vsam" command or ZWECSVSM JCL
        # this is required if storage mode is VSAM
        name: "DKELOSKY.ZOWE.NOT.NEEDED"
      infinispan:
        # this is required if storage mode is infinispan
        jgroups:
          port: 7600
          host:
          keyExchange:
            port: 7601

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  app-server:
    enabled: false
    port: 7556
    debug: false
    # we can customize any component with custom certificate
    # the missing definitions will be picked from "zowe.certificate"
    # certificate:
    #   keystore:
    #     alias: app-server
    #   pem:
    #     key: /global/zowe/keystore/localhost/localhost.keystore.app-server.key
    #     certificate: /global/zowe/keystore/localhost/localhost.keystore.app-server.cer-ebcdic

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  zss:
    enabled: false
    port: 7557
    crossMemoryServerName: ZWESIS_STD
    agent:
      jwt:
        fallback: true
      64bit: true

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  explorer-jes:
    enabled: false

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  explorer-mvs:
    enabled: false

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  explorer-uss:
    enabled: false

  # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  # Each extension can have dedicated definition similar to core components.
  # my-extension:
  #   enabled: true


#-------------------------------------------------------------------------------
# Zowe high availability instances customizations
#
# This section includes customizations for each Zowe high availability instance.
#
# You can start each HA instance with `zwe start --ha-instance <ha-instance>`.
#-------------------------------------------------------------------------------
# haInstances:
#
#   # HA instance ID
#   lpar1:
#     # hostname where this instance will be started
#     hostname: lpar1.my-company.com
#     # Your &SYSNAME for this LPAR
#     # This sysname will be used to route your JES command to target system.
#     sysname: LPR1
#     # for this HA instance, we did not customize "components", so it will use default value.

#   # HA instance ID, we will start 2 instances on LPAR2
#   # **NOTE**, we can only start one gateway in same LPAR.
#   lpar2a:
#     # hostname where this instance will be started
#     hostname: lpar2.my-company.com
#     # Your &SYSNAME for this LPAR
#     # This sysname will be used to route your JES command to target system.
#     sysname: LPR2

#     # These configurations will overwrite highest level default "components" configuration
#     components:
#       discovery:
#         # use customized port on this instance
#         port: 17553
#       api-catalog:
#         port: 17552
#       app-server:
#         # no app-server in this instance
#         enabled: false
#       zss:
#         # no app-server in this instance
#         enabled: false
#       caching-service:
#         port: 17555
#
#   lpar2b:
#     hostname: lpar2.my-company.com
#     # your &SYSNAME for this LPAR
#     sysname: LPR2

#     # These configurations will overwrite highest level default "components" configuration
#     components:
#       zaas:
#         enabled: false
#       gateway:
#         enabled: false
#       discovery:
#         enabled: false
#       api-catalog:
#         enabled: false
#       app-server:
#         enabled: false
#         port: 28544
#       zss:
#         port: 28542
#       caching-service:
#         enabled: false

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Logs
If applicable, add server logs collected at the time of your problem.

2024-11-13 18:26:56.742 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0021I Zowe Launcher starting
2024-11-13 18:26:56.744 <ZWELNCH:67108934> ZWESVUSR INFO -- If you cant see '500' at the end of the line, your log is too short to read!80--------90------ 100----------------------125----------------------150----------------------175----------------------200----------------------225----------------------250----------------------275----------------------300----------------------325----------------------350----------------------375----------------------400----------------------425----------------------450----------------------475----------------------500
2024-11-13 18:26:56.787 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0023I Zowe YAML config file is 'FILE(/u/users/dkelosky/zowe-smpe/zowe.yaml)'
2024-11-13 18:26:56.787 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0024I HA_INSTANCE_ID is '__ha_instance_id__'
2024-11-13 18:26:56.799 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0017I ROOT_DIR is '/u/users/dkelosky/zowe-smpe'
2024-11-13 18:26:56.806 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0069I Configuration is valid
2024-11-13 18:26:56.806 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0058I WORKSPACE_DIR is '/u/users/dkelosky/zowe-smpe/workspace'
bos extend currSize=0x0 dataSize=0x10dd chunk=0x1000 extend=0x10dd
2024-11-13 18:26:57 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare) Zowe version: v3.0.0
2024-11-13 18:26:57 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare) build and hash: v3.x/master#6219 (c39ba1eb36ba7b8f3ad334a72f2e4359a5798931)
2024-11-13 18:26:57 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare) starting Zowe instance <removed> with /u/users/dkelosky/zowe-smpe/workspace/.env/.zowe-merged.yaml ...
2024-11-13 18:26:57 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare,global_validate) process global validations ...
2024-11-13 18:26:58 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare,global_validate) global validations are successful
2024-11-13 18:26:58 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare,validate_components) process component validations ...
2024-11-13 18:27:01 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare,configure_components) process component configurations ...
2024-11-13 18:27:01 <ZWELS:16777306> ZWESVUSR INFO (zwe-internal-start-prepare) Zowe runtime environment prepared
2024-11-13 18:27:01.588 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0018I Zowe instance prepared successfully
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0016I start component list: 'gateway,zaas,api-catalog,discovery,caching-service'
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0003I new component initialized gateway, restart_cnt=10, min_uptime=90 seconds, share_as=yes
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0025I restart_intervals for component 'gateway'= 1 1 1 5 5 10 20 60 120 240
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0003I new component initialized zaas, restart_cnt=10, min_uptime=90 seconds, share_as=yes
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0025I restart_intervals for component 'zaas'= 1 1 1 5 5 10 20 60 120 240
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0003I new component initialized api-catalog, restart_cnt=10, min_uptime=90 seconds, share_as=yes
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0025I restart_intervals for component 'api-catalog'= 1 1 1 5 5 10 20 60 120 240
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0003I new component initialized discovery, restart_cnt=10, min_uptime=90 seconds, share_as=yes
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0025I restart_intervals for component 'discovery'= 1 1 1 5 5 10 20 60 120 240
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0003I new component initialized caching-service, restart_cnt=10, min_uptime=90 seconds, share_as=yes
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0025I restart_intervals for component 'caching-service'= 1 1 1 5 5 10 20 60 120 240
2024-11-13 18:27:01.589 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0006I starting components
2024-11-13 18:27:01.592 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0001I component gateway started
2024-11-13 18:27:01.642 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0001I component zaas started
2024-11-13 18:27:01.667 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0001I component api-catalog started
2024-11-13 18:27:01.714 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0001I component discovery started
2024-11-13 18:27:01.749 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0001I component caching-service started
2024-11-13 18:27:01.752 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0007I components started
2024-11-13 18:27:01.765 <ZWELNCH:67108934> ZWESVUSR INFO ZWEL0012I starting console listener
jar file: /u/users/dkelosky/zowe-smpe/components/gateway/bin/gateway-service.jar
Setting loader path: /u/users/dkelosky/zowe-smpe/workspace/gateway/sharedLibs/
jar file: /u/users/dkelosky/zowe-smpe/components/zaas/bin/zaas-service-lite.jar
Setting loader path: /u/users/dkelosky/zowe-smpe/workspace/zaas/sharedLibs/,../apiml-common-lib/bin/api-layer-lite-lib-all.jar,/usr/include/java_classes/IRRRacf.jar
jar file: /u/users/dkelosky/zowe-smpe/components/api-catalog/bin/api-catalog-services-lite.jar
jar file: /u/users/dkelosky/zowe-smpe/components/discovery/bin/discovery-service-lite.jar
jar file: /u/users/dkelosky/zowe-smpe/components/caching-service/bin/caching-service.jar
pid=50331784
pid=50331783
pid=16777406
pid=83886207
pid=16777389
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:12.424 <ZWEAGW1:background-preinit:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.h.v.i.u.Version)) HV000001: Hibernate Validator 8.0.1.Final
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:21.599 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.c.c.s.GenericScope)) BeanFactory id=6b79a015-1fdc-34df-b32e-4e3fc4fa194f
2024-11-13 18:27:22.629 <ZWEAZS1:main: > �[35mZWESVUSR�[0;39m �[36mERROR�[0;39m ((o.z.a.p.v.BuildInfo)) ZWEAM100E Could not read properties from: 'META-INF/git.properties'
2024-11-13 18:27:24.361 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat initialized with port 7554 (https)
2024-11-13 18:27:24.403 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.a.c.h.Http11NioProtocol)) Initializing ProtocolHandler ["https-jsse-nio-0.0.0.0-7554"]
2024-11-13 18:27:24.438 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.a.c.c.StandardService)) Starting service [Tomcat]
2024-11-13 18:27:24.441 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.a.c.c.StandardEngine)) Starting Servlet engine: [Apache Tomcat/10.1.30]
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:28.558 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.s.b.w.e.t.TomcatWebServer)) Tomcat initialized with port 7555 (https)
2024-11-13 18:27:30.574 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.c.n.e.InstanceInfoFactory)) Setting initial instance status as: STARTING
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:30.368 �[33m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mWARN �[0;39m �[33m�[0;39m((o.z.a.c.s.v.VsamStorage)) ZWECS155W 'vsam' storage option is deprecated.
2024-11-13 18:27:33.906 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mERROR�[0;39m ((o.z.a.p.v.BuildInfo)) ZWEAM100E Could not read properties from: 'META-INF/git.properties'
2024-11-13 18:27:40.657 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.a.e.w.EndpointLinksResolver)) Exposing 3 endpoints beneath base path '/application'
2024-11-13 18:27:42.502 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.c.n.e.c.DiscoveryClientOptionalArgsConfiguration)) Eureka HTTP Client uses Jersey
2024-11-13 18:27:42.024 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.s.b.w.e.t.TomcatWebServer)) Tomcat started on port 7555 (https) with context path '/cachingservice'
2024-11-13 18:27:42.559 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.z.a.s.HttpsFactory)) Original truststore keyring URL from configuration: safkeyringjce://ZWESVUSR/ZoweKeyring
2024-11-13 18:27:42.559 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.z.a.s.HttpsFactory)) Loading trusted certificates from keyring: safkeyringjce://ZWESVUSR/ZoweKeyring
2024-11-13 18:27:42.590 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.z.a.s.HttpsFactory)) Original keyring URL from configuration: safkeyringjce://ZWESVUSR/ZoweKeyring
2024-11-13 18:27:42.590 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.z.a.s.HttpsFactory)) Loading keyring from updated URL: safkeyringjce://ZWESVUSR/ZoweKeyring
2024-11-13 18:27:42.985 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.c.n.e.s.EurekaServiceRegistry)) Registering application GATEWAY with eureka with status UP
2024-11-13 18:27:43.136 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.a.c.h.Http11NioProtocol)) Starting ProtocolHandler ["https-jsse-nio-0.0.0.0-7554"]
2024-11-13 18:27:43.188 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.a.t.u.n.N.certificate)) Connector [https-jsse-nio-0.0.0.0-7554], TLS virtual host [_default_], certificate type [UNDEFINED] configured from keystore [/tmp/.keystore] using alias [ROOTSTAR] with trust store [null]
2024-11-13 18:27:43.377 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat started on port 7554 (https)
2024-11-13 18:27:43.379 <ZWEAGW1:main:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.c.n.e.s.EurekaAutoServiceRegistration)) Updating port to 7554
2024-11-13 18:27:44.768 <ZWEAZS1:main:50331783> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat initialized with port 7558 (https)
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:46.658 <ZWEADS1:main:83886207> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat started on port 7553 (https) with context path '/'
2024-11-13 18:27:48.535 <ZWEADS1:main:83886207> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.z.a.p.s.ServiceStartupEventHandler)) ZWEAM000I Discovery Service started in 42.502 seconds
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2024-11-13 18:27:49.487 �[32m�[0;39m<ZWEACS1:main:16777389> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m �[32m�[0;39m((o.z.a.p.s.ServiceStartupEventHandler)) ZWEAM000I Caching Service started in 39.057 seconds
2024-11-13 18:27:55.151 <ZWEAZS1:main:50331783> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat started on port 7558 (https) with context path '/'
2024-11-13 18:27:56.393 <ZWEAAC1:main:16777406> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.s.b.w.e.t.TomcatWebServer)) Tomcat started on port 7552 (https) with context path '/apicatalog'
2024-11-13 18:27:56.534 <ZWEAAC1:main:16777406> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.z.a.p.s.ServiceStartupEventHandler)) ZWEAM000I API Catalog Service started in 46.121 seconds
2024-11-13 18:28:25.198 <ZWEAZS1:Timer-1:50331783> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.z.a.p.s.ServiceStartupEventHandler)) ZWEAM000I ZAAS started in 74.095 seconds
2024-11-13 18:28:26.517 <ZWEAAC1:DiscoveryClient-CacheRefreshExecutor-%d:16777406> �[35mZWESVUSR�[0;39m �[36mWARN �[0;39m ((o.z.a.a.s.c.CachedProductFamilyService)) ZWEAC705W The home page url for service CACHINGSERVICE was not transformed. Not able to select route for url https://removed:7555/cachingservice of the service cachingservice. Original url used.
2024-11-13 18:28:26.518 <ZWEAAC1:DiscoveryClient-CacheRefreshExecutor-%d:16777406> �[35mZWESVUSR�[0;39m �[36mERROR�[0;39m ((o.z.a.a.s.c.CachedProductFamilyService)) ZWEAC708E The API base path for service CACHINGSERVICE was not retrieved. Not able to select API base path for the service cachingservice. Original url used.
2024-11-13 18:28:26.522 <ZWEAAC1:DiscoveryClient-CacheRefreshExecutor-%d:16777406> �[35mZWESVUSR�[0;39m �[36mWARN �[0;39m ((o.z.a.a.s.c.CachedProductFamilyService)) ZWEAC705W The home page url for service IBMZOSMF was not transformed. Not able to select route for url https://removed:1443/ of the service ibmzosmf. Original url used.
2024-11-13 18:28:33.314 <ZWEAGW1:DiscoveryClient-InstanceInfoReplicator-%d:50331784> �[35mZWESVUSR�[0;39m �[36mINFO �[0;39m ((o.z.a.g.c.GatewayHealthIndicator)) ZWEAM001I API Mediation Layer started
2024-11-13 18:29:04.006 <ZWEAGW1:parallel-2:50331784> �[35mZWESVUSR�[0;39m �[36mWARN �[0;39m ((i.n.u.NetUtilInitializations)) Failed to find the loopback interface

Details

• Version and build number: [e.g. 0.4.4-SNAPSHOT build # 155]
• Test environment: [either defined Zowe test environment: Marist (1, 2, 3), River, or your own environment: z/OS version and z/OSMF version]

API Catalog Web UI (in case of API Catalog issue):

• OS: [e.g. macOS, Windows]
• Browser [e.g. Chrome, Safari]
• Version [e.g. 71.0.3578.98]

REST API client (in case of REST API issue):

• Technology: [e.g. Spring Boot, Node.js]
• OS: [e.g. Windows 10]

Additional context
Add any other context about the problem here.

Willingness to help
If you are willing to help with debugging and reproducing the problem or
with fixing the problem, please state your willingness to assist in the created issue.

https://github.com/zowe/api-layer/wiki/Issue-management

[richard-salac]

This could happen when APIML receives unexpected token type from authentication provider. The recommended token type for APIML is JWT (as shown in your zowe yaml), whereas the default one of z/OSMF is ltpa2. Check Zowe docs: https://docs.zowe.org/stable/user-guide/systemrequirements-zos/#zosmf-optional

We should consider improving the message though.