[BUG] PYTHON Generated Client unable to parse response including array: [SCHEMA_OBJECT]

[nmartins]

Description

PYTHON generated client is crashing, unable to parse response containing an array[SCHEMA_OBJECT]

this is the stack trace:
Traceback (most recent call last):
File "/gen-cli-python/start-python-query-example.py", line 47, in
pprint(api_instance.group_query_external_id(external_id))
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 53, in pprint
printer.pprint(object)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 148, in pprint
self._format(object, self._stream, 0, 0, {}, 0)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 170, in _format
rep = self._repr(object, context, level)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 431, in _repr
repr, readable, recursive = self.format(object, context.copy(),
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 444, in format
return _safe_repr(object, context, maxlevels, level, self._sort_dicts)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pprint.py", line 596, in _safe_repr
rep = repr(object)
File "/gen-cli-python/openapi_client/model_utils.py", line 176, in repr
return self.to_str()
File "/gen-cli-python/openapi_client/model_utils.py", line 517, in to_str
return pprint.pformat(self.to_dict())
File "/gen-cli-python/openapi_client/model_utils.py", line 513, in to_dict
return model_to_dict(self, serialize=False)
File "/gen-cli-python/openapi_client/model_utils.py", line 1662, in model_to_dict
res.append(model_to_dict(v, serialize=serialize))
File "/gen-cli-python/openapi_client/model_utils.py", line 1662, in model_to_dict
res.append(model_to_dict(v, serialize=serialize))
File "/gen-cli-python/openapi_client/model_utils.py", line 1634, in model_to_dict
if model_instance._composed_schemas:
AttributeError: 'dict' object has no attribute '_composed_schemas'

openapi-generator version

5.2.1

OpenAPI declaration file content or url

specs_and_example.zip

This is the Json Response example

{
    "adminCount": 0,
    "adminCursor": 0,
    "currentStatusTransitionTime": "2021-10-08T13:39:58.000000Z",
    "deviceCount": 0,
    "externalId": "group123456789",
    "groupMemberCount": 0,
    "groupMemberCursor": 0,
    "lastActivityTime": "2021-10-08T13:39:58.000000Z",
    "maxSubscriberCount": 4294967295,
    "maxUserCount": 4294967295,
    "mtx_container_name": "MtxResponseGroup",
    "mtx_ext_ver": 1,
    "mtx_result_code": 0,
    "mtx_result_text": "OK",
    "mtx_result_type": "get",
    "mtx_sys_ver": 5240,
    "name": "This is the first group",
    "notificationPreference": 1,
    "objectId": "0-1-5-6",
    "relatedUserArray": [
        {
            "externalId": "user123456789",
            "mtx_container_name": "MtxRelatedUserObject",
            "mtx_ext_ver": 1,
            "mtx_sys_ver": 5240,
            "objectId": "0-1-5-1",
            "roleInfoArray": [
                {
                    "externalId": "Admin",
                    "mtx_container_name": "MtxPricingRoleInfo",
                    "mtx_ext_ver": 1,
                    "mtx_sys_ver": 5240,
                    "name": "Admin",
                    "pricingId": 2
                }
            ]
        }
    ],
    "relatedUserCursor": 0,
    "result": 0,
    "resultText": "OK",
    "routeId": 1,
    "status": 1,
    "statusDescription": "Active",
    "subscriberCount": 0,
    "subscriberMemberCount": 0,
    "subscriberMemberCursor": 0,
    "userCount": 1
}

It's failing to parse relatedUserArray into MtxRelatedUserObject

Generation Detail

java -jar openapi-generator-cli-5.2.1.jar generate -i openapi3.json --verbose -g python -o gen-cli-python

Steps to reproduce

1. generate the client using openapi3.json file in attach
2. using command: java -jar openapi-generator-cli-5.2.1.jar generate -i openapi3.json --verbose -g python -o gen-cli-python
3. run python3 gen-cli-python/setup.py install
4. copy start-python-query-example.py in attach into gen-cli-python/
5. run python example in attach, against a server responding the json response example provided above with following command: "python3 start-python-query-example.py", and it'll output the stack trace provided above.

[SamyMe]

Hello, Did you make it work ? I'm running into the same problem.

[flashgorman-chorus]

Experiencing the same thing. Please advise on the status of this bug.

[mickelsonmichael]

More support behind this issue, I'm also experiencing it

[TomTom101]

Same here

[MasterDuke17]

I believe this was fixed in 5.4 with #11234.

[spacether]

@nmartins your spec has an issue

• you define type object adjacent to $ref definition

That is not allowed per https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.3.md#fixed-fields-19
This object (schema containing $ref) cannot be extended with additional properties and any properties added SHALL be ignored.

Using python v6.2.0 your spec just successfully generated a client

Your updated sample is:

#!/usr/bin/python3
#
# $Id: start-python-example.py 82197 2021-04-27 20:45:45Z ed.stitt $
#
# @2to3-3 --no-diffs -x input -x print -w  : Mon 2021-08-30T19:26:49
#
# @futurize --stage2 --no-diffs -n -w  : Fri 2021-02-19T13:28:08
#
# @futurizeManager mark.germain : Thu 2021-03-25T20:24:56
#
import urllib3
import json
from unittest.mock import patch
import typing

import openapi_client
from pprint import pprint
from openapi_client.apis.tags import service_api
from openapi_client.apis.tags import user_api
from openapi_client.apis.tags import subscription_api
from openapi_client.apis.tags import device_api
from openapi_client.apis.tags import group_api
from openapi_client.model.create_user_subscription_and_device import CreateUserSubscriptionAndDevice
from openapi_client.model.mtx_request_user_create import MtxRequestUserCreate
from openapi_client.model.mtx_request_subscription_create import MtxRequestSubscriptionCreate
from openapi_client.model.mtx_request_device_create import MtxRequestDeviceCreate
from openapi_client.model.mtx_billing_cycle_data import MtxBillingCycleData
from openapi_client.model.mtx_role_data import MtxRoleData
from openapi_client.model.mtx_request_group_create import MtxRequestGroupCreate
from openapi_client.model.mtx_request_group_add_user import MtxRequestGroupAddUser
from openapi_client.model.mtx_user_search_data import MtxUserSearchData
from openapi_client.model.mtx_subscription_search_data import MtxSubscriptionSearchData

# Defining the host is optional and defaults to http://localhost:8080/rsgateway/data
# See configuration.py for a list of all supported configuration parameters.
configuration = openapi_client.Configuration(
    host="http://localhost:8080/rsgateway/data"
)


def create_response(
    body: typing.Union[str, bytes],
    status: int = 200,
    content_type: str = 'application/json',
    headers: typing.Optional[typing.Dict[str, str]] = None,
    preload_content: bool = True
) -> urllib3.HTTPResponse:
    if headers is None:
        headers = {}
    headers.update({'content-type': content_type})
    return urllib3.HTTPResponse(
        body,
        headers=headers,
        status=status,
        preload_content=preload_content
    )

def json_bytes(in_data: typing.Any) -> bytes:
    return json.dumps(in_data, separators=(",", ":"), ensure_ascii=False).encode('utf-8')



with patch.object(urllib3.PoolManager, 'request') as mock_request:

    response_json_data = {
        "adminCount": 0,
        "adminCursor": 0,
        "currentStatusTransitionTime": "2021-10-08T13:39:58.000000Z",
        "deviceCount": 0,
        "externalId": "group123456789",
        "groupMemberCount": 0,
        "groupMemberCursor": 0,
        "lastActivityTime": "2021-10-08T13:39:58.000000Z",
        "maxSubscriberCount": 4294967295,
        "maxUserCount": 4294967295,
        "mtx_container_name": "MtxResponseGroup",
        "mtx_ext_ver": 1,
        "mtx_result_code": 0,
        "mtx_result_text": "OK",
        "mtx_result_type": "get",
        "mtx_sys_ver": 5240,
        "name": "This is the first group",
        "notificationPreference": 1,
        "objectId": "0-1-5-6",
        "relatedUserArray": [
            {
                "externalId": "user123456789",
                "mtx_container_name": "MtxRelatedUserObject",
                "mtx_ext_ver": 1,
                "mtx_sys_ver": 5240,
                "objectId": "0-1-5-1",
                "roleInfoArray": [
                    {
                        "externalId": "Admin",
                        "mtx_container_name": "MtxPricingRoleInfo",
                        "mtx_ext_ver": 1,
                        "mtx_sys_ver": 5240,
                        "name": "Admin",
                        "pricingId": 2
                    }
                ]
            }
        ],
        "relatedUserCursor": 0,
        "result": 0,
        "resultText": "OK",
        "routeId": 1,
        "status": 1,
        "statusDescription": "Active",
        "subscriberCount": 0,
        "subscriberMemberCount": 0,
        "subscriberMemberCursor": 0,
        "userCount": 1
    }
    mock_request.return_value = create_response(json_bytes(response_json_data))

    api_client = openapi_client.ApiClient(configuration)
    api_instance = group_api.GroupApi(api_client)
    external_id="group123456789"
    response = api_instance.group_query_external_id(
        path_params={'ExternalId': external_id}
    )
    pprint(response)

And running it one gets:
(venv) Justins-MacBook-Air:opneapi3_python justinblack$ python start-python-query-example.py openapi_client.exceptions.ApiValueError: Invalid inputs given to generate an instance of <class 'openapi_client.model.mtx_related_user_object.MtxRelatedUserObject.MetaOapg.properties.roleInfoArray.MetaOapg.items'>. Multiple oneOf schemas [<class 'openapi_client.model.mtx_pricing_role_info.MtxPricingRoleInfo'>, <class 'openapi_client.model.mtx_pricing_role_detail_info.MtxPricingRoleDetailInfo'>] matched the inputs, but a max of one is allowed.

Which makes sense because the payload:

{
    "externalId": "Admin",
    "mtx_container_name": "MtxPricingRoleInfo",
    "mtx_ext_ver": 1,
    "mtx_sys_ver": 5240,
    "name": "Admin",
    "pricingId": 2
}

Matches both of those schemas:

"MtxPricingRoleInfo": {
    "description": "Basic information about a role object.",
    "properties": {
        "externalId": {
            "description": "External ID of the role object.",
            "type": "string"
        },
        "mtx_container_name": {
            "description": "MtxPricingRoleInfo",
            "type": "string"
        },
        "mtx_ext_ver": {
            "description": "Extension Schema Version",
            "type": "integer"
        },
        "mtx_sys_ver": {
            "description": "System Schema Version",
            "type": "integer"
        },
        "name": {
            "description": "Descriptive name of the role object.",
            "type": "string"
        },
        "pricingId": {
            "description": "Pricing ID of the role object.",
            "format": "uint64",
            "type": "integer"
        }
    },
    "required": ["mtx_container_name"],
    "type": "object"
},

And

"MtxPricingRoleDetailInfo": {
    "description": "Detailed information about a role object. (extends MtxPricingRoleInfo)",
    "properties": {
        "externalId": {
            "description": "External ID of the role object.",
            "type": "string"
        },
        "metadataList": {
            "description": "Internal Use Only",
            "items": {},
            "type": "array",
            "x-matrixx-internal": "true"
        },
        "mtx_container_name": {
            "description": "MtxPricingRoleDetailInfo",
            "type": "string"
        },
        "mtx_ext_ver": {
            "description": "Extension Schema Version",
            "type": "integer"
        },
        "mtx_sys_ver": {
            "description": "System Schema Version",
            "type": "integer"
        },
        "name": {
            "description": "Descriptive name of the role object.",
            "type": "string"
        },
        "permissionArray": {
            "description": "Array of permissions associated with this role.",
            "enum": [
                "owner",
                "receive_notifications",
                "subscription_aggregator"
            ],
            "type": "string"
        },
        "pricingId": {
            "description": "Pricing ID of the role object.",
            "format": "uint64",
            "type": "integer"
        }
    },
    "required": ["mtx_container_name"],
    "type": "object"
},

Note: json schema allows in additional (unknown) properties by default. If you want only your defined properties to be allowed in set additionalProperties to false in your object schema.