[BUG][PYTHON] Generated models and optional parameters

[mihai80]

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

I am using the version 7.0.0 dated 2023/05/16 to generate Python code and the models do not take into account the readonly and optional flags from the spec file. As an example, my API includes a component called DataCollectionPlan, and a POST for the same component schema.

In order to use the example code, I must declare the Python object with all the optional and readonly parameters, such as id or description. The id is not necessary when using POST, while the optionals can be set to None, but this is not the correct way to treat optional object parameters.

openapi-generator version

7.0.0 dated 2023/05/16

OpenAPI declaration file content or url

The YAML section of the component.

    DataCollectionPlan:
      type: object
      properties:
        id:
          type: string
          readOnly: true
        name:
          type: string
        description:
          type: string
        kafkaTopic:
          type: string
          description:
        intervalInMinutes:
          type: number
        equipment:
          type: object
          properties:
            id: 
             type: string
        equipmentConnection:
          type: object
          properties:
            id: 
             type: string
        subscribers:
          type: array
          items:
            $ref: '#/components/schemas/Subscriber'
        requestList:
          type: array
          items:
            oneOf:
              - $ref: '#/components/schemas/AlarmRequest'
              - $ref: '#/components/schemas/EventRequest'
              - $ref: '#/components/schemas/TraceRequest'
        appKey:
          type: string
          readOnly: true
          description: UUID of the App which created this DCP.
      required:
        - id
        - name
        - equipment
        - equipmentConnection
        - requestList

The YAML section of the POST.

    post:
      summary: Create a Data Collection Plan
      description:       
      tags:
        - DataCollectionPlan
      parameters: 
      - in: header
        required: true
        name: X-App-Key
        schema:
          type: string
        description: App Key in UUID format
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/DataCollectionPlan'

Generation Details

I tried both with Java version 8 and using the Docker container from openapitools/openapi-generator-cli:latest.

Steps to reproduce

Just generated code as normal.

Related issues/PRs

None.

Suggest a fix

This was not an issue in the version 7.0.0 dated 2022/05/26. It seems like the model files had a different structure, and they took into account when model is used for POST or for results from PUT.

[mihai80]

Also is anyone kind enough to point me to these new changes to the Python generator? I cannot really see the model change in the pull requests, but I might be wrong of course.

Edit: Found it, the Python generator changed #15486

On another note, does 7.0.0 treat the naming of components in a different way? I am asking this because for me the stable version 6.6.0 does not work, while 7.0.0 works just fine. Version 6.6.0 returns the error "ValueError: Invalid name, name may not be one of" {'Accept', 'Content-Type', 'Authorization'}". I suspect this is due to the naming of the parameter called "Authorization" used in my API as the bearer token auth.

[spacether]

Per the openapi spec:
https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#fixed-fields-10
You may not use these names as header keys:

• Authorization
• Accept
• Content-Type

If you need a header with the name Authorization, use

• SecurityScheme.type="apiKey"
• SecurityScheme.name="Authorization"

Or use

• SecurityScheme.type="http"
• SecurityScheme.scheme="bearer"

read all about it in the spec here

One can see this working in the new repo for the v6.6.0 python generator:

• api_key: https://github.com/openapi-json-schema-tools/openapi-json-schema-generator/blob/master/samples/openapi3/client/features/security/python/test_manual/test_paths/test_path_with_one_explicit_security/test_get.py#L50
• http bearer: https://github.com/openapi-json-schema-tools/openapi-json-schema-generator/blob/18864f4c21ebd2167a8eb911bfa070da9a678d13/samples/openapi3/client/features/security/python/test_manual/test_paths/test_path_with_two_explicit_security/test_get.py#L75

[mihai80]

Hi, thank you for answering. I already use bearerAuth and get the error. I do not get it with version 7.0.0.

In any case, should it not just ignore my "Authorization" parameter, as suggested by the spec "If in is "header" and the name field is "Accept", "Content-Type" or "Authorization", the parameter definition SHALL be ignored."

  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

and

  parameters:
    auth:
      in: header
      name: Authorization
      required: true
      schema:
        type: string
      description: "Bearer JWT token verified to ensure request is from valid consumer. Format \"Bearer <jwt-token>\"\
          .\nExample, \"Bearer eyJ0eXAiOiJKV1QiLCJhbGc....\""

[nezhar]

I encountered the same issue after generating a Python client for https://demo.openremote.app/api/master/openapi.json

I previously used the Docker image openapitools/openapi-generator-cli:v6.6.0. Now, I've switched to openapitools/openapi-generator-cli:latest, which resolved the problem. However, I also noticed that the structure of the final module and the documentation have changed.

[mihai80]

I am coming back with some remarks, which I can open various "issues" on a bit later when I have time.

I tested various versions and found out that:

• Version 5.4.0 works with the "Authorization" parameter and http bearer
• Version 6.5.0 does not work with the "Authorization" parameter and http bearer unless changing name to "_Authorization" or something else
• Version 6.6.0 does not work with the "Authorization" parameter and http bearer unless changing name to "_Authorization" or something else
• Version 7.0.0 release 20220526 to 20230516 works with the "Authorization" parameter and http bearer

I also found out that:

• Version 5.4.0 generates wrong documentation for configuration parameter, adding
  configuration = example.Configuration(host="") and configuration = example.Configuration(access_token="") on two different lines, resulting in the redeclaration of configuration without "host" parameter
• Version 6.5.0 same as above
• Version 6.6.0 same as above
• Version 7.0.0 20220526 to 20230516 same as above

This configuration for host and access_token is fixed by putting them into one configuration = example.Configuration(host="", access_token="") or by assigning variable to object using configuration = Configuration() and then configuration.host = "" and configuration.access_token = ""

[mihai80]

Concerning the main topic of this thread, I have found that using stable version 5.4.0 has the readonly and optional flags from the spec file correctly generated and present in the model files. This is also present in version 7.0.0 release 20220526.

I also found that the documentation is correctly rendered using stable version 5.4.0 and 7.0.0 release 20220526 and pylint that comes with VSCode. This means that I get the correct description while hovering over assigned object models, with their types.

Otherwise:

• Version 6.5.0 and 6.6.0 return the message "Example" is not accessedPylance when hovering over imported objects models and do not show any description of parameters
• Version 6.5.0 and 6.6.0 return the message this is needed to fix 'Unexpected argument' warning in pycharm this code does nothing because all Schema instances are immutable this means that all input data is passed into and used in new, and after the new instance is made no new attributes are assigned and init is not used while hovering over assigned object models, together with missing parameters which return the following description class Example( *_args: dict | frozendict | list | tuple | Decimal | float | int | str | date | datetime | bool | Schema | None, _configuration: Configuration | None = None, **kwargs: dict | frozendict | list | tuple | Decimal | float | int | str | date | datetime | bool | Schema | Unset | None )
• Version 7.0.0 release 20230516 have a different format when hovering over assigned object models, somewhat more confusing than version 7.0.0 release 20220526.

[gneissone]

Based on the initial bug report, I may be having the same issue, but the rest of the thread doesn't jive...
In any case, I'm using openapi-generator to create a Python client. It's adding example mock values for optional parameters into the example Python code it creates.

Example schema fragment:

        - name: modifiedTime
          in: query
          required: false
          schema:
            type: string
          description: Limit results to modified time value

then in the provided example Python code in the generated README.md:

# Enter a context with an instance of the API client
with biz.product.client.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = biz.product.client.DocumentsApi(api_client)
    modified_time = 'modified_time_example' # str | Limit results to modified time value (optional)

The autogenerated 'modified_time_example' string doesn't mean anything to the API and it breaks if it's run without replacing it with a valid string. Seems to me that if the schema has the param as optional and no example is provided, it shouldn't be used in the example code...