Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds python-experimental which uses dynamic base classes #8325

Conversation

spacether
Copy link
Contributor

@spacether spacether commented Jan 4, 2021

Adds python-experimental which uses dynamic base classes

In this PR:

  • when deserializing payloads or creating Schema/Model instances the new instance will dynamically inherit from the correct Schema classes (models) and immutable python primitive classes (frozendict, tuple, str, list, Decimal, bytes, NoneClass, BoolClass)

This is amore robust composition implementation than the standard python generator implementation because:

  • any level of inline schema is supported (one won't need to extract composed schemas into components anymore)
  • schema property names are used so there's no mutation of data when converting to python style variable names because python style variable names are not used

Separate endpoint parameters are made for for query, path, cookie, and header params because each of those types can contain parameters with colliding names. So having them separate allows us to preserve the original key naming in our inputs to the endpoint. One can see this working in the parameter_collisions endpoint that I added.

Examples

  • Map payload w/ Object component Schema (Model): An object model Box will inherit from Box, DictSchema, and frozendict
  • Map payload w/ untyped composed component Schema (Model): For a composed model Triangle oneOf includes ScaleneTriangle, the resulting instance will inherit from Triangle + ScaleneTriangle, DictSchema, and frozendict

Caviats:

  • None, True, and False are singletons in python so if they were used, one would not be able to subclass them. I made a new Singleton class and used it to make base classes for those cases, NoneClass and BoolClass
  • the Singleton subclasses NoneClass + BoolClass will allow me to generate type hints for XSchema + y where y is a base class like (frozendict, tuple, str, Decimal, NoneClass, BoolClass. XSchema is a class like Triangle or DictSchema.
  • the Singleton class is used to generate Enum options, and enum classes can inherit from eachother. This is necessary when EnumA and EnumB are both validated for the same payload.
  • when BinarySchemas are used, either files or bytes may be stored in them. When files are stored, the schema instance will be mutable. When bytes are stored, the schema instance is immutable.

Feature Additions:

  • uniqueItems validations added
  • model instances are now immutable. This was done to prevent users from adding/mutating model instance data after validations have already been run. For example if a composed discriminated model is made and a user was allowed to change the value of the discriminated variable className, that would invalidate the validation that was done on that instance.
  • type inheritance is clear and deterministic, when multiple overlapping types are validated. Note: int/float/number are all stored as Decimal instances
    • object type inherits from frozendict
    • array type inherits from tuple
    • null type inherits from NoneClass
    • boolean type inherits from BoolClass
    • number/int/float types inherit from Decimal
    • string/date/datetime types inherit from str
      • string Decimals/uuid etc can also inherit from str in the future
  • openapi 3.0 parameter styles implemented (except for deepObject)
  • api import time reduced; one endpoint or api can now be imported and used without loading others
  • multiple content types supported in endpoint request bodies, see add_pet
  • accept header content types added as an input, user overrides are also allowed for the accept header
  • multiple response content types are allowed, see find_pets_by_status

Long Term Benefits of Using Dynamic Base Classes + This New structure

  • the code is more maintainable: casting is done in one place, validation is much clearer
  • any depth of inline models can be supported. Many filed issues + bugs occur because users are trying to use composed inline schemas.
  • model definition is shorter
  • This will save us from traversing composed schema relationship trees to figure out if NullableString/XSchema is allowed to contain None/True/False/some other primitive type
  • This supports complicated use cases like
    • allOf -OneOfComposedSchema. OneOfComposedSchema oneOf: -schemaA -schemaB
    • enums containing multiple types
    • a schema that combines two schemas where the type definition of a variable is different str, number vs number
    • composed schemas with type constraints
  • The number of times that validations and type conversion are done is reduced to only when the data is ingested and the schemas are validated
    • once to make the dynamic class from a payload, then skip it when assigning object values at init time
    • no need to run this code again when sending an instance of NullableString to the server
  • when one has an instance of a composed schema, one will know what schemas passed validation for that payload
  • simplicity of storing payload info once. Previously, dict properties were stored in multiple instances inside a composed schema, now property values are only stored in one place inside a single model dict
  • make it easy to incorporate multiple types in openapi >= 3.1.0 by using StrBase, ListBase etc classes
    • this is already implemented for nullable schemas, like str + nullable=True, see NullableString

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples:
    ./mvnw clean package 
    ./bin/generate-samples.sh
    ./bin/utils/export_docs_generators.sh
    
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    For Windows users, please run the script in Git BASH.
  • File the PR against the correct branch: master (5.3.0), 6.0.x
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

@spacether spacether marked this pull request as draft January 15, 2021 19:13
@spacether spacether changed the title Adds python experimental dynamic base classes Adds python-experimental which uses dynamic base classes Jan 15, 2021
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch from 16357f1 to 3a55984 Compare January 18, 2021 18:13
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 2 times, most recently from 8848551 to 0f34f4f Compare January 21, 2021 18:44
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 2 times, most recently from 87b7f0b to 30c02b3 Compare March 8, 2021 15:51
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 4 times, most recently from 156e66b to 573a3f4 Compare May 27, 2021 02:16
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 3 times, most recently from 36c6151 to 764e68e Compare July 13, 2021 15:37
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 3 times, most recently from 70e66d3 to eba9916 Compare August 9, 2021 01:55
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 2 times, most recently from 6529f36 to 3644311 Compare September 6, 2021 03:09
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 5 times, most recently from e374308 to a47830e Compare December 20, 2021 20:27
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 4 times, most recently from 9dd1a14 to 5196228 Compare December 28, 2021 22:34
@spacether spacether force-pushed the adds_python_experimental_dynamic_base_classes branch 2 times, most recently from f0b2bcc to 5615747 Compare January 4, 2022 02:27
@spacether spacether marked this pull request as ready for review January 5, 2022 18:33
@spacether spacether added this to the 5.4.0 milestone Jan 5, 2022
@spacether spacether added Bleeding Edge Experimental features (e.g. non-standard spec, "futures", or IETF draft) Client: Python and removed Bleeding Edge Experimental features (e.g. non-standard spec, "futures", or IETF draft) labels Jan 5, 2022
@spacether
Copy link
Contributor Author

If people want to continue using this generator in the future please use it from https://github.com/openapi-json-schema-tools/openapi-json-schema-generator as it is being moved there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant