Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for explicit HTTP parts #3046

Closed
Tracked by #609
bterlson opened this issue Mar 22, 2024 · 2 comments · Fixed by #3342
Closed
Tracked by #609

Support for explicit HTTP parts #3046

bterlson opened this issue Mar 22, 2024 · 2 comments · Fixed by #3342
Assignees
Labels
design:accepted Proposal for design has been discussed and accepted. triaged:core
Milestone

Comments

@bterlson
Copy link
Member

bterlson commented Mar 22, 2024

We have made some initial forays into multipart/form-data by piggy-backing off of the OpenAPI approach, which essentially maps a JSON-like structure to parts through various rules, like the object keys are the form name, "bytes" means "files", arrays mean multiple parts, etc. This works well for simple cases, but leaves behind some gaps, and has some non-optimal behavior. In particular:

  1. No way to override the content type of a specific part
  2. No natural way to represent files with optional/required file name/content-type
  3. Ambiguity between array-of-parts vs. part-that-is-array
  4. No way to define custom headers for a particular part
  5. No way to control the various serialization options like explode, deepObject, etc.
  6. No way to explicitly configure a boundary (unsure if necessary)

Overall, while the current design of making MFD endpoints "feel like" they take an object, and emitters doing the best they can to represent an endpoint that takes that object across multiple parts, this leaves a lot of gaps.

OpenAPI addresses these limitations with the encoding property that lives alongside schema:

requestBody:
  content:
    multipart/form-data:
      schema:
        type: object
        properties:
          file: # a part
      encoding:
        file: #encoding for file

multipart/form-data is a more specialized form of multipart requests. OpenAPI has support for multipart/mixed, and uses the same definition for it (NB: it is not clear whether the schema properties are reflected in a multipart/mixed request in the same way they are for multipart/form-data using the content disposition header, but this would be strange given that multipart/mixed doesn't require usage of the content-disposition header).

This proposal aims to address these deficiencies.

Full example

model Person {
  fullName: string;
  address: string;
}

model File {};

@maxItems(1)
@minItems(3)
model FileArray is HttpPart<File, #{ name: "File" }>[];

model ImageFile is File {
  contentType: "image/jpeg" | "image/png"
}

model RequiredFileMetadata is File {
  contentType: string;
  filename: string;
}

op upload(
  @header `content-type`: "multipart/form-data",
  @multipartBody body: [
    // # String body
    HttpPart<string, #{ name: "fullName" }>,
    // # String body, name inferred from body:
    HttpPart<{ @body fullName: string }>,
    // # Object body
    HttpPart<Person, #{ name: "Person" }>,
    // # Array body (JSON array)
    HttpPart<Person[], #{ name: "People" }>,
    // # Multiple parts
    HttpPart<string, #{ name: "Names"}>[],
    // # Custom headers
    HttpPart<{@header custom: "hi!" , @body name: string }>,

    // # File using Bytes
    HttpPart<bytes, #{ name: "file" }>,
    // # File using File
    HttpPart<File, #{ name: "file" }>,
    // # Multiple files
    HttpPart<File, #{ name: "file" }>[],
    // # Between 1 and 3 files
    FileArray,
    // # Limited content types
    ImageFile,
    // # Limited content types, but inline
    HttpPart<{ @header `content-type`: "image/jpeg", @body image: File }>,
    // # With required metadata
    HttpPart<RequiredFileMetadata, #{ name: "file" }>,
    

    // # ERROR CASES
    // ## No name specified, required for multipart/form-data parts
    HttpPart<Person>
  ]
]): void;

This same spec would work also for multipart/mixed. However, for the case of multipart/form-data in particular, where every part is required to have a name, we have a convenience form that takes an object:

op upload(
  @header `content-type`: "multipart/form-data",
  @multipartBody body: {
    // # String body
    fullName: HttpPart<string>,
    headShots: HttpPart<Image>[]
  }
): void;

This form is very similar to what we have today, but the values must be wrapped in HttpPart. So for determining the name of a part, the following is the precedence order:

  1. The explicitly provided name on the part
  2. Name of the property in the object form
  3. Inferred from body parameter

If none of these are specified, it's an error.

@multipartBody

This decorator is used in place of @body to signify a multipart request/response. It is an error for an HttpPart to appear in a regular @body. It is an error for a multipartBody to be anything except an array of HttpParts.

HttpPart<TBodyRoot extends {}, TOptions extends valueof PartOptions>

The HttpPart has two parameters, the body root of the part, and options. The body root behaves identically to the argument list of an operation. It should always be safe to refactor an operation into a multipart operation by copying the parameter list of the operation into an anonymous model passed to HttpPart.

It is an error for a multipart/form-data part to specify the content-disposition header, as this is essentially fixed for this content type.

File

The file type is a well-known type representing a File:

@Internal.httpFile
model File {
  contentType?: string;
  filename?: string;
  contents: bytes
}

Usage of file is not required when uploading file contents for multipart requests. When merely bytes is used, our existing behavior continues (which is essentially to treat bytes in a multipart/form-data request as if it were a File).

File exists for two reasons:

  1. Allows changing the requiredness of the optional metadata properties
  2. Allows uniform API descriptions across content types, e.g. when uploading a file over application/json, the File is just a regular object with three properties, but when uploading a file over a multipart request, emitters can know that the additional properties on the well-known File can be serialized in the content-disposition and content-type headers.

Internal @httpFile

Marks a model as an http file. Must have at least a contentType, filename, and contents property.

Deprecations

We should deprecate multipart/form-data with implicit or explicit body.

@timotheeguerin timotheeguerin added the design:proposed Proposal has been added and ready for discussion label Mar 22, 2024
@markcowl markcowl added this to the [2024] April milestone Mar 25, 2024
@markcowl markcowl added design:accepted Proposal for design has been discussed and accepted. and removed design:proposed Proposal has been added and ready for discussion labels Apr 2, 2024
@markcowl markcowl modified the milestones: [2024] April, Backlog Apr 8, 2024
@timotheeguerin timotheeguerin self-assigned this May 10, 2024
@MaryGao
Copy link
Member

MaryGao commented May 11, 2024

@timotheeguerin One more ask...
Recently we received the requirement of setting the default value for filename(see comment), currently JS implement this with customization code.

But I was wondering if we plan to accept this requirement in typespec side. Some service may want the default value for filename, some may not. Is it possible to allow service team to define this? maybe define in client-level setting or property-level one?

Also I notice that when the filename is optional, whether to set default value may be inconsistant cross language, so what is the final decision?

@timovv any insight?

image

@timotheeguerin
Copy link
Member

I don't think this was part of this design, this only covers the fact that you can make the filename a required property in a part.(You can also set a default with a property default but that would be the service default) For client we don't have any defaults yet so feels like that design needs to be sorted out first.

github-merge-queue bot pushed a commit that referenced this issue Jun 3, 2024
resolve #3046
[Playground](https://cadlplayground.z22.web.core.windows.net/prs/3342/)

Add the following:
- `@multipartBody` decorator
- `File` type
- `HttpPart<Type, Options>` type


Had to do a decent amount of refactoring to be able to reuse the body
parsing, this result in a much cleaner resolution of the body and other
metadata properties across request, response and metadata.

The way it works now is instead of calling `gatherMetadata` that would
just get the properties that are metadata but also ones with `@body` and
`@bodyRoot` we now call a `resolveHtpProperties`, this does the same
resolution in term of filtering properties but it also figure out what
is the kind of property in the concept of http(header, query, body,
etc.) this leaves the error resolution to this function for duplicate
annotations.
What is nice is now we don't need to keep asking oh is this a query or a
header or a body we can just check the kind of `HttpProperty`

also resolve #1311
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design:accepted Proposal for design has been discussed and accepted. triaged:core
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants