Restier Swagger support for AspNetCore : PUT doesn't appears

[RichySano]

Swagger is well generated but PUT method doesn't appears in swagger UI. But PUT method is working well with POSTMAN for example.

Assemblies affected

Restier RTM 1.1.0

Program.cs

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRestier(configureApisAction =>
{
configureApisAction.AddRestierApi((routeServices) =>
{
routeServices.AddOptions();
routeServices.AddEFCoreProviderServices((services, options) =>
{
options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection"));
});
});
}, true);

builder.Services.AddRestierSwagger();

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}

app.UseRestierBatching();
app.UseRouting();
app.UseAuthorization();

app.UseEndpoints(endpoints =>
{
_ = endpoints.Select().Expand().Filter().OrderBy().MaxTop(100).Count().SetTimeZoneInfo(TimeZoneInfo.Utc);
_ = endpoints.MapRestier(builder =>
{
builder.MapApiRoute("", "", true);
});
});

app.UseClaimsPrincipals();
app.UseHttpsRedirection();
app.UseRestierSwagger(true);
app.MapControllers();
app.Run();

[DennyFiguerres]

we might want to make this a discussion .....
my prior reads of the OData Standards and the HTTP Verbs i personally feel that HTTP PATCH is a better match in most cases.

PUT implies that a new entity will fully replace the current entity.
i have always used HTTP PATCH
HTTP PATCH is more in line with a SQL UPDATE statement where you only need to update some parts of a current entity.
where implementing HTTP PUT would be more like a sql delete and then an insert.

just my thoughts at this point....

[RichySano]

I agree with you that the HTTP PATCH action is better suited to SQL with the UPDATE method but the fact remains that Restier exposes the two HTTP actions with a different behavior for each HTTP action.

For example, I have a Company entity in the database with three fields

Name: 'Company 1'
Street: 'Street 1'
Zip: 'Zip 1'

If I do a PATCH with only the Street field set to "Street 1-v1"

I get in the database:

Name: 'Company 1'
Street: 'Street 1-v1'
Zip: 'Zip 1'

Now, if I do a PUT with only the Street field set to "Street-1-v2"

I get in the database:

Name: NULL
Street: 'Street 1-v2'
Zip: NULL

Which clearly shows a difference in behavior between PUT and PATCH actions under Restier.

So if Restier exposes the two HTTP actions, why not expose the PUT action in the Swagger UI by indicating that all fields must be filled in?

Furthermore, in certain applications which will implement an OData client of this API, we will obtain a class like this:

public class company
{
public string Name { get; set; }
public string Street { get; set; }
public string Zip { get; set }
}

And when this client returns the Company object to the API, all the field values of the Company object will be populated and will use the HTTP PUT method.

This is just my opinion and a question about the implementation of PUT in OData and Restier.

Looking forward to continuing this discussion.

[Tiberriver256]

So, RESTier exposes both PATCH and PUT functionality.

Regardless of what functionality we build into RESTier, the OpenAPI should reflect that functionality. If we decide to remove the PUT functionality, we should remove it from OpenAPI. Still, until then, the documentation of what functionality RESTier exposes should be correct.

The OpenAPI is generated from the EdmModel, highlighting the fact that we have a bug in the EdmModel RESTier building. It should add an annotation that PUT is supported (PUT is not typically required for OData APIs and is thus not added by default by the convention model builder).

The nice thing is that when we fix it in the EdmModel, our OpenAPI document improves, and our /$metadata CSDL improves. It just so happens that the open API is much easier for humans to read than CSDL, so it's easier for us to spot bugs there.

[DennyFiguerres]

true, there is a point on the "correctness" and then my point on "recommended use"
both are valid items to look at.

[RichySano]

In this page of the Microsoft site

https://learn.microsoft.com/en-us/aspnet/web-api/overview/odata-support-in-aspnet-web-api/odata-v4/create-an-odata-v4-endpoint

In the Update an entity section, it is stated that OData supports two different semantics for updating an entity: PATCH and PUT.

It is stated that the disadvantage of the PUT action is having to specify all the property values of an entity even if all the properties have not changed. Of course, the PATCH action is preferred. But it is well supported by OData as specified in section 11.4.3 of OData V4.0.

http://docs.oasis-open.org/odata/odata/v4.0/os/part1-protocol/odata-v4.0-os-part1-protocol.html#_Toc372793719

"Services MAY additionally support PUT, but should be aware of the potential for data-loss in round-tripping properties that the client may not know about in advance, such as open or added properties, or properties not specified in metadata"

In my opinion, the PUT method should therefore be present in the Swagger UI.

In any case, thank you for your interest in this discussion.

Long live Restier.

[robertmclaws]

So here's my take on everything. Restier is a highly-opinionated framework. It has a very specific way that it does things, and as a core tenet it defaults to being "magical".

The documentation @RichySano pointed to clearly demonstrates that PUTs are NOT magical. The default behavior has the potential for significant data loss, including in situations where there are audit fields being persisted to entities.

Given the fact that the actual issue is in Microsoft.OpenApi.OData, and the default behavior points developers in what is arguably the right direction, I'm going to say that we should not make any changes for this feature at this time.

If you want a different behavior, you can always subclass the RestierSwaggerProvider and override GetSwagger() to leverage the returned model and append the additional attributes.

And feel free to open an issue in the Microsoft.OpenApi.OData repo if you feel the behavior of ConvertToOpenApi() should be different.

Since we pretty much have every feature of ASP.NET Core covered now, moving forward we're going to be pretty exclusively focused on redesigning Restier for v2, unless there are blocking issues or bugs.

[robertmclaws]

If Restier is actually broken then of course it will get fixed.

From everything I've been reading from you folks, the EdmModel and $metadata are correct (as they don't specify ANY HTTP methods) but Swagger doesn't show PUT requests in the UI.

This would indicate the output from Microsoft.OpenApi.OData is potentially incorrect, which is not our responsibility to fix.

[Tiberriver256]

I reviewed the specs this morning, and I believe you are correct, @robertmclaws.

I will type something out here, though, just for everyone involved in this conversation and future people interested in OpenApi stuff to reference.

RESTier OpenApi

How do you think we should prioritize work here?

All conversations about OpenApi and the EdmModel that generates the OpenApi are strictly conversations about documentation, not behavior.

With that in mind, I'd like to propose that we:

• Try to ensure the documentation is correct
• We set the expectation that issues in the backlog around broken behavior will be prioritized above issues around broken documentation (this seems entirely fair to me)

How does it work?

The history of CSDL

To understand how the OpenApi is generated, we have to take a quick step back and look at OData.

One of the biggest pitches in OData is that the API is self-documenting. There is always a /$metadata endpoint at the root of any OData API that provides documentation about the OData API. The documentation is given in the Common Schema Definition Language (CSDL) and is typically serialized into XML.

This document has always been the best way to auto-generate API clients (see ClientCodeGenerationTool for a C# example), and it's so stable in fact that you can build API clients that work for all OData APIs and consume the CSDL to check what API calls are allowed (For example, the OData client in Excel).

The problem with the CSDL is that, while there have always been ways in the eco-system to generate API clients, the only well-known way to generate human-readable documentation from the CSDL is XOData which has long been abandoned as a project.

The history of Swagger

OpenApi came along (Swagger back then) with a similar goal to the CSDL; it is a schema typically serialized in JSON or YAML. The goal of OpenApi, though, was to be generic enough to describe any REST API. This, of course, is only possible with some APIs (for example, an OData API with a recursive relationship and infinite navigation property nesting could not be statically documented in OpenApi). Still, it gets the majority of APIs the documentation that they need to do two things:

1. Generate documentation for developers
2. Generate API clients

Swagger is more generic and has a much wider community than people interested in CSDL. This gives it two significant advantages over CSDL:

1. There are a lot more documentation generators for Swagger/OpenApi than there are for CSDL
2. There are a lot more API client generators for swagger/OpenApi than there are for CSDL

Why don't we bring the two together?

Both Microsoft and SAP (the two most prominent backers of OData) are highly interested in bringing the advantages that OpenApi brings over to the OData world and thus were born a couple of initiatives for converting a CSDL over to OpenAPI:

• odata-openapi - The oldest of the two (7 years old). It's owned by the same organization that manages the OData spec and uses an XSL for transformation, so any runtime that supports XSL transforms could use it. Since then, it's also been added to a pure JavaScript transformer.
• OpenApi.NET.OData - This is a Microsoft initiative (6 years old) targeted at making that same transformation easy to do in the .NET world. In the .NET world of OData, the CSDL starts as an IEdmModel. If we were using XSLT, we would have to serialize the IEdmModel into an XML string before converting it into OpenApi. That operation might be expensive for an extensive OData API like MSGraph. This library aims to skip the serialization and convert straight from an IEdmModel over to an OpenApiDocument, which is a class provided by Microsoft/OpenApi.Net.

The primary benefit both these tools provided was an automated conversion from CSDL into OpenApi. Now, anyone using OData can leverage all the existing OpenApi/Swagger tooling!

This does come with a couple of drawbacks, though:

1. The auto-generated OpenApi schema is going to need to be able to reflect the OData schema perfectly.
2. The OpenApi can only be as good as your CSDL

Point #2 is the most important thing to remember here. The default CSDL needs to include a lot of stuff that people might want in their developer documentation (OAuth/update restrictions/permissions/maxTop/etc.). There are, however, several common extensions (known as annotations) to the standard CSDL.

There are collections of annotations known as 'vocabularies,' and there are both standard vocabularies and several vendor-specific vocabularies (example).

Microsoft's EdmModel, as far as I'm aware, supports the standard vocabularies out of the box, and I believe the OpenApi.NET.OData library also supports the standard vocabularies.

RESTier OpenApi toolchain

Step 1: Construct an EdmModel

This is done as far as I can tell via RestierWebApiModelBuilder.cs which under the hood uses ODataConventionModelBuilder. The ODataConventionModelBuilder constructs a pretty decent EdmModel by inspecting the C# classes used in EntitySets.

Step 2: Convert the EdmModel to an OpenApiDocument using OpenApi.NET.OData

This is a pretty straightforward call. All the options from OpenAPI.NET.OData are exposed to developers so they can provide extra information in the OpenApi that isn't possible to include in the CSDL.

Step 3: We serve the OpenApiDocument using the SwashBuckle middleware

This isn't 100% necessary. We could probably just as easily make an endpoint called /$openapi that serves the same OpenApiDocument and drop our dependency on SwashBuckle. Still, for now, it's probably the simplest way to serve up an OpenApi file.

Triaging RESTier OpenApi issues

Given all the information above, I propose that the triage of OpenApi issues should look like this:

1. Check CSDL to validate it is correct
2. Check CSDL to see if it can be expanded with more verbose vocabulary
3. Check OpenApi settings to see if the desired behavior is an option
4. Defer user to request feature/bugfix from microsoft/OpenAPI.NET.OData

Let's triage this HTTP PUT issue

Is the CSDL generated by RESTier correct?

Yes.

Can the CSDL be extended with a vocabulary?

Yes, but it wouldn't get us beyond what's currently available.

As far as I can tell, the best vocabulary to describe what we want in the CSDL is Org.OData.Capabilities.V1.UpdateRestrictionsType. It does seem to communicate what HTTP methods are allowed for update (i.e., PATCH vs. PUT) and is supported by OpenApi.NET.OData. The only problem is that the vocabulary isn't specific enough for what we need to communicate:

Supported HTTP Methods (PUT or PATCH) for updating an entity. If null, PATCH SHOULD be supported, and PUT MAY be supported.

So, what we have now is the equivalent of null, which communicates PATCH SHOULD be supported and PUT MAY be supported. The whole 'MAY be supported' is probably why the OpenApi team decided to leave it out of the conversion.

[DennyFiguerres]

nice summary, i think its accurate at a glance.....

[robertmclaws]

@Tiberriver256 Thanks for taking the time to do this, Micah!

[RichySano]

Thank you all for your answers. @Tiberriver256 thank you for your time