Implement Standard Schema interface

[marcomuser]

Have you already considered implementing the standard schema interface: https://github.com/standard-schema/standard-schema. This would provide third-party libraries a uniform integration to automatically support multiple schema libraries at once. Currently, zod, valibot and arktype implement the interface. I'm working on a library integrating with schema libs and I would love to also integrate with typebox without writing a custom adapter.

[sinclairzx81]

@marcomuser Hi, thanks for the suggestion.

There is a provisional Adapter to Standard Schema being developed and documented at the following URL

https://github.com/sinclairzx81/typebox/tree/master/example/standard

I have a lot to say about Standard Schema, will outline my thoughts on it below.

---

Standard Schema (Review)

Unfortunately, I won't be able to implement the Standard Schema interface on TypeBox types because doing so would require TypeBox to include non-standard keyword vocabulary, such as ~standard on schematics, as well as require it to augment schematics with TypeBox specific validation functions (which wouldn't make sense for users who use TypeBox solely with Ajv)

Overall, I am not very keen on the design of Standard Schema, not only for the reasons mentioned above, but primarily because the specification would rule out any library that separates schematics from validation. This is a principled design decision that libraries make (i.e. adhering to separation of concerns) and unfortunately Standard Schema would require such libraries to violate these principles.

const result = schema['~standard'].validate(value) // Incorrect: Because schemas should not have 
                                                   // functions and should not validate themselves. 

const result = validate(schema, value)             // Correct: Schemas are parameters to validators

While the former is possible in Zod, Valibot and Ark (which do carry validation around on their types), it's not feasible for TypeBox as it only builds Json Schema, and where those schematics are intended to be passed to any Json Schema compliant validator.

const result = (new Ajv()).validate({ type: 'string' }, value) 
//                                  ^
//                                  |
//                                  Optional: Replace with Type.String()

These are the main concerns I have with the specification (Jan 2025)

---

More Issues

Beyond the above concerns, I have several additional issues with the specification:

• The naming is misleading, as API interfaces have nothing to do with schematics (unless the intent is to conflate these terms).
• The return type for validate suggests a parse operation. Additionally, the returned value requires initialization for each validation call, hindering optimization. The specification should consider separating parse and validate into distinct API calls.
• Validation errors should be accessible via a separate API call rather than always being generated. Validation errors are diagnostics that incur a processing cost and may not be necessary in all environments. A system may prefer a boolean result instead of detailed diagnostics under the "minimal-disclosure" or "give no hints" principle.
• Validation errors currently expand all errors into potentially large objects, with no DoS mitigation mechanism for types that may generate large error sets. An error function that returns an iterator may address this issue (but do think a larger discussion should be had around error generation)
• The async return for validation does not make sense and should be removed.
• The specification makes no provisions for JIT (Just-In-Time) evaluation.

---

Adapter

Needless to say, with all of the above, it isn't reasonable for TypeBox to break compatibility with a already widely adopted industry standard specification (Json Schema) just to unify with a few popular JavaScript validation libraries.

Irrespective of the ergonomics or perceived DX wins integrators are expected to get (which I expect was largely used to inform the specification design), I really don't consider Standard Schema to be forward looking enough in terms of overall design, or representative of the myriad of other ways integrators can approach uniform type library adoption in frameworks.

As such, an Adapter will be required.

import { StandardSchema } from './standard' // is it really standard?

const T = StandardSchema(Type.Object({  // Breaks Json Schema compliance by augmenting
  x: Type.Number(),                     // a TypeBox schematic with functions and keywords
  y: Type.Number(),                     // required by the Standard Schema interface.
  z: Type.Number(),
}))

---

Alternatives

If you are looking for an alternative to Standard Schema, I would recommend the following projects (which are supportive of a wider range of type libraries and validators)

Current Gold Standard

https://github.com/decs/typeschema

Something I am working on which is a generalization of Fastify Type Providers

https://github.com/sinclairzx81/type-adapters

Something else I am exploring

https://github.com/sinclairzx81/typebox-adapter

---

Hope this brings some insight into where TypeBox is at with regards to Standard Schema. Happy to discuss more on this thread if you have any questions though.

I have been asked to comment on the Standard Schema repository. Unfortunately, it is very difficult for me to communicate aspects of the specification without me suggesting that the authors adopt a design that runs contrary to how their libraries are written. I do note that the specification is written in service to frameworks such as tRPC where the specification does a better job of integration than previously, but falls short of servicing frameworks that are built upon industry specifications like Json Schema.

There is a lot to discuss, but too broader context in which to discuss it. But hopefully the above communicates my thoughts around it.

Cheers
S

[marcomuser]

Thank you very much for providing such a detailed response. What you're saying makes sense to me! I wasn't aware of the implications for json schema tooling compatibility.

Since you're not planning to implement it, would it make sense to convert this issue to a discussion (instead of closing it) so that your arguments are somewhat documented? I have a feeling that this will not be the last time that someone will wonder how simple it would be for typebox to implement the interface.

[sinclairzx81]

@marcomuser Hi, sorry for the delay in response (busy start to the year)

Since you're not planning to implement it, would it make sense to convert this issue to a discussion (instead of closing it) so that your arguments are somewhat documented?

Ah, will keep this issue open for a while. Keeping it as an issue helps to make it a bit more visible as well as reminding me to keep an eye on the updates to the spec.

As mentioned though, bar substantial changes to the specification to split validation from schematics, an adapter to Standard Schema is going to be the way to go. The current plan is to continue to revise the reference implementation based on updated spec info, but I may ultimately end up publishing the formal implementation to the following project.

https://github.com/sinclairzx81/typebox-adapter

import { StandardSchema } from '@sinclair/typebox-adapter'

const T = StandardSchema(Type.String())

Will keep you posted on any changes / updates.
Cheers
S

[sinclairzx81]

@marcomuser Hi, quick update

I have started work on a new project called TypeMap. This project offers advanced translation between libraries and supports Standard Schema by way of projects Compile function. This function makes supported libraries compatible with Standard Schema and also offers accelerated type checking by allowing remote libraries to run on TypeBox infrastructure.

Project

TypeMap Project

Example

Reference

[sinclairzx81]

@marcomuser Hiya,

I might convert this issue into a discussion as TypeMap has been formally published as well as added to the Standard Schema supported library list. TypeMap will be the primary way of integrating TypeBox with Standard Schema moving forward, as well as serve as a vehicle for innovation in the type inference / mapping space.

Project

TypeMap Project

---

Happy to keep the conversation going, feel free to reach out if you have anything thoughts or suggestions on the design.
Cheers
S

[marcomuser]

Perfect! This looks awesome. Thank you for your effort!!