Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proprietary schemas using native zod methods and Metadata #1442

Merged
merged 67 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
e995432
Exp: upload schema using z.custom() and metadata.
RobinTail Jan 4, 2024
3c9ceba
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 4, 2024
8a10fd6
Renaming test.
RobinTail Jan 4, 2024
2f0a691
Ref: moving meta wrapper.
RobinTail Jan 4, 2024
d6cfe1b
Direct re-export in prop schema root.
RobinTail Jan 4, 2024
7b898fe
Ref: extracting helper.
RobinTail Jan 4, 2024
e3d6d4c
Date In schema using pipeline.
RobinTail Jan 5, 2024
3cb7ac5
DateOut schema using native methods.
RobinTail Jan 5, 2024
0d85bdc
Ref: Tests naming.
RobinTail Jan 5, 2024
e76e40f
Ref: adj todo.
RobinTail Jan 5, 2024
af412b0
Draft: ez.file() using native methods.
RobinTail Jan 5, 2024
ad1414c
Utilizing walker for all proprietary schemas.
RobinTail Jan 5, 2024
00a16c2
Ref: test naming.
RobinTail Jan 5, 2024
656db7a
Ref: minor, DNRY.
RobinTail Jan 5, 2024
2ca478f
Ref: hasRaw better implementation, removing maxDepth from hasNestedSc…
RobinTail Jan 5, 2024
ba3c083
Cleanup.
RobinTail Jan 5, 2024
03c7f20
Fix: Binary should be string|Buffer, Ref: restoring backward compatib…
RobinTail Jan 5, 2024
c8df2c5
Ref: extracting DeprecatedMethods inteface.
RobinTail Jan 5, 2024
4e33972
Ref: correcting type for binary case.
RobinTail Jan 5, 2024
dd04c33
Using overloads for type narrowing of the file schema.
RobinTail Jan 5, 2024
153f9a9
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 5, 2024
8656fbd
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 5, 2024
b85f029
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 5, 2024
db22c35
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 5, 2024
390f2cf
Ref: shortening isProprietary.
RobinTail Jan 5, 2024
3f8aaf3
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 5, 2024
43ed466
Ref: DNRY in test.
RobinTail Jan 5, 2024
63ab3a6
Ref: DNRY: extrcting proprietary() helper.
RobinTail Jan 5, 2024
46b1ebf
Ref: moving regular expressions to schema helpers.
RobinTail Jan 5, 2024
02cfde3
Ref: simpler backward compatibility for ez.file().
RobinTail Jan 5, 2024
a249404
Ref: remove redundant const.
RobinTail Jan 5, 2024
aec8b37
Ref: and one more.
RobinTail Jan 5, 2024
1b4c546
Ref: reducing any, using defineProperty instead.
RobinTail Jan 6, 2024
2f39a54
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 6, 2024
232dd89
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 6, 2024
6f523f1
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 6, 2024
3aa2eb2
Ref: less diff.
RobinTail Jan 6, 2024
55eaa40
Ref: making raw properietary and simplifying hasRaw() that way.
RobinTail Jan 7, 2024
de4ce3c
Removing zod prefix from proprietary kinds.
RobinTail Jan 7, 2024
2835845
Ref: Keeping ProprietaryKinds next to ez namespace.
RobinTail Jan 7, 2024
235f3ac
Enforcig constraints on proprietary metadata.
RobinTail Jan 7, 2024
af4960f
Todo: potential future alternative on native branding (if accepted by…
RobinTail Jan 7, 2024
123e7fa
Rev: maxDepth feature of hasNestedSchema.
RobinTail Jan 7, 2024
336de50
Simpler handler finding in schema walker.
RobinTail Jan 7, 2024
319b02e
Ref: regular flow for depicting Raw.
RobinTail Jan 7, 2024
7df8865
Ref: shortening for Integration.
RobinTail Jan 7, 2024
f1f010d
REF: better implementation for ez.file() using inversive approach.
RobinTail Jan 7, 2024
fe40199
Ref: fixing conversion to unknown.
RobinTail Jan 8, 2024
2972770
Ref: reducing diff in zts.
RobinTail Jan 8, 2024
62e9463
Additional types testing for ez.file().
RobinTail Jan 8, 2024
a6d4f85
Ref: removing redundant comments.
RobinTail Jan 8, 2024
e9dfa20
Ref: marking narrowing methods of ez.file() as deprecated.
RobinTail Jan 8, 2024
8852fab
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 8, 2024
7c58cef
Ref: naming, variants.
Jan 8, 2024
1c7e533
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 8, 2024
de56a6d
Using proprietary kind references in walker rules for maintainability.
RobinTail Jan 8, 2024
d2fbd14
Ref: simpler depiction of ez.raw() by introducing RawSchema type.
RobinTail Jan 8, 2024
118a48d
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 9, 2024
4dcba17
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 10, 2024
dc3f6c5
Merge branch 'master' into experimental-prop-schemas
RobinTail Jan 10, 2024
c8f2ffd
Changelog: the future 16.3.0.
Jan 10, 2024
0b40c2b
Readme: adjusting documentation.
Jan 10, 2024
c0646f3
Adjusting the example.
Jan 10, 2024
a6a2ffc
16.3.0-beta1
Jan 10, 2024
50eb27d
Ref: naming: proprietaryKind -> kind.
RobinTail Jan 10, 2024
0d9c079
Better backward compatibility: supporting the refinement message (not…
RobinTail Jan 10, 2024
ce7c50a
Adjusting the client type for file:binary.
RobinTail Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

## Version 16

### 16.3.0

- Switching to using native `zod` methods for proprietary schemas instead of custom classes (`ez` namespace):
- Each proprietary schema now relies on internal Metadata;
- Validation errors for `ez.file()` are changed slightly;
- The following refinements of `ez.file()` are deprecated and will be removed later:
- ~~`ez.file().string()`~~ — use `ez.file("string")` instead,
- ~~`ez.file().buffer()`~~ — use `ez.file("buffer")` instead,
- ~~`ez.file().base64()`~~ — use `ez.file("base64")` instead,
- ~~`ez.file().binary()`~~ — use `ez.file("binary")` instead.

### 16.2.2

- Fixed issue #1458 reported by [@elee1766](https://github.com/elee1766):
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -708,14 +708,14 @@ You can find two approaches to `EndpointsFactory` and `ResultHandler` implementa
[in this example](https://github.com/RobinTail/express-zod-api/blob/master/example/factories.ts).
One of them implements file streaming, in this case the endpoint just has to provide the filename.
The response schema generally may be just `z.string()`, but I made more specific `ez.file()` that also supports
`.binary()` and `.base64()` refinements which are reflected in the
`ez.file("binary")` and `ez.file("base64")` variants which are reflected in the
[generated documentation](#creating-a-documentation).

```typescript
const fileStreamingEndpointsFactory = new EndpointsFactory(
createResultHandler({
getPositiveResponse: () => ({
schema: ez.file().buffer(),
schema: ez.file("buffer"),
mimeType: "image/*",
}),
getNegativeResponse: () => ({ schema: z.string(), mimeType: "text/plain" }),
Expand Down Expand Up @@ -899,7 +899,7 @@ Some APIs may require an endpoint to be able to accept and process raw data, suc
file as an entire body of request. In order to enable this feature you need to set the `rawParser` config feature to
`express.raw()`. See also its options [in Express.js documentation](https://expressjs.com/en/4x/api.html#express.raw).
The raw data is placed into `request.body.raw` property, having type `Buffer`. Then use the proprietary `ez.raw()`
schema (which is an alias for `z.object({ raw: ez.file().buffer() })`) as the input schema of your endpoint.
schema (which is an alias for `z.object({ raw: ez.file("buffer") })`) as the input schema of your endpoint.

```typescript
import express from "express";
Expand Down
2 changes: 1 addition & 1 deletion example/example.documentation.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: Example API
version: 16.2.2
version: 16.3.0-beta1
paths:
/v1/user/retrieve:
get:
Expand Down
2 changes: 1 addition & 1 deletion example/factories.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ export const fileStreamingEndpointsFactory = new EndpointsFactory({
config,
resultHandler: createResultHandler({
getPositiveResponse: () => ({
schema: ez.file().buffer(),
schema: ez.file("buffer"),
RobinTail marked this conversation as resolved.
Show resolved Hide resolved
mimeType: "image/*",
}),
getNegativeResponse: () => ({
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "express-zod-api",
"version": "16.2.2",
"version": "16.3.0-beta1",
"description": "A Typescript library to help you get an API server up and running with I/O schema validation and custom middlewares in minutes.",
"license": "MIT",
"repository": {
Expand Down
10 changes: 5 additions & 5 deletions src/common-helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ import { xprod } from "ramda";
import { z } from "zod";
import { CommonConfig, InputSource, InputSources } from "./config-type";
import { InputValidationError, OutputValidationError } from "./errors";
import { ZodFile } from "./file-schema";
import { IOSchema } from "./io-schema";
import { AbstractLogger } from "./logger";
import { getMeta } from "./metadata";
import { getMeta, isProprietary } from "./metadata";
import { AuxMethod, Method } from "./method";
import { mimeMultipart } from "./mime";
import { ZodUpload } from "./upload-schema";
import { ezRawKind } from "./raw-schema";
import { ezUploadKind } from "./upload-schema";

export type FlatObject = Record<string, unknown>;

Expand Down Expand Up @@ -237,13 +237,13 @@ export const hasNestedSchema = ({
export const hasUpload = (subject: IOSchema) =>
hasNestedSchema({
subject,
condition: (schema) => schema instanceof ZodUpload,
condition: (schema) => isProprietary(schema, ezUploadKind),
});

export const hasRaw = (subject: IOSchema) =>
hasNestedSchema({
subject,
condition: (schema) => schema instanceof ZodFile,
condition: (schema) => isProprietary(schema, ezRawKind),
maxDepth: 3,
});

Expand Down
79 changes: 15 additions & 64 deletions src/date-in-schema.ts
Original file line number Diff line number Diff line change
@@ -1,64 +1,15 @@
import {
INVALID,
ParseInput,
ParseReturnType,
ZodIssueCode,
ZodParsedType,
ZodType,
ZodTypeDef,
addIssueToContext,
} from "zod";
import { isValidDate } from "./schema-helpers";

// simple regex for ISO date, supports the following formats:
// 2021-01-01T00:00:00.000Z
// 2021-01-01T00:00:00.0Z
// 2021-01-01T00:00:00Z
// 2021-01-01T00:00:00
// 2021-01-01
export const isoDateRegex =
/^\d{4}-\d{2}-\d{2}(T\d{2}:\d{2}:\d{2}(\.\d+)?)?Z?$/;

const zodDateInKind = "ZodDateIn";

export interface ZodDateInDef extends ZodTypeDef {
typeName: typeof zodDateInKind;
}

export class ZodDateIn extends ZodType<Date, ZodDateInDef, string> {
_parse(input: ParseInput): ParseReturnType<Date> {
const { status, ctx } = this._processInputParams(input);
if (ctx.parsedType !== ZodParsedType.string) {
addIssueToContext(ctx, {
code: ZodIssueCode.invalid_type,
expected: ZodParsedType.string,
received: ctx.parsedType,
});
return INVALID;
}

if (!isoDateRegex.test(ctx.data as string)) {
addIssueToContext(ctx, {
code: ZodIssueCode.invalid_string,
validation: "regex",
});
status.dirty();
}

const date = new Date(ctx.data);

if (!isValidDate(date)) {
addIssueToContext(ctx, {
code: ZodIssueCode.invalid_date,
});
return INVALID;
}

return { status: status.value, value: date };
}

static create = () =>
new ZodDateIn({
typeName: zodDateInKind,
});
}
import { z } from "zod";
import { proprietary } from "./metadata";
import { isValidDate, isoDateRegex } from "./schema-helpers";

export const ezDateInKind = "DateIn";

export const dateIn = () =>
proprietary(
ezDateInKind,
z
.string()
.regex(isoDateRegex)
.transform((str) => new Date(str))
.pipe(z.date().refine(isValidDate)),
);
53 changes: 11 additions & 42 deletions src/date-out-schema.ts
Original file line number Diff line number Diff line change
@@ -1,45 +1,14 @@
import {
INVALID,
ParseInput,
ParseReturnType,
ZodIssueCode,
ZodParsedType,
ZodType,
ZodTypeDef,
addIssueToContext,
} from "zod";
import { z } from "zod";
import { proprietary } from "./metadata";
import { isValidDate } from "./schema-helpers";

const zodDateOutKind = "ZodDateOut";
export const ezDateOutKind = "DateOut";

export interface ZodDateOutDef extends ZodTypeDef {
typeName: typeof zodDateOutKind;
}

export class ZodDateOut extends ZodType<string, ZodDateOutDef, Date> {
_parse(input: ParseInput): ParseReturnType<string> {
const { status, ctx } = this._processInputParams(input);
if (ctx.parsedType !== ZodParsedType.date) {
addIssueToContext(ctx, {
code: ZodIssueCode.invalid_type,
expected: ZodParsedType.date,
received: ctx.parsedType,
});
return INVALID;
}

if (!isValidDate(ctx.data)) {
addIssueToContext(ctx, {
code: ZodIssueCode.invalid_date,
});
return INVALID;
}

return { status: status.value, value: (ctx.data as Date).toISOString() };
}

static create = () =>
new ZodDateOut({
typeName: zodDateOutKind,
});
}
export const dateOut = () =>
proprietary(
ezDateOutKind,
z
.date()
.refine(isValidDate)
.transform((date) => date.toISOString()),
);
42 changes: 25 additions & 17 deletions src/documentation-helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,17 @@ import {
FlatObject,
getExamples,
hasCoercion,
hasRaw,
hasTopLevelTransformingEffect,
isCustomHeader,
makeCleanId,
tryToTransform,
ucFirst,
} from "./common-helpers";
import { InputSource, TagsConfig } from "./config-type";
import { ZodDateIn, isoDateRegex } from "./date-in-schema";
import { ZodDateOut } from "./date-out-schema";
import { ezDateInKind } from "./date-in-schema";
import { ezDateOutKind } from "./date-out-schema";
import { DocumentationError } from "./errors";
import { ZodFile } from "./file-schema";
import { ezFileKind } from "./file-schema";
import { IOSchema } from "./io-schema";
import {
LogicalContainer,
Expand All @@ -42,14 +41,16 @@ import {
} from "./logical-container";
import { copyMeta } from "./metadata";
import { Method } from "./method";
import { RawSchema, ezRawKind } from "./raw-schema";
import { isoDateRegex } from "./schema-helpers";
import {
HandlingRules,
HandlingVariant,
SchemaHandler,
walkSchema,
} from "./schema-walker";
import { Security } from "./security";
import { ZodUpload } from "./upload-schema";
import { ezUploadKind } from "./upload-schema";

/* eslint-disable @typescript-eslint/no-use-before-define */

Expand Down Expand Up @@ -130,7 +131,7 @@ export const depictAny: Depicter<z.ZodAny> = () => ({
format: "any",
});

export const depictUpload: Depicter<ZodUpload> = (ctx) => {
export const depictUpload: Depicter<z.ZodType> = (ctx) => {
assert(
!ctx.isResponse,
new DocumentationError({
Expand All @@ -144,11 +145,14 @@ export const depictUpload: Depicter<ZodUpload> = (ctx) => {
};
};

export const depictFile: Depicter<ZodFile> = ({
schema: { isBinary, isBase64, isBuffer },
}) => ({
export const depictFile: Depicter<z.ZodType> = ({ schema }) => ({
type: "string",
format: isBuffer || isBinary ? "binary" : isBase64 ? "byte" : "file",
format:
schema instanceof z.ZodString
? schema._def.checks.find((check) => check.kind === "regex")
? "byte"
: "file"
: "binary",
});

export const depictUnion: Depicter<z.ZodUnion<z.ZodUnionOptions>> = ({
Expand Down Expand Up @@ -245,7 +249,7 @@ export const depictObject: Depicter<z.AnyZodObject> = ({
* */
export const depictNull: Depicter<z.ZodNull> = () => ({ type: "null" });

export const depictDateIn: Depicter<ZodDateIn> = (ctx) => {
export const depictDateIn: Depicter<z.ZodType> = (ctx) => {
assert(
!ctx.isResponse,
new DocumentationError({
Expand All @@ -264,7 +268,7 @@ export const depictDateIn: Depicter<ZodDateIn> = (ctx) => {
};
};

export const depictDateOut: Depicter<ZodDateOut> = (ctx) => {
export const depictDateOut: Depicter<z.ZodType> = (ctx) => {
assert(
ctx.isResponse,
new DocumentationError({
Expand Down Expand Up @@ -587,6 +591,9 @@ export const depictLazy: Depicter<z.ZodLazy<z.ZodTypeAny>> = ({
);
};

export const depictRaw: Depicter<RawSchema> = ({ next, schema }) =>
next({ schema: schema.shape.raw });

export const depictExamples = (
schema: z.ZodTypeAny,
isResponse: boolean,
Expand Down Expand Up @@ -744,8 +751,6 @@ export const depicters: HandlingRules<
ZodNumber: depictNumber,
ZodBigInt: depictBigInt,
ZodBoolean: depictBoolean,
ZodDateIn: depictDateIn,
ZodDateOut: depictDateOut,
ZodNull: depictNull,
ZodArray: depictArray,
ZodTuple: depictTuple,
Expand All @@ -754,8 +759,6 @@ export const depicters: HandlingRules<
ZodLiteral: depictLiteral,
ZodIntersection: depictIntersection,
ZodUnion: depictUnion,
ZodFile: depictFile,
ZodUpload: depictUpload,
ZodAny: depictAny,
ZodDefault: depictDefault,
ZodEnum: depictEnum,
Expand All @@ -770,6 +773,11 @@ export const depicters: HandlingRules<
ZodPipeline: depictPipeline,
ZodLazy: depictLazy,
ZodReadonly: depictReadonly,
[ezFileKind]: depictFile,
[ezUploadKind]: depictUpload,
[ezDateOutKind]: depictDateOut,
[ezDateInKind]: depictDateIn,
[ezRawKind]: depictRaw,
};

export const onEach: Depicter<z.ZodTypeAny, "each"> = ({
Expand Down Expand Up @@ -1048,7 +1056,7 @@ export const depictRequest = ({
const bodyDepiction = excludeExamplesFromDepiction(
excludeParamsFromDepiction(
walkSchema({
schema: hasRaw(schema) ? ZodFile.create().buffer() : schema,
schema,
isResponse: false,
rules: depicters,
onEach,
Expand Down
Loading