Skip to content

feat: allow serialization/deserialization of custom data types (alternative API) #13149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9c5aa14
feat: allow serialization/deserialization of custom data types
trueadm Dec 9, 2024
d20bf5d
feat: allow serialization/deserialization of custom data types
trueadm Dec 9, 2024
71abcac
feat: allow serialization/deserialization of custom data types
trueadm Dec 9, 2024
c74e6af
feat: allow serialization/deserialization of custom data types
trueadm Dec 9, 2024
5a08ec0
feat: allow serialization/deserialization of custom data types
trueadm Dec 9, 2024
1e78784
add test
trueadm Dec 9, 2024
c3eea41
fix bugs
trueadm Dec 9, 2024
79617ca
lint
trueadm Dec 9, 2024
6c31b80
improve test name
trueadm Dec 9, 2024
3b4fa6d
lint
trueadm Dec 9, 2024
97e1bb2
alternative approach
trueadm Dec 9, 2024
65324a6
tweak
trueadm Dec 9, 2024
64db6fe
added more tests and moved to basics
trueadm Dec 9, 2024
c478485
lint
trueadm Dec 9, 2024
005222c
fix typo
trueadm Dec 9, 2024
755886c
fix test
trueadm Dec 9, 2024
750d3b7
address feedback
trueadm Dec 9, 2024
bae67b0
comment
trueadm Dec 9, 2024
1f7b31a
Merge branch 'main' into serialize-deserialize-non-pojo
dummdidumm Dec 10, 2024
8389b7b
make it work
Rich-Harris Dec 11, 2024
66e6b2e
Update packages/kit/src/core/sync/write_client_manifest.js
trueadm Dec 11, 2024
40e9e05
use universal transport hook
Rich-Harris Dec 11, 2024
627b9e2
fix
Rich-Harris Dec 11, 2024
8dbd4d8
Update packages/kit/src/core/sync/write_client_manifest.js
Rich-Harris Dec 11, 2024
8db3f33
tweaks
Rich-Harris Dec 11, 2024
5d1c4f3
Merge branch 'serialize-deserialize-non-pojo-universal' of github.com…
Rich-Harris Dec 11, 2024
8b4381a
add types, rename to encode/decode
Rich-Harris Dec 11, 2024
4fff30f
docs
Rich-Harris Dec 11, 2024
177d2b9
regenerate
Rich-Harris Dec 11, 2024
6921408
changeset
Rich-Harris Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fast-dragons-own.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@sveltejs/kit': minor
---

feat: transport custom types across the server/client boundary
17 changes: 17 additions & 0 deletions documentation/docs/30-advanced/20-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,23 @@ The `lang` parameter will be correctly derived from the returned pathname.

Using `reroute` will _not_ change the contents of the browser's address bar, or the value of `event.url`.

### transport

This is a collection of _transporters_, which allow you to pass custom types — returned from `load` and form actions — across the server/client boundary. Each transporter contains an `encode` function, which encodes values on the server (or returns `false` for anything that isn't an instance of the type) and a corresponding `decode` function:

```js
/// file: src/hooks.js
import { Vector } from '$lib/math';

/** @type {import('@sveltejs/kit').Transport} */
export const transport = {
Vector: {
encode: (value) => value instanceof Vector && [value.x, value.y],
decode: ([x, y]) => new Vector(x, y)
}
};
```


## Further reading

Expand Down
8 changes: 6 additions & 2 deletions packages/kit/src/core/sync/write_client_manifest.js
Original file line number Diff line number Diff line change
Expand Up @@ -152,10 +152,14 @@ export function write_client_manifest(kit, manifest_data, output, metadata) {
client_hooks_file ? 'client_hooks.handleError || ' : ''
}(({ error }) => { console.error(error) }),
${client_hooks_file ? 'init: client_hooks.init,' : ''}

reroute: ${universal_hooks_file ? 'universal_hooks.reroute || ' : ''}(() => {})
reroute: ${universal_hooks_file ? 'universal_hooks.reroute || ' : ''}(() => {}),
transport: ${universal_hooks_file ? 'universal_hooks.transport || ' : ''}{}
};

export const decoders = Object.fromEntries(Object.entries(hooks.transport).map(([k, v]) => [k, v.decode]));

export const decode = (type, value) => decoders[type](value);

export { default as root } from '../root.${isSvelte5Plus() ? 'js' : 'svelte'}';
`
);
Expand Down
6 changes: 4 additions & 2 deletions packages/kit/src/core/sync/write_server.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,14 +71,16 @@ export async function get_hooks() {
${server_hooks ? `({ handle, handleFetch, handleError, init } = await import(${s(server_hooks)}));` : ''}

let reroute;
${universal_hooks ? `({ reroute } = await import(${s(universal_hooks)}));` : ''}
let transport;
${universal_hooks ? `({ reroute, transport } = await import(${s(universal_hooks)}));` : ''}

return {
handle,
handleFetch,
handleError,
reroute,
init,
reroute,
transport
};
}

Expand Down
37 changes: 37 additions & 0 deletions packages/kit/src/exports/public.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -742,6 +742,43 @@ export type ClientInit = () => MaybePromise<void>;
*/
export type Reroute = (event: { url: URL }) => void | string;

/**
* The [`transport`](https://svelte.dev/docs/kit/hooks#Universal-hooks-transport) hook allows you to transport custom types across the server/client boundary.
*
* Each transporter has a pair of `encode` and `decode` functions. On the server, `encode` determines whether a value is an instance of the custom type and, if so, returns a non-falsy encoding of the value which can be an object or an array (or `false` otherwise).
*
* In the browser, `decode` turns the encoding back into an instance of the custom type.
*
* ```ts
* import type { Transport } from '@sveltejs/kit';
*
* declare class MyCustomType {
* data: any
* }
*
* // hooks.js
* export const transport: Transport = {
* MyCustomType: {
* encode: (value) => value instanceof MyCustomType && [value.data],
* decode: ([data]) => new MyCustomType(data)
* }
* };
* ```
* @since 2.11.0
*/
export type Transport = Record<string, Transporter>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much use will people really make of this specific type, since it's not helping much with type safety? Should we just omit it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should at least offer the defineTransport function for type safety. We can call it something else if we don't like it but it would be nice

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's helping plenty — it might not be able to provide inference for the decode parameter, but it enforces the correct shape of transport (i.e. you can't omit either encode or decode for a given transporter, and you can't misspell them). And exposing Transporter means you can add the missing type safety yourself:

export const transport: Transport = {
  Foo: {
    encode: (value) => ...,
    decode: (data) => ...
  } satisfies Transporter<Foo, [blah]>
};

We could always add defineTransport later if we feel like we need to, but I don't think we should break with existing conventions just yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dummdidumm can we add this to 0 effort typesafe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and no - we can add satisfies Transport for the top level, but we can't do much for the individual entries


/**
* A member of the [`transport`](https://svelte.dev/docs/kit/hooks#Universal-hooks-transport) hook.
*/
export interface Transporter<
T = any,
U = Exclude<any, false | 0 | '' | null | undefined | typeof NaN>
> {
encode: (value: T) => false | U;
decode: (data: U) => T;
}

/**
* The generic form of `PageLoad` and `LayoutLoad`. You should import those from `./$types` (see [generated types](https://svelte.dev/docs/kit/types#Generated-types))
* rather than using `Load` directly.
Expand Down
6 changes: 4 additions & 2 deletions packages/kit/src/runtime/app/forms.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import * as devalue from 'devalue';
import { DEV } from 'esm-env';
import { invalidateAll } from './navigation.js';
import { applyAction } from '../client/client.js';
import { app, applyAction } from '../client/client.js';

export { applyAction };

Expand Down Expand Up @@ -29,9 +29,11 @@ export { applyAction };
*/
export function deserialize(result) {
const parsed = JSON.parse(result);

if (parsed.data) {
parsed.data = devalue.parse(parsed.data);
parsed.data = devalue.parse(parsed.data, app.decoders);
}

return parsed;
}

Expand Down
3 changes: 2 additions & 1 deletion packages/kit/src/runtime/client/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ let container;
/** @type {HTMLElement} */
let target;
/** @type {import('./types.js').SvelteKitApp} */
let app;
export let app;

/** @type {Array<((url: URL) => boolean)>} */
const invalidated = [];
Expand Down Expand Up @@ -2493,6 +2493,7 @@ async function load_data(url, invalid) {
*/
function deserialize(data) {
return devalue.unflatten(data, {
...app.decoders,
Promise: (id) => {
return new Promise((fulfil, reject) => {
deferreds.set(id, { fulfil, reject });
Expand Down
6 changes: 5 additions & 1 deletion packages/kit/src/runtime/client/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ export interface SvelteKitApp {

hooks: ClientHooks;

decode: (type: string, value: any) => any;

decoders: Record<string, (data: any) => any>;

root: typeof SvelteComponent;
}

Expand Down Expand Up @@ -54,7 +58,7 @@ export type NavigationFinished = {
state: NavigationState;
props: {
constructors: Array<typeof SvelteComponent>;
components?: Array<SvelteComponent>;
components?: SvelteComponent[];
page: Page;
form?: Record<string, any> | null;
[key: `data_${number}`]: Record<string, any>;
Expand Down
3 changes: 3 additions & 0 deletions packages/kit/src/runtime/server/data/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,9 @@ export function get_data_json(event, options, nodes) {
const { iterator, push, done } = create_async_iterator();

const reducers = {
...Object.fromEntries(
Object.entries(options.hooks.transport).map(([key, value]) => [key, value.encode])
),
/** @param {any} thing */
Promise: (thing) => {
if (typeof thing?.then === 'function') {
Expand Down
6 changes: 4 additions & 2 deletions packages/kit/src/runtime/server/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ export class Server {
handle: module.handle || (({ event, resolve }) => resolve(event)),
handleError: module.handleError || (({ error }) => console.error(error)),
handleFetch: module.handleFetch || (({ request, fetch }) => fetch(request)),
reroute: module.reroute || (() => {})
reroute: module.reroute || (() => {}),
transport: module.transport || {}
};

if (module.init) {
Expand All @@ -90,7 +91,8 @@ export class Server {
},
handleError: ({ error }) => console.error(error),
handleFetch: ({ request, fetch }) => fetch(request),
reroute: () => {}
reroute: () => {},
transport: {}
};
} else {
throw error;
Expand Down
37 changes: 30 additions & 7 deletions packages/kit/src/runtime/server/page/actions.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,22 @@ export async function handle_action_json_request(event, options, server) {
// @ts-expect-error we assign a string to what is supposed to be an object. That's ok
// because we don't use the object outside, and this way we have better code navigation
// through knowing where the related interface is used.
data: stringify_action_response(data.data, /** @type {string} */ (event.route.id))
data: stringify_action_response(
data.data,
/** @type {string} */ (event.route.id),
options.hooks.transport
)
});
} else {
return action_json({
type: 'success',
status: data ? 200 : 204,
// @ts-expect-error see comment above
data: stringify_action_response(data, /** @type {string} */ (event.route.id))
data: stringify_action_response(
data,
/** @type {string} */ (event.route.id),
options.hooks.transport
)
});
}
} catch (e) {
Expand Down Expand Up @@ -254,26 +262,41 @@ function validate_action_return(data) {
* Try to `devalue.uneval` the data object, and if it fails, return a proper Error with context
* @param {any} data
* @param {string} route_id
* @param {import('types').ServerHooks['transport']} transport
*/
export function uneval_action_response(data, route_id) {
return try_deserialize(data, devalue.uneval, route_id);
export function uneval_action_response(data, route_id, transport) {
const replacer = (/** @type {any} */ thing) => {
for (const key in transport) {
const encoded = transport[key].encode(thing);
if (encoded) {
return `app.decode('${key}', ${devalue.uneval(encoded, replacer)})`;
}
}
};

return try_serialize(data, (value) => devalue.uneval(value, replacer), route_id);
}

/**
* Try to `devalue.stringify` the data object, and if it fails, return a proper Error with context
* @param {any} data
* @param {string} route_id
* @param {import('types').ServerHooks['transport']} transport
*/
function stringify_action_response(data, route_id) {
return try_deserialize(data, devalue.stringify, route_id);
function stringify_action_response(data, route_id, transport) {
const encoders = Object.fromEntries(
Object.entries(transport).map(([key, value]) => [key, value.encode])
);

return try_serialize(data, (value) => devalue.stringify(value, encoders), route_id);
}

/**
* @param {any} data
* @param {(data: any) => string} fn
* @param {string} route_id
*/
function try_deserialize(data, fn, route_id) {
function try_serialize(data, fn, route_id) {
try {
return fn(data);
} catch (e) {
Expand Down
32 changes: 23 additions & 9 deletions packages/kit/src/runtime/server/page/render.js
Original file line number Diff line number Diff line change
Expand Up @@ -321,12 +321,20 @@ export async function render_response({
deferred.set(id, { fulfil, reject });
})`);

// When resolving, the id might not yet be available due to the data
// be evaluated upon init of kit, so we use a timeout to retry
properties.push(`resolve: ({ id, data, error }) => {
const { fulfil, reject } = deferred.get(id);
deferred.delete(id);

if (error) reject(error);
else fulfil(data);
const try_to_resolve = () => {
if (!deferred.has(id)) {
setTimeout(try_to_resolve, 0);
return;
}
const { fulfil, reject } = deferred.get(id);
deferred.delete(id);
if (error) reject(error);
else fulfil(data);
}
try_to_resolve();
}`);
}

Expand All @@ -342,12 +350,11 @@ export async function render_response({
if (page_config.ssr) {
const serialized = { form: 'null', error: 'null' };

blocks.push(`const data = ${data};`);

if (form_value) {
serialized.form = uneval_action_response(
form_value,
/** @type {string} */ (event.route.id)
/** @type {string} */ (event.route.id),
options.hooks.transport
);
}

Expand All @@ -357,7 +364,7 @@ export async function render_response({

const hydrate = [
`node_ids: [${branch.map(({ node }) => node.index).join(', ')}]`,
'data',
`data: ${data}`,
`form: ${serialized.form}`,
`error: ${serialized.error}`
];
Expand Down Expand Up @@ -573,6 +580,13 @@ function get_data(event, options, nodes, csp, global) {
);

return `${global}.defer(${id})`;
} else {
for (const key in options.hooks.transport) {
const encoded = options.hooks.transport[key].encode(thing);
if (encoded) {
return `app.decode('${key}', ${devalue.uneval(encoded, replacer)})`;
}
}
}
}

Expand Down
5 changes: 4 additions & 1 deletion packages/kit/src/types/internal.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ import {
Emulator,
Adapter,
ServerInit,
ClientInit
ClientInit,
Transporter
} from '@sveltejs/kit';
import {
HttpMethod,
Expand Down Expand Up @@ -111,12 +112,14 @@ export interface ServerHooks {
handle: Handle;
handleError: HandleServerError;
reroute: Reroute;
transport: Record<string, Transporter>;
init?: ServerInit;
}

export interface ClientHooks {
handleError: HandleClientError;
reroute: Reroute;
transport: Record<string, Transporter>;
init?: ClientInit;
}

Expand Down
9 changes: 9 additions & 0 deletions packages/kit/test/apps/basics/src/hooks.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { browser } from '$app/environment';
import { Foo } from './lib';

const mapping = {
'/reroute/basic/a': '/reroute/basic/b',
Expand Down Expand Up @@ -29,3 +30,11 @@ export const reroute = ({ url }) => {
return mapping[url.pathname];
}
};

/** @type {import("@sveltejs/kit").Transport} */
export const transport = {
Foo: {
encode: (value) => value instanceof Foo && [value.message],
decode: ([message]) => new Foo(message)
}
};
9 changes: 9 additions & 0 deletions packages/kit/test/apps/basics/src/lib/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
export class Foo {
constructor(message) {
this.message = message;
}

bar() {
return this.message + '!';
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { Foo } from '../../lib';

export function load() {
return { foo: new Foo('It works') };
}
Loading
Loading