Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to save glTF as msgpack format (or BSON) from the commandline and avoid crashing from JSON.stringify(this.manifest.. #55

Closed
wallabyway opened this issue Mar 4, 2022 · 2 comments

Comments

@wallabyway
Copy link
Collaborator

wallabyway commented Mar 4, 2022

in the src/common/gltf/writer.ts file, this line 119...

    fse.writeFileSync(gltfPath, JSON.stringify(this.manifest, null, 4));

... will crash JSON.stringify, because the this.manifest object is too large to serialize into a string (nodejs 8GB option).

Instead, allow the writer to generate a msgpack (or BSON) output file, without invoking the JSON.stringify().

The current branch experiment/BSON uses a postProcess, but still executes JSON.stringify, and hence still crashes.

In my limited testing, exporting to msgpack does the following:

  1. avoids crashing
  2. the .msgpack file is significantly smaller than the json version
  3. gzipping the .msgpack is slightly smaller than the gzipped json file
  4. generating a msgpack output file, is significantly faster than outputting a json file
  5. decoding a msgpack gltf file, is about the same speed as a json gltf decoder.

note: I used the faster...
const msgpack = require('msgpackr'); //https://github.com/kriszyp/msgpackr
and not
const msgpack = require('msgpack');
https://github.com/petrbroz/forge-convert-utils/blob/experiment/bson/samples/output-bson-msgpack.js

@petrbroz
Copy link
Owner

In b3397eb I've updated the GltfWriter class so that you can override the actual serialization process. And the https://github.com/petrbroz/forge-convert-utils/blob/develop/samples/serialize-msgpack.js sample shows how you can pack the in-memory glTF manifest directly to msgpack.

@wallabyway
Copy link
Collaborator Author

related to this.. KhronosGroup/glTF#1699

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants