Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a freeze: unpacker option #194

Merged
merged 1 commit into from
Sep 17, 2020

Conversation

casperisfine
Copy link

As discussed in #190

If set to true all parsed objects will be immediately frozen, and strings will be deduplicated if the Ruby implementation allows it.

This feature is meant to be similar to Pysch's freeze: option: ruby/psych#414

For now this option is only useful if you plan to keep the parsed objects around, and want to reduce their memory footprint.

But hopefully Ruby 3.0 should soon expose rb_fstring_new(const char *ptr, long len) (likely under another name) which should allow to reduce allocations by directly looking up existing strings without allocating a new one.

cc @tagomoris

NB: I presume that a java implementation is needed, I'm not too well versed into Java by I can try my hand at it. However I'd rather have your feedback on the MRI implementation first.

@casperisfine
Copy link
Author

I went ahead and implemented the jruby version.

Copy link
Member

@tagomoris tagomoris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most things look good. Great!
I left a minor comment about how to get frozen strings in JRuby. Could you check that?

return runtime.newString(byteList);
RubyString string = runtime.newString(byteList);
if (this.freeze) {
string.setFrozen(true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruby#freezeAndDedupString looks to freeze&dedup the string, not only dedup.
Is there any reason to call setFrozen(true) explicitly here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, today it's a bit useless, however in theory if you call it with a frozen string, it could directly intern that string without duplicating it first. That's what MRI 3.0 now does, see: ruby/spec#788

I do hope to change that behavior in JRuby at some point, so I figured we might as well call it with a frozen string directly.

@tagomoris
Copy link
Member

Plz ping me when you solve the problem (and failing CI).

If set to true all parsed objects will be
immediately frozen, and strings will be
deduplicated if the Ruby implementation
allows it.
@casperisfine
Copy link
Author

Plz ping me when you solve the problem (and failing CI).

It should be good now. CI is green. Sorry for the last change.

@tagomoris
Copy link
Member

I'll merge this change tomorrow if there's no further commits/comments.
The release will be issued after merging #193

@casperisfine
Copy link
Author

Works for me, thanks!

@tagomoris tagomoris merged commit eeff7ad into msgpack:master Sep 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants