HJson fails to quote control character with \uXXXX in String

[treimers]

We are using HJson to parse regular Json files and found an issue with control characters in Strings.

The input file looks like

{ "aaa" : "xxx\u000byyy" }

and we parse it with HJson. When the resulting JsonValue object is written using toString() or toString(Stringify.FORMATTED) the output contains invalid control characters. It is expected that these control characters are quoted using \u again according to Json standard.

This example program shows the problem:

package de.softquadrat.jdbc.json;

import java.nio.charset.StandardCharsets;

import org.hjson.JsonValue;
import org.hjson.Stringify;

public class Test {
	public static void main(String[] args) {
		JsonValue hjson = JsonValue.readHjson("{ \"aaa\" : \"xxx\\u000byyy\" }");
		String s = hjson.toString(Stringify.FORMATTED);
		System.out.println(hexdump(s));
		JsonValue.readHjson(s);
	}

	// from https://gist.github.com/geekman/5243537
	public static String hexdump(String s) {
		byte[] data = s.getBytes(StandardCharsets.UTF_8);
		final int perRow = 16;
		final String hexChars = "0123456789ABCDEF";
		StringBuilder dump = new StringBuilder();
		StringBuilder chars = null;
		for (int i = 0; i < data.length; i++) {
			int offset = i % perRow;
			if (offset == 0) {
				chars = new StringBuilder();
				dump.append(String.format("%04x", i))
					.append("  ");
			}
			int b = data[i] & 0xFF;
			dump.append(hexChars.charAt(b >>> 4))
				.append(hexChars.charAt(b & 0xF))
				.append(' ');
			chars.append((char) ((b >= ' ' && b <= '~') ? b : '.'));
			if (i == data.length - 1 || offset == perRow - 1) {
				for (int j = perRow - offset - 1; j > 0; j--)
					dump.append("-- ");
				dump.append("  ")
					.append(chars)
					.append('\n');
			}
		}
		return dump.toString();
	}
}

The output of the test program gives:

0000  0A 7B 0A 20 20 22 61 61 61 22 3A 20 22 78 78 78   .{.  "aaa": "xxx
0010  0B 79 79 79 22 0A 7D -- -- -- -- -- -- -- -- --   .yyy".}

Exception in thread "main" org.hjson.ParseException: Expected valid string character at 3:13
        at org.hjson.HjsonParser.error(HjsonParser.java:500)
        at org.hjson.HjsonParser.expected(HjsonParser.java:494)
        at org.hjson.HjsonParser.readStringInternal(HjsonParser.java:292)
        at org.hjson.HjsonParser.readString(HjsonParser.java:282)
        at org.hjson.HjsonParser.readValue(HjsonParser.java:117)
        at org.hjson.HjsonParser.readObject(HjsonParser.java:199)
        at org.hjson.HjsonParser.readValue(HjsonParser.java:119)
        at org.hjson.HjsonParser.parse(HjsonParser.java:88)
        at org.hjson.JsonValue.readHjson(JsonValue.java:130)
        at de.softquadrat.jdbc.json.Test.main(Test.java:13)

The 0B character (Ctrl-K) in the generated Json is invalid. It should be quoted as \u000b. HJson is unable to parse the Json generated by itself and throws the ParseException.

[trobro]

Thanks for the detailed issue description, I'll have a look. The issue seems to be specific for the Java implementation of Hjson so I transferred the issue to the hjson-java repository.

[treimers]

Upps, thanks and sorry for the wrong repo.

[trobro]

Fix included in the v3.1.0 release.

[treimers]

Thank you very much, the fix works!