Support Unicode escape sequences

[hollasch]

Model after C++:

• \unnnn — code point U+nnnn (4 hexadecimal digits)
• \u{nnnn} — code point U+n... (arbitrary number of hexadecimal digits)
• \Unnnnnnnn — code point U+nnnnnnnn (8 hexadecimal digits)

[hollasch]

Well duh — you can't use backslash escapes, as that would collide with Windows style directory separators. Instead, let's go with this:

• %uXXXX — XXXX is four hex digits. Unicode code point in the BMP: 0x0000–0xD7FF, 0xE000–0xFFFF
• %UXXXXXXXX – XXXXXXXX is eight hex digits. Unicode code points above the BMP: 0x10000– 0x10FFFF.

[hollasch]

Of course, with the % escape sequence, we need %% to represent a literal %.

[hollasch]

Could also use #, as in foo#u0278bar versus foo%u0278bar.

Just thinking out loud, we could also use the special character as a symmetric delimiter. For example: foo%278%bar. Or could specify a special character as the ending delimiter, like foo%278)bar, or foo#278;bar. In these cases, a doubled beginning delimiter represents the literal value, so %% denotes %, or ## denotes #, respectively for the two prior examples.

Rats — using ; or ) (for the end delimiter) is problematic on Windows. You'd have to escape these on the command line. I'd rather avoid having to escape special characters at the command line, as that gets you into double-escaping for scripts and such.

[hollasch]

All right. I think the best option for now is to with #AB…C# to denote Unicode codepoint AB…C, and ## to denote a literal #.