feat(martin-core): Allow glyph ranges more than 0xFFFF

[yutannihilation]

Fix #2366

This pull request simply increases MAX_UNICODE_CP to 0x10FFFF
Resolves #2366.

The below is my rough understanding after reading https://en.wikipedia.org/wiki/Unicode_block. I think it's appropriate to include all of these.

• U+0000 - U+FFFF is Basic Multilingual Plane, which covers characters for almost all modern languages
• U+10000 - U+3347F covers minor characters such as historic scripts and emojis
• U+E0000 - U+E01EF is for tags and variation selectors
• U+F0000 - U+10FFFF is for private use (i.e. can be assigned arbitrary custom characters without worrying about possible conflict with the future standards)

The font files for testing contain only one glyph that the file name indicates. These are generated by https://github.com/yutannihilation/dummyfont, which I implemented with Codex. I'm not sure all the metadata is valid, but I believe this is good enough for testing purposes.

[Copilot]

Pull request overview

This PR extends Unicode glyph support from the Basic Multilingual Plane (U+0000 - U+FFFF) to the full Unicode range (U+0000 - U+10FFFF), enabling Martin to handle characters from supplementary planes including historic scripts, tags/variation selectors, and private use areas.

Key Changes:

• Increased MAX_UNICODE_CP constant from 0xFFFF to 0x10FFFF
• Added unit test to verify font processing for codepoints beyond the BMP
• Added test font files containing glyphs for U+3320 and U+1F60A

Reviewed changes

Copilot reviewed 1 out of 3 changed files in this pull request and generated 1 comment.

File	Description
martin-core/src/resources/fonts/mod.rs	Updated MAX_UNICODE_CP constant to support full Unicode range up to U+10FFFF, improved documentation explaining the rationale, and added test for extended Unicode support
martin-core/src/resources/fonts/tests/u+3320.ttf	Test font file containing a single glyph at U+3320 for validating support beyond the BMP
martin-core/src/resources/fonts/tests/u+1f60a.ttf	Test font file containing a single glyph at U+1F60A (emoji range) for validating extended Unicode plane support

[CommanderStorm]

Nice work, thank you ❤️

I am going to bless the test output and this will be merged afterwards.

[yutannihilation]

Thanks for handling the test cases! It seems deploying lambda fails with this error:

Error: Container invocation failed due to maximum memory usage

I suspected 0x10FFFF might be too big for some value, but I don't find yet where it matters in actual...

[CommanderStorm]

I think that the extra memory should be tiny.
A bit sus, but likly unrelated to your PR.

Let me try a few things

[nyurik]

i fixed this by adding MemorySize param. Note that I had to go from the default 128MB to 350MB - it was failing without that.

  MartinFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Runtime: provided.al2023
      Layers:
        - Ref: MartinLayer
      CodeUri: lambda-function/
      Handler: config.yaml
+     MemorySize: 350

[yutannihilation]

Thanks! So, this actually increased memory usage a bit. Is it acceptable to you?

[nyurik]

@yutannihilation I don't think the code increased the memory - the font file did. In other words, each user of Martin on an AWS lambda has to configure how much RAM to allocate to that service, depending on their specific use case. 128MB is a very small (by today's standards) amount of RAM for a service, so if it uses a large font file, it simply won't fit within the allowed space.

[yutannihilation]

I see. Thanks for the explanation!