Crash when signing large payloads

[ebourg]

Using the firmware 1.2.2-alpha.20221130, signing a too large payload crashes the Nitrokey (the LED turns red, the key is unresponsive and must be unplugged). Here is an example with a RSA 2048 key:

CommmandAPDU: 11 bytes, nc=6, ne=0
00000000 00 20 00 81 06 31 32 33 34 35 36                . ...123456

ResponseAPDU: 2 bytes, SW=9000 (50 ms)

CommmandAPDU: 263 bytes, nc=256, ne=0
00000000 00 2A 9E 9A 00 01 00 00 00 00 00 00 00 00 00 00 .*..............
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000100 00 00 00 00 00 00 00                            .......

I guess the size of the payload should be checked, and an error 0x6700 (Wrong Length) returned if it's too big.

[sosthene-nitrokey]

Thank you for the report.

This was caused by a bug in the RSA integration in Trussed for which a fix is ready: Nitrokey/trussed#11

The issue was on the contrary that the length was checked, but the error was not handled, which translated to a Rust panic, aka red LED + the key stops working.

Future releases will actually use a different approach to RSA integration in Trussed that also contains the fix: #116

[sosthene-nitrokey]

This was fixed by #116 which is now included in the 1.4.0 Nitrokey 3 stable firmware.