GPG Card error after booting with full-disk encryption

[pfactum]

Environment:

1. Arch Linux
2. latest version of available packages: gpg 2.2.41, kwallet 5.106.0, kwalletmanager 23.04.1 at this very moment
3. full disk encryption with unlocking using FIDO token, the token was enrolled using systemd-cryptenroll
4. KDE wallet encrypted using GPG keys
5. GPG keys reside on Nitrokey 3A NFC
6. FW version is 1.5.0, but the same was observed with 1.4.0 as well

Sequence of events:

1. insert token, start booting
2. during the initramfs phase PIN is asked; enter PIN, touch the token, disk gets unlocked, system proceeds with booting
3. KDE starts, KDE wallet tries to decrypt the wallet, hence the smart card PIN is asked
4. enter the smart card PIN and get "card error" message
5. each time the PIN is entered, the token blinks briefly before "card error" message appears
6. the token doesn't panic (at least, there's no red LED observed)
7. restarting gpg-agent doesn't help
8. restarting scdaemon doesn't help
9. gpg --card-status shows usual information and does not produce any error
10. pcscd is not used
11. replugging the token fixes the issue till next time

There's no such issue observed with YubiKey 5 NFC.

[sosthene-nitrokey]

Thank you for the report.

I do not have a systemd-cryptenroll setup on hand but I did try to reproduce the issue with kwalletmanager, and failed, with the same versions of GPG and kwallet, even right after boot or after a FIDO2 operation.

Are you able to reproduce the same issue on another machine?

Do you have any special configuration in ~/.gnupg/scdaemon.conf or in ~/.gnupg/gpg-agent.conf?

Can you try adding to ~/.gnupg/scdaemon.conf:

log-file "/tmp/scdaemon.logs"

and in ~/.gnupg/gpg-agent.conf

log-file "/tmp/gpg-agent.logs"

Then after reproducing issue send us the those logs (note that those logs may contain personal information (name stored in the card, key fingerprints, device serial number...), therefore you may want to either inspect and redact them or send it through a private channel.

[sosthene-nitrokey]

Also, which GPG key algorithm are you using?

[pfactum]

Hello.

Are you able to reproduce the same issue on another machine?

I haven't tried yet, but I think I will be able to do that later.

Do you have any special configuration in ~/.gnupg/scdaemon.conf or in ~/.gnupg/gpg-agent.conf?

Except of no-allow-external-cache in ~/.gnupg/gpg-agent.conf — no.

Then after reproducing issue send us the those logs […]

Collected both. Where should I send them to? Also, please provide me with your GPG public key so that I can encrypt the logs before sending.

Also, which GPG key algorithm are you using?

RSA4096.

Thanks.

[pfactum]

FWIW, if I replug the token during boot after unlocking the disk but before KDE and gpg is even started, things work OK. So, the problem should not be with the gpg config.

Also, if I unlock the wallet in KDE successfully and then reboot the machine without replugging the token, the token panics with red LED on disk FIDO decryption.

[sosthene-nitrokey]

Collected both. Where should I send them to? Also, please provide me with your GPG public key so that I can encrypt the logs before sending.

Thank you
Either through element were we have already discussed or you can send it at [email protected], and if you want encrypted with my public key, which you can find from github: https://github.com/sosthene-nitrokey.gpg.

Also, if I unlock the wallet in KDE successfully and then reboot the machine without replugging the token, the token panics with red LED on disk FIDO decryption.

Thank you for reporting this. I opened another issue for this: #286

[pfactum]

Logs sent.

[pfactum]

Are you able to reproduce the same issue on another machine?

I haven't tried yet, but I think I will be able to do that later.

I can also confirm the issue on a freshly installed system on a different machine.

[sosthene-nitrokey]

Thank you. Sadly the log from gpg-agent and scdaemon do not contain enough information to pinpoint the source of the error.

There appear to be multiple ways to get systemd-cryptenroll to work on Arch Linux, would you be willing to provide a virtual machine image capable of reproducing the bug, or detailed instruction on how to create such an image?

[pfactum]

There appear to be multiple ways to get systemd-cryptenroll to work on Arch Linux

Not sure I understand what you mean. All I did was:

$ sudo systemd-cryptenroll --fido2-device=auto /dev/nvme0n1p2

Could you please elaborate?

[sosthene-nitrokey]

How did you first setup the full disk encryption?
Using systemd-cryptenroll does work but then it doesn't prompt for the fido device on boot (I used archinstall to install it in a virtual machine and setup encryption).

From the arch wiki:

First, you will need to setup your /etc/crypttab file, or customize your initramfs if you wish to unlock your root partition. The full procedure is similar to the use of a TPM chip for unlocking. See Trusted Platform Module#systemd-cryptenroll.

Enabling the systemd and sd-encrypt mkinitcpio hooks makes the virtual machine fail to boot, stuck at "loading initial ramdisk"

[pfactum]

In /etc/mkinitcpio.conf I have:

HOOKS=(base systemd autodetect modconf kms keyboard sd-vconsole plymouth block mdadm_udev sd-encrypt filesystems fsck)

In /etc/crypttab.initramfs I have:

system  UUID="xxx-xxx-xxx-xxx-xxx" none    discard,fido2-device=auto

And my kernel cmdline is:

root=/dev/mapper/system threadirqs io_delay=none zswap.enabled=0 amd_pstate=guided bgrt_disable splash quiet

I have never used archinstall.

[sosthene-nitrokey]

Thank you. I am able to reproduce the bug now. 

First thing I noticed is that it only affects RSA keys, curve25519 keys work fine.

Looking at the logs from the device I get the following error:

Failed to load key: FilesystemWriteFailure

For the fido panic, I get an Err(NoSpace) error: That's weird.
It could be that for some reason something fills up the volatile storage?

[sosthene-nitrokey]

It looks like we have an issue with the volatile little fs filesystem.

The filesystem appears full (1 free block) when there is an RSA 4096 key stored in it. But the key itself is only 2.378 KB, while the volatile filesystem is defined to be 8KiB.

Is littlefs that inefficient?

[sosthene-nitrokey]

Reducing Littlefs block size will solve the issue. To avoid this issue from also happening when multiple RSA keys are loaded in volatile storage, Nitrokey/opcard-rs#164 will limit their number.