Partial KMS session encryption

[mitchellhenke]

Currently, we double encrypt the entire web session for a user. When saving a session, first it is converted to JSON, then encrypted with AES, then with KMS, and then put into Redis. The reverse process happens when the session is loaded.

This safe-by-default method is straightforward conceptually and in implementation, but imposes costs and friction in other ways. We have added more and more to session storage, and KMS can only encrypt 4kB per API call, which leads to us chunking the string and having to make multiple calls to KMS. Certain values we store can be quite large, which inflates the session size significantly in some contexts. The KMS API calls now take up a sizeable portion of our network and server time.

This method also places KMS in the critical path since it is used on most requests, including the home page.

From a security perspective, it means we decrypt sensitive data even when it won't be used. The most relevant part in this regard is the PII bundle, which we only need to KMS decrypt when we are displaying it to the user or passing it to the service provider.

To meet our security requirements and ensure that sensitive data continues to be KMS encrypted while vastly reducing our KMS usage otherwise, this PR attempts to split the session into sensitive and less sensitive data. The entire session is still AES encrypted, but the sensitive parts are KMS encrypted still. These changes aim to handle PII in two ways:

1. For the PII bundle, it is decrypted with the user's password when we receive it and then it is stored in the session. Subsequent requests encrypt it with KMS inside the AES encrypted session. It is now KMS decrypted on-demand via the Pii::Cacher interface that we've moved our PII bundle handling towards using (Create common interface for accessing PII bundle in session #6054, Do not save additional PII to session when not needed #6072, Move more PII bundle usage to the common interface in Pii::Cacher #6273, Rename reactivate account session :personal_key key to be more descriptive #6283).
2. For personal keys and IDV-related PII where the usage is a bit more dynamic and harder to standardize, the KMS encryption/decryption happens transparently similar to how it is today.

Alerting has been added if we detect a potentially sensitive key not being KMS encrypted as it should be, and sends the session structure without values to help troubleshoot if it does. In tests and development, it will raise exceptions if a sensitive key is not encrypted properly.

This change is not backwards compatible on its own, and will require deploying the code that contains the ability to parse both formats (without writing the new format immediately). Writing the new format to Redis is currently behind a configuration flag that we can switch on if and when we feel comfortable. If the new format is problematic after being enabled, the switch can be turned off with full backwards compatibility.

Document
Discussion Thread

[jmhooper]

Can you help me understanding why the logic below is necessary? It looks like SessionEncryptor#kms_encrypt_pii! should prevent user_session[:encrypted_pii] from being an issue.

[jmhooper]

Slacked with Mitchell and see where I got hung up. The session encryptor encrypts the user_session[:encrypted_pii] value, but never actually decrypts it. It leaves this to the session encryptor so we don't need to run KMS on every request for a proofed user.

[mitchellhenke]

Attempted to clarify this better in cfea392

[orenyk]

I got a little lost in the weeds in the actual encryption stuff (hoping to take a closer look later) but in the meantime here are some minor style comments/suggestions. Great work!

[aduth]

Is it worth doing something similar to what was done in FakeAnalytics, referencing the constant from which these values are generated?

identity-idp/spec/support/fake_analytics.rb

Lines 23 to 31 in 0e8c7e1

	DocAuth::Mock::ResultResponseBuilder::DEFAULT_PII_FROM_DOC.slice(
	:first_name,
	:last_name,
	:address1,
	:zipcode,
	:dob,
	:state_id_number,
	).each do |key, default_pii_value|
	if string_payload.match?(Regexp.new('\b' + Regexp.quote(default_pii_value) + '\b', 'i'))

[mitchellhenke]

Thank you that's a great catch, I had forgotten we had them in a constant.

SessionEncryptor being in lib/ means it can't reference things outside of lib/ so I moved it into our constants file in e11e91946, but I'm not sure if that's the best place

[aduth]

When I was using it for spec/factories/profiles.rb in #6229, I had felt it was kinda weird to reference the constant where it was at, so I like the idea of having a single constant of fake PII in a common location like that.