Passphrase-based API of Secure Cell

[ilammy]

Implement passphrase-based API of Secure Cell in Themis Core:

themis_status_t themis_secure_cell_encrypt_seal_with_passphrase(
        const uint8_t*  passphrase,
        size_t          passphrase_length,
        const uint8_t*  user_context,
        size_t          user_context_length,
        const uint8_t*  message,
        size_t          message_length,
        uint8_t*        encrypted_message,
        size_t*         encrypted_message_length
);

themis_status_t themis_secure_cell_decrypt_seal_with_passphrase(
        const uint8_t*  passphrase,
        size_t          passphrase_length,
        const uint8_t*  user_context,
        size_t          user_context_length,
        const uint8_t*  encrypted_message,
        size_t          encrypted_message_length,
        uint8_t*        plain_message,
        size_t*         plain_message_length
);

The API is described in Themis RFC 2 (not available publicly at the moment). This pull request implements the API in Themis Core on which we’ll base implementations in all high-level language wrappers.

Main issue that is solved by passphrase API is inadeqacy of ZRTP KDF used by master key API when applied to human-readable passphrases with low entropy. Previously we suggested and allowed such use in high-level languages, however it is not secure enough. Master key API should be used only with randomly-generated binary keys. Our symmetric key generation API produces 32-byte-long keys.

New API can be safely used with human-readable strings containing way less entropy. This is made possible by using a proper password key derivation function — PBKDF2 — to generate intermediate key used for encryption. Currently we are using 200,000 iterations — the slowdown is not noticeable for single-time use but is frustrating enough for repeated computation during brute-force attacks.

Passphrase support is currently implemented only for Seal mode of Secure Cell. It can be extended to Token Protect mode in the future. Context Imprint mode will never support passphrases directly because it does not provide storage space for KDF context. However, we may provide separate KDF interface later which will make it possible to use passphrases with Context Imprint mode too.

The implementation is split into five main parts:

• Define authentication token format

  I’ve opted for more explicit approach to parsing and producing data formats. I believe that such approach is less error-prone and contains less magic. Though it is much more verbose that fast, direct, and incorrect casts to struct pointers that we are currently doing in master key API code.

  Give it a close look as this will be our main sponsor of hidden segmentation faults.

• Prepend authentication token in Seal mode

  Token Protect mode keeps the token as a separate buffer while in Seal mode it is prepended to the encrypted message. We have to carefully gauge the output buffer size during encryption, and carefully parse input data during decryption.

• Secure Cell encryption

  It is pretty straightforward. There are some Soter quirks that we need to deal with, and there are a couple of restrictions on the input data that we need to honor.

  Also note that we encrypt data using default KDF parameters which are fixed for Themis release. However, we may review and change them in future releases. Unless we introduce a new KDF algorithm, simple parameter tweaks are backwars and forward-compatible. With PBKDF2 we can adjust salt length (unlikely) and iteration count.

• Secure Cell decryption

  This code path needs close attention as here we have to deal with actively hostile user input. Separate data format parsing makes it easier to follow, but still be alert.

• Tests, documentation comments, changelog updates, etc.

  The tests are basic but they cover the most important points. New additions include explicit tests for all AES key lengths that we support. (Yes, I did fail to take that into account during development.)

  As for what they lack, we currently do not verify that Secure Cell is able to detect data corruption, for example. I hope to not forget to add them later.

Have fun reviewing these 1.5k lines of code next week!

Checklist

• Change is covered by automated tests
• Benchmark results are attached (coming soon)
• The coding guidelines are followed
• Public API has proper documentation
• Example projects and code samples are updated (later)
• Changelog is updated

Address feedback:

• add stream_ prefix to parsing helpers
• generalize KDF handling
• try avoiding auth_tag_length comparisons

[Lagovas]

it's very cool to use such helpers and encapsulate hard to read logic into function with a human name

[Lagovas]

cool functions documentation

[ignatk]

Reuse total_length here?

[ilammy]

This is intentional because the themis_auth_sym_encrypt_message_with_passphrase() call might have modified those values. Of course, it does not modify them and accurately reports required lengths so we could reuse the value. But I'd leave that for future-proofing.

For all functions that return buffers, it's not strictly incorrect behavior to return slightly bigger required buffer size during measurement stage (BUFFER_TOO_SMALL) and then return accurate – and possibly smaller – actual size of the buffer used on SUCCESS. Currently we are able to accurately determine required length without actually processing the data, but it might not be the case in the future.

[Lagovas]

lgtm

[ilammy]

This PR blocks a number of other PRs so I'm merging it right now with only one approval. Since most of the issues have been addressed and this PR has received some reviewer attention, I guess it's okay. @ignatk, @storojs72, if you have any concerns about these changes, please write them here, I'll address them separately.