-
Notifications
You must be signed in to change notification settings - Fork 519
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Signed Migrations #905
Comments
I'd like to get the API labeling part of #764 done before this lands, so we can ensure the local repository and the migrations have the right labels on disk. That would act as a further mitigation for the replay scenario. EDIT: having thought about it more, I don't think we want |
Edit: We have decided to support a compatibility migration path. We plan to release a version that can support both signed and unsigned migrations (a 'pivot' version). After that, a version that does not support unsigned migrations will be released, and the repo URL will be changed to ensure no instances can skip the 'pivot' version. |
Feature: Signed Migrations
Summary
During an upgrade or downgrade sequence,
migrator
runs migrations that have not been cryptographically verified since the prior boot. We propose a change to Bottlerocket that would improve security by verifying migrations using The Update Framework (TUF) at the time that they are executed. Originally documented in #91, the current issue serves as an RFC for the proposed solution. (Formalizing an RFC process is #36.)Motivation
Security is a primary tenet of Bottlerocket. Signed Migrations will make Bottlerocket more secure by cryptographically verifying migrations at boot time.
Background
When a Bottlerocket upgrade or downgrade takes place, if the data store needs to be changed, executable binaries called migrations are used to alter the data store.
updog
determines what migrations will be needed based on data found in themanifest.json
file (the manifest) stored in the TUF repo.updog
downloads the needed migrations and uncompresses them into/var/lib/bottlerocket-migrations
Upon boot, a program namedmigrator
runs these binaries to make the necessary alterations to the data store.updog
uses TUF (via thetough
library), which means that the manifest and migrations are cryptographically verified byupdog
.migrator
, on the other hand, lists the contents of/var/lib/bottlerocket-migrations
to discover migrations and runs them based on a file naming convention.This introduces a vulnerability because
migrator
is not aware if the migrations have been manipulated since the time thatupdog
verified them.Proposed Changes
We propose a change such that
migrator
will no longer discover and run unverified binaries. Insteadmigrator
will usetough
and the manifest to determine which migrations are needed and load them into memory thereby ensuring that migrations are verified.Proposed migration process
updog
usestough
to obtain the manifest (as before).updog
uses the manifest to determine which migrations are needed.updog
savestuf
-friendly filenames.tuf
repo metadata needed formigrator
to verify them. (timestamp.json
,targets.json
,snapshot.json
, and all versions ofroot.json
.)migrator
usestough
to load the manifest.migrator
uses the manifest to determine which migrations are needed.migrator
loads the migrations viatough
which ensures they are cryptographically secure.migrator
decompresses the migrations.migrator
runs the migrations from memory using pentacle.Shared Code
Because
tough
andmigrator
will both use the manifest to determine which migrations are needed, the code that does this will need to be shared by bothupdog
andmigrator
.root.json
migrator
works the same way during a downgrade as it does during an upgrade with the following caveat: once booted into the older partition,migrator
may be referencing a root.json file older than the one used to sign the manifest and migrations. In that case,migrator
relies on the fact thattough
has all the intermediate root.json files needed to work properly.Drawbacks, Rationale and Alternatives
Backward Compatibility
The proposal does not handle the case of downgrading into a version of Bottlerocket that existed prior to the roll-out of signed migrations. We do not take breaking of downgrades lightly, however we believe this change is needed to better secure Bottlerocket.
The issue of backward compatibility could arise during a failed upgrade boot when Bottlerocket would fallback to the previous partition. The backward compatibility issue could also arise during an explicit attempt to downgrade. In either case, if the manner in which migrations have been saved by
updog
is different from what an old version ofmigrator
expects, the boot will fail.To mitigate this issue we could have
updog
continue to write the uncompressed migrations where old versions ofmigrator
expect them (in addition to all the files described in the changes above). Then either an oldmigrator
version or a new one can find what it needs. A newmigrator
version would only run signed migrations and an old version would only run the unsigned migrations.However, leaving this downgrade path in place also leaves in place the possibility of unsigned binaries running at boot. In other words, to completely prevent unsigned binaries from running at boot we need to prevent hosts from being downgraded to an old version of
migrator
, and thus we propose a change that breaks downgrades to prevent it.Additional
migrator
ComplexityBecause
migrator
is crucial to the Bottlerocket boot process, we want to keep it as simple as possible. This proposal increases the complexity ofmigrator
by addingtough
, decompression, and usage of the manifest.LZ4
We considered retaining the responsibility for uncompressing the binaries in
updog
. However this proves to be unwieldy as we would need to add the uncompressed migration checksums into the TUF repo. This is far more complicated than adding an LZ4 deflate routine tomigrator
.The Manifest
We considered having
updog
write out a list of migrations to a file, or havingmigrator
rely on the list of files thatupdog
has written (as it does now) in order to avoid havingmigrator
run logic against the manifest. We rejected these ideas because they leave a vulnerability where an attacker could affect the list or order of migrations to be run.TUF Timestamps
TUF uses expiration timestamps to limit a replay attack window. However, in the case of Signed Migrations it is possible for a TUF repo timestamp to expire between the time that
updog
downloads the information and the time the migrator runs. In this case, we would rather ignore the timestamp expiration than fail the boot.The rationale is that the TUF repo timestamps serve their purpose when
updog
downloads the information (updog
would fail on an expired timestamp). Onceupdog
has downloaded the data, we explicitly want it to remain valid until migrator has had a chance to use it.We propose to have migrator ignore TUF timestamps. Using the TUF framework in an ‘ignore timestamps’ mode will serve our use case without affecting other uses of
tough
.Open Questions
/var/lib/bottlerocket-metadata
, for example, instead of e.g./var/lib/bottlerocket-migrations/metadata
?The text was updated successfully, but these errors were encountered: