[PAL/Linux-SGX] Add AEX-Notify support

[lzha101]

AEX-Notify is a security feature used to mitigate the SGX-Step type attacks on SGX. The SGX-Step type attacks rely on frequent enclave preemptions (e.g. interrupts) to execute an SGX enclave in small increments and strategically extract secret data from the enclave through one or more side channels. The AEX Notify feature allows a mitigation handler to run after ERESUME from each Async Exit, aiming to mitigate the side-channel information exposed by the attacks.

This commit adds the support of AEX-Notify for SGX.

• Add a new manifest option sgx.enable_aex_notify
• Change the exception handling flow for AEX-Notify support
• Provide a default mitigation AEX-Notify handler

Description of the changes

A public white paper of AEX-Notify with SGX SDK: https://cdrdv2-public.intel.com/736463/aex-notify-white-paper-public.pdf
SGX SDK adds the AEX-Notify support starting from 2.20 release, while the default AEX-Notify handler is an updated version, which is different from what describes in the paper.

Similar as SGX SDK, Gramine also implements a two-stage exception handler (with NSSA=2). The stage-1 handler
runs in the SSA[1] context, and the stage-2 handler runs in the SSA[0] context. The trusted main flow also runs in the SSA[0] context. So we can act like SGX SDK to extend the existing exception handling flow to handle AEX Notifications.

Here is the workflow for how an enclave thread enables AEX-Notify, handles an AEX notification, and then returns back to where the AEX had occurred:

1. The enclave thread (in the SSA[0] context) enables AEX-Notify by setting bit 0 within SSA[0].GPRSGX.AEXNOTIFY
2. The enclave encounters an interrupt or exception
3. An AEX is triggered, which increments TCS.CSSA
4. The untrusted pal calls sgx_handle_aex_signal() to perform a ENCLU[EENTER] to continue enclave execution.
5. The enclave entry point notices that TCS.CSSA=1 and jumps to the stage-1 handler (possibly after some other checks)
6. Stage-1 handler flow:
   a. Setup stack on sig stack for Stage-2 handler
   b. Copy the SSA[0] context and others to sig stack
   c. Disable AEX-Notify in SSA[0] by clearing bit 0 within SSA[0].GPRSGX.AEXNOTIFY
   d. Execute ENCLU[EDECCSSA] and jump to Stage-2 handler
7. Stage-2 handler flow:
   a Check for and handle an exception, if necessary
   b. Restore XSAVE context, rflags and some GPRs that are not used in AEX-Notify mitigation handler
   c. Enable AEX-Notify by setting bit 0 within SSA[0].GPRSGX.AEXNOTIFY
   d. Trigger the AEX-Notify mitigation handler
   e. Restore other GPRs and jump to the point where the AEX occurred.

We need to handle the case that the AEX-Notify mitigation handler is WIP (at step 7.d ) while the other interrupt happens. In this case, the execution will be back to step 3. To ensure the original GPRs are restored, we allocate an extra memory on sig stack on step 6.1 to save the necessary GPRs in previous Stage-2 handler flow.

How to test this PR?

Prerequisites: Prepare a machine that supports AEX-Notify

1. Add sgx.enable_aex_notify=true to the manifest files for both PAL and LibOS regression tests.
2. Go to the regression test directories and run gramine-test --sgx pytest -v

Maybe later we can create the other PR to enable aex-notify feature in the all or some the regression tests based on the platform feature, similar as EDMM.

---

This change is 

[dimakuv]

Reviewable status: 0 of 20 files reviewed, 2 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @lzha101)

a discussion (no related file):
This PR is very hard to review.

I strongly suggest to split this PR into three dependent PRs:

1. Description and enablement of AEX Notify feature -- these are mostly documentation stuff, new manifest option (that will be dummy in this first PR), new bits, code for initialization/finalization of these bits for the whole enclave and for each enclave thread.
2. The actual AEX Notify feature implementation -- mainly changes in .S files + the untrusted exception handler. The mitigation path will be dummy/empty.
3. The actual mitigation.

Then PR 1 will be trivial to review. PR 2 will be the bulk of the reviewing process. And finally PR 3 will be the most controversial -- I guess we will argue about the mitigations a lot...

---

---

pal/src/host/linux-sgx/enclave_entry.S line 1332 at r1 (raw file):

.sect   _to_be_discarded, "e", @nobits
.space MITIGATION_CODE_ALIGNMENT - (.ct_aexnotify_end - .ct_enable_aexnotify)
.previous

I feel that the whole mitigation code should be in a separate file like enclave_aex_notify_mitigations.S

[lzha101]

Reviewable status: 0 of 20 files reviewed, 2 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ) (waiting on @dimakuv)

a discussion (no related file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

This PR is very hard to review.

I strongly suggest to split this PR into three dependent PRs:

1. Description and enablement of AEX Notify feature -- these are mostly documentation stuff, new manifest option (that will be dummy in this first PR), new bits, code for initialization/finalization of these bits for the whole enclave and for each enclave thread.
2. The actual AEX Notify feature implementation -- mainly changes in .S files + the untrusted exception handler. The mitigation path will be dummy/empty.
3. The actual mitigation.

Then PR 1 will be trivial to review. PR 2 will be the bulk of the reviewing process. And finally PR 3 will be the most controversial -- I guess we will argue about the mitigations a lot...

OK. I will try to split the PR according to your suggestion.

---