AESM service returned error 20 #1297

blank-vax · 2023-04-20T03:16:03Z

blank-vax
Apr 20, 2023

Some errors occur when I execute the commands as the flow in CI-Example/ra-tls-secret-prov/README.md. The following operations are performed in the gramineproject/gramine docker in my Ubuntu 20.04 server with the code of branch v1.4.
After executing the commands:

make app epid RA_TYPE=epid RA_CLIENT_SPID=my_intel_SPID RA_CLIENT_LINKABLE=0

I successfully got the built results. Then I perform the command ./server_epid wrap_key & to start the server after the setting operations of RA_TLS_ALLOW_DEBUG_ENCLAVE_INSECURE, RA_TLS_ALLOW_OUTDATED_TCB_INSECURE AND RA_TLS_EPID_API_KEY=my_intel_SPID_primary_key.
When I perform the command gramine-sgx ./client, some unexpected errors appear:

error: AESM service returned error 20; this may indicate that infrastructure for the EPID attestation requested by Gramine is missing on this machine
error: load_enclave() failed with error: Operation not permitted (EPERM)

Here are the outputs after the command is-sgx-available:

SGX supported by CPU: true
SGX1 (ECREATE, EENTER, ...): true
SGX2 (EAUG, EACCEPT, EMODPR, ...): false
Flexible Launch Control (IA32_SGXPUBKEYHASH{0..3} MSRs): true
SGX extensions for virtualizers (EINCVIRTCHILD, EDECVIRTCHILD, ESETCONTEXT): false
Extensions for concurrent memory management (ETRACKC, ELDBC, ELDUC, ERDINFO): false
CET enclave attributes support (See Table 37-5 in the SDM): false
Key separation and sharing (KSS) support (CONFIGID, CONFIGSVN, ISVEXTPRODID, ISVFAMILYID report fields): false
Max enclave size (32-bit): 0x80000000
Max enclave size (64-bit): 0x1000000000
EPC size: 0x5e00000
SGX driver loaded: true
AESMD installed: true
SGX PSW/libsgx installed: true
#PF/#GP information in EXINFO in MISC region of SSA supported: false
#CP information in EXINFO in MISC region of SSA supported: false

As the above shows, the AESMD is successfully installed in my machine, but what should I do to figure out the error mentioned above?

kailun-qin · 2023-04-20T03:43:33Z

kailun-qin
Apr 20, 2023
Maintainer

Hi @blank-vax!

AESM error 20 corresponds to AESM_UPDATE_AVAILABLE (see https://github.com/intel/linux-sgx/blob/1efe23c20e37f868498f8287921eedfbcecdc216/psw/ae/inc/internal/aesm_error.h#L56) -- this usually means that your SGX PSW need be updated.

You could double check with 1) the installed SGX PSW version in your env (via sudo apt list --installed | grep sgx); and 2) the AESM service status via sudo systemctl status aesmd.

3 replies

blank-vax Apr 20, 2023
Author

Thank you so much! It does work after I perform apt-get update & apt-get upgrade which upgrade the version of libsgx libraries from 2.18.100.1-focall to 2.19.100.3-focall.
However, new error occurs when I repeat the command gramine-sgx ./client as follows:

error: AES service returned error 1;
error: load_enclave() failed with error: Operation not permitted (EPERM)

I tend to https://github.com/intel/linux-sgx/issues/702 for help, but unfortunately, the reason why this error occurs in that issue is that the platform with SGX-1 only support EPID remote attestation.
However, I indeed only perform the EPID remote attestation with SGX-1, but the error mentioned above still appears. Here are the outputs of the execution of the test of :

eax: a0655 ebx: 2100800 ecx: 7ffafbff edx: bfebfbff
stepping 5
model 5
family 6
processor type 0
extended model 10
extended family 0
smx: 1

Extended feature bits (EAX=07H, ECX=0H)
eax: 0 ebx: 29c67af ecx: 40000018 edx: bc000600
sgx available: 1
sgx launch control: 1

CPUID Leaf 12H, Sub-Leaf 0 of Intel SGX Capabilities (EAX=12H,ECX=0)
eax: 1 ebx: 0 ecx: 0 edx: 241f
sgx 1 supported: 1
sgx 2 supported: 0
MaxEnclaveSize_Not64: 1f
MaxEnclaveSize_64: 24

CPUID Leaf 12H, Sub-Leaf 1 of Intel SGX Capabilities (EAX=12H,ECX=1)
eax: 36 ebx: 0 ecx: 21f edx: 0

CPUID Leaf 12H, Sub-Leaf 2 of Intel SGX Capabilities (EAX=12H,ECX=2)
eax: 90200001 ebx: 0 ecx: 5e00001 edx: 0
size of EPC section in Processor Reserved Memory, 94 M

CPUID Leaf 12H, Sub-Leaf 3 of Intel SGX Capabilities (EAX=12H,ECX=3)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 4 of Intel SGX Capabilities (EAX=12H,ECX=4)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 5 of Intel SGX Capabilities (EAX=12H,ECX=5)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 6 of Intel SGX Capabilities (EAX=12H,ECX=6)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 7 of Intel SGX Capabilities (EAX=12H,ECX=7)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 8 of Intel SGX Capabilities (EAX=12H,ECX=8)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

CPUID Leaf 12H, Sub-Leaf 9 of Intel SGX Capabilities (EAX=12H,ECX=9)
eax: 0 ebx: 0 ecx: 0 edx: 0
size of EPC section in Processor Reserved Memory, 0 M

dimakuv Apr 20, 2023

What is the output of sudo systemctl status aesmd?

blank-vax Apr 20, 2023
Author

With the --priviledge=true command added in the docker run commands I successfully create a new docker with systemctl available.
Then I perform the command systemctl status aesmd to check the status of aesmd and get the following output:

aesmd.service - Intel(R) Architectural Enclave Service Manager
    Loaded: loaded (/usr/lib/systemd/system/aesmd.service, enabled)
    Active: failed (dead)

When I try to start the aesmd service with systemctl start aesmd , I received the error code :

ERROR:systemctl:the ExecStartPre control process exited with error code

blank-vax · 2023-04-20T10:58:26Z

blank-vax
Apr 20, 2023
Author

When I try to start the aesmd service with systemctl start aesmd, the aforementioned error occurs. When I turn to /var/log/journal/aesmd.service.log for help, I got the following information:

Root privilege is required.

However, I start this docker with --priviledge=true command, I wonder how to figure out such problems.
Any suggetstion is welcome, thank you so much!

4 replies

llly Apr 21, 2023

sgx-aesm-service from apt is not expected to run in docker. Usually, we run aesm service in host and expose /var/run/aesmd/aesm.socket of host to docker. Then Gramine app in docker can utilize aesm service.
You also need to register to get your own my_intel_SPID. Read more in https://api.portal.trustedservices.intel.com/EPID-attestation.

blank-vax Apr 21, 2023
Author

The command systemctl status aesmd is utilized in my host machine and the output is here:

aesmd.service - Intel(R) Architectural Enclave Service Manager
     Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-04-20 21:27:17 CST; 12h ago
   Main PID: 126482 (aesm_service)
      Tasks: 4 (limit: 18739)
     Memory: 4.2M
     CGroup: /system.slice/aesmd.service
             └─126482 /opt/intel/sgx-aesm-service/aesm/aesm_service

4月 20 21:27:17 SGX systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
4月 20 21:27:17 SGX usermod[126467]: add 'aesmd' to group 'sgx_prv'
4月 20 21:27:17 SGX usermod[126467]: add 'aesmd' to shadow group 'sgx_prv'
4月 20 21:27:17 SGX aesm_service[126481]: aesm_service: warning: Turn to daemon. Use "--no-daemon" option to execute in foreground.
4月 20 21:27:17 SGX systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
4月 20 21:27:17 SGX aesm_service[126482]: [get_qpl_handle ../qe_logic.cpp:294] Cannot open Quote Provider Library libdcap_quoteprov.so.1 and libdcap_quoteprov.so
4月 20 21:27:17 SGX aesm_service[126482]: The server sock is 0x55ca99a245b0

It seems that there is no DCAP support for the aesmd.service, but it doesn't matter as there is only the EPID remote attestation in my project.
However, after I use the command sudo docker run --device /dev/sgx_enclave --device /dev/sgx_provision --privileged=true --name=SGX2 -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket -it gramineproject/gramine to mount the /var/run/aesmd/aesm.socket in my docker container, I could indeed find the aesm.socket in /var/run/aesmd in my docker, but when I use the command systemctl status aesmd, the status of aesmd.service is also keep failed(dead) as I mentioned above.
From my point of view, it seems that the mapping from the host to my container by -v command failed.

kailun-qin Apr 21, 2023
Maintainer

I could indeed find the aesm.socket in /var/run/aesmd in my docker, but when I use the command systemctl status aesmd, the status of aesmd.service is also keep failed(dead) as I mentioned above. From my point of view, it seems that the mapping from the host to my container by -v command failed.

You actually do not need to run systemctl start/status aesmd in your container -- it should just work if your host AESM service works fine and you've also mounted it into your container.

Have you subscribed a SPID already? If not, you should do it here first: https://api.portal.trustedservices.intel.com/EPID-attestation. If this is already done, you have to use the SPID which you have subscribed and based on the type of SPID to set RA_CLIENT_SPID and RA_CLIENT_LINKABLE in Gramine with the corresponding info and retry the example.

blank-vax Apr 21, 2023
Author

Thank you so much. I use the command -v to mount the aesmd.service to my docker, then I go into it and run the commands as the README.md located in CI-Examples/ra-tls-secret-prov directory to build them.
Also, I have subscribed a SPID already, and the SPID and Primary key are utilized during the building and execution of the programs. Just like what shows below:

make app epid RA_TYPE=epid RA_CLIENT_SPID=my_SPID \
     RA_CLIENT_LINKABLE=0

RA_TLS_ALLOW_DEBUG_ENCLAVE_INSECURE=1 \
> RA_TLS_ALLOW_OUTDATED_TCB_INSECURE=1 \
> RA_TLS_EPID_API_KEY=my_primary_key \
> ./server_epid wrap_key &

However, when I execute the command gramine-sgx ./client I got the errors as below:

(host_platform.c:156:init_quoting_enclave_targetinfo) error: AESM service returned error 20; this may indicate that infrastructure for the EPID attestation requested by Gramine is missing on this machine
(host_main.c:1296:main) error: load_enclave() failed with error: Operation not permitted (EPERM)

It seems that my SGX PSW is not up-to-date, however, after the commands apt-get update & apt-get upgrade, I still got that error. Here are is the output after execution of the command apt list --installed | grep sgx:

libsgx-ae-epid/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-id-enclave/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-ae-le/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-pce/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-qe3/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-ae-qve/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-aesm-ecdsa-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed]
libsgx-aesm-epid-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed]
libsgx-aesm-launch-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed]
libsgx-aesm-pce-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-aesm-quote-ex-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed]
libsgx-dcap-quote-verify/unknown,now 1.16.100.2-focal1 amd64 [installed]
libsgx-enclave-common/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-launch/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-pce-logic/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-qe3-logic/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-urts/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
sgx-aesm-service/unknown,now 2.19.100.3-focal1 amd64 [installed]

blank-vax · 2023-04-21T09:35:39Z

blank-vax
Apr 21, 2023
Author

I tried to finish the steps mentioned above in my host machine with SGX-1 following the instructions in CI-Examples/ra-tls-secret-prov/README.md, unfortunately I still got the errors as below:

error: AESM service returned error 20; this may indicate that infrastructure for the EPID attestation requested by Gramine is missing on this machine
error: load_enclave() failed with error: Operation not permitted (EPERM)

And here are the outputs of the command sudo apt list --installed | grep sgx and sudo systemctl status aesmd separately:

libsgx-ae-epid/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-ae-id-enclave/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-ae-le/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-ae-pce/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-ae-qe3/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-ae-qve/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-aesm-ecdsa-plugin/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-aesm-epid-plugin/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-aesm-launch-plugin/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-aesm-pce-plugin/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-aesm-quote-ex-plugin/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-dcap-default-qpl-dev/未知,now 1.16.100.2-focal1 amd64 [已安装]
libsgx-dcap-default-qpl/未知,now 1.16.100.2-focal1 amd64 [已安装]
libsgx-dcap-ql-dev/未知,now 1.16.100.2-focal1 amd64 [已安装]
libsgx-dcap-ql/未知,now 1.16.100.2-focal1 amd64 [已安装]
libsgx-dcap-quote-verify/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-enclave-common-dev/未知,now 2.19.100.3-focal1 amd64 [已安装]
libsgx-enclave-common/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-epid/未知,now 2.19.100.3-focal1 amd64 [已安装]
libsgx-headers/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]
libsgx-launch/未知,now 2.19.100.3-focal1 amd64 [已安装]
libsgx-pce-logic/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-qe3-logic/未知,now 1.16.100.2-focal1 amd64 [已安装，自动]
libsgx-quote-ex/未知,now 2.19.100.3-focal1 amd64 [已安装]
libsgx-urts/未知,now 2.19.100.3-focal1 amd64 [已安装]
sgx-aesm-service/未知,now 2.19.100.3-focal1 amd64 [已安装，自动]

● aesmd.service - Intel(R) Architectural Enclave Service Manager
     Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-04-21 17:30:57 CST; 4min 0s ago
    Process: 412119 ExecStartPre=/opt/intel/sgx-aesm-service/aesm/linksgx.sh (code=exited, status=0/SUCCESS)
    Process: 412128 ExecStartPre=/bin/mkdir -p /var/run/aesmd/ (code=exited, status=0/SUCCESS)
    Process: 412129 ExecStartPre=/bin/chown -R aesmd:aesmd /var/run/aesmd/ (code=exited, status=0/SUCCESS)
    Process: 412130 ExecStartPre=/bin/chmod 0755 /var/run/aesmd/ (code=exited, status=0/SUCCESS)
    Process: 412131 ExecStartPre=/bin/chown -R aesmd:aesmd /var/opt/aesmd/ (code=exited, status=0/SUCCESS)
    Process: 412132 ExecStartPre=/bin/chmod 0750 /var/opt/aesmd/ (code=exited, status=0/SUCCESS)
    Process: 412133 ExecStart=/opt/intel/sgx-aesm-service/aesm/aesm_service (code=exited, status=0/SUCCESS)
   Main PID: 412134 (aesm_service)
      Tasks: 4 (limit: 18739)
     Memory: 3.9M
     CGroup: /system.slice/aesmd.service
             └─412134 /opt/intel/sgx-aesm-service/aesm/aesm_service

4月 21 17:30:57 SGX systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
4月 21 17:30:57 SGX aesm_service[412133]: aesm_service: warning: Turn to daemon. Use "--no-daemon" option to execute in foreground.
4月 21 17:30:57 SGX systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
4月 21 17:30:57 SGX aesm_service[412134]: The server sock is 0x55af930565b0
4月 21 17:31:30 SGX aesm_service[412134]: [ADMIN]EPID Provisioning initiated
4月 21 17:31:32 SGX aesm_service[412134]: The Request ID is 11acf4e785ff481fbe0b529f7b967b71
4月 21 17:31:34 SGX aesm_service[412134]: The Request ID is 15a44916f0f848538e5900a9c5443a8a
4月 21 17:31:34 SGX aesm_service[412134]: [ADMIN]EPID Provisioning protocol error reported by Backend (9)
4月 21 17:31:34 SGX aesm_service[412134]: [ADMIN]EPID Provisioning failed

Also, my CPU is Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz, I wonder whether it supports the EPID remote attestation.

1 reply

llly Apr 23, 2023

Your platform supports EPID attestation.
Intel EPID backend server return TCB out of date error. That's why we still get AESM_UPDATE_AVAILABLE. You need to perform a BIOS update to apply Intel Platform Update (IPU) in order to mitigate security issues.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AESM service returned error 20 #1297

{{title}}

Replies: 3 comments 8 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

AESM service returned error 20 #1297

blank-vax Apr 20, 2023

Replies: 3 comments · 8 replies

kailun-qin Apr 20, 2023 Maintainer

blank-vax Apr 20, 2023 Author

dimakuv Apr 20, 2023

blank-vax Apr 20, 2023 Author

blank-vax Apr 20, 2023 Author

llly Apr 21, 2023

blank-vax Apr 21, 2023 Author

kailun-qin Apr 21, 2023 Maintainer

blank-vax Apr 21, 2023 Author

blank-vax Apr 21, 2023 Author

llly Apr 23, 2023

blank-vax
Apr 20, 2023

Replies: 3 comments 8 replies

kailun-qin
Apr 20, 2023
Maintainer

blank-vax Apr 20, 2023
Author

blank-vax Apr 20, 2023
Author

blank-vax
Apr 20, 2023
Author

blank-vax Apr 21, 2023
Author

kailun-qin Apr 21, 2023
Maintainer

blank-vax Apr 21, 2023
Author

blank-vax
Apr 21, 2023
Author