Faster parsing of TOML file / caching the result of TOML parsing?

[jogi343]

I am trying to implement a service that accepts a Javascript (nodejs) function from a user and saves it. The user then calls the same function multiple times sending different function parameters each time and the service executes the function inside the enclave using Gramine (the function I am using is similar to this: https://github.com/gramineproject/examples/tree/master/nodejs).

In this scenario, the saved function file never changes. Currently, for every function execution, the TOML file is being parsed which seems to be a time-consuming operation.
(i.e. it takes a relatively long time (about 8 secs) between printing of these two statements in the terminal)

Parsing TOML manifest file, this may take some time...

and

-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:

So I wanted to know if it is possible to somehow speed up this operation, especially, when the function doesn't change for every execution. I have the following questions:

1. Is the long execution time exclusively due to parsing of the TOML file, or the operations that follow also contribute to it?
2. What happens to the result of parsing the TOML? (what exactly is it used for?)
3. Is there any way to cache the result of parsing the TOML file? (especially when the same file is used for subsequent executions)
4. Is it possible to completely disable the TOML file parsing? (e.g. if I ignore attestation etc.)

Thank you in advance.

[boryspoplawski]

1. No, I doubt that it's even a meaningful part of that. The most time is spent initializing the enclave.
2. The file that is parsed is Gramine manifest https://gramine.readthedocs.io/en/stable/manifest-syntax.html
3. No.
4. No.

What you can do, is run all of the functions in the same enclave, if that makes sense security-wise in your scenario.

[jogi343]

Thank you @boryspoplawski for your help.
But how can I access the same enclave (that executed the previous function call) for the next function call as well? Right now, my service runs make SGX=1 check for every new function call from the user.

[boryspoplawski]

First of all, I do not understand completely what is your goal and what you call "a new function" (you mean .js file?). What are you trying to protect with SGX? What is your thread model? How do you get these inputs (functions/files/whatever) from users? What are the tenants and who trusts who?

[jogi343]

Yes, I mean a .js file.
What we are trying to do is that, we have a .js file that only contains a simple secret key generating function. E.g.

function getKey(a, b) {
  ..
  ..
  return secretKey;
} 

The user calls our web-service to call this function (i.e. the .js file) multiple times within a short period by sending different values for parameters a and b. Hence, for every call to our web-service, the user gets a new secret key in return.

We want to hide this computation (i.e. the execution of function getKey) from the OS, and that's why we are thinking of using the enclave. Not sure if enclave is suitable for such a scenario.

In short:
We want to protect: the function execution from the OS
Threat model: OS is untrusted, so hide this computation from the OS
What we get from the user: the function itself and later, the parameters to this function (a user can send these using a web-browser, our web-server (enabled with SGX) accepts these parameters and then executes the key generation function inside the enclave).

(Ideally, we also want to have a common symmetric key between the user and the enclave for decryption of the parameters a and b (as of now, we are not sure if it is possible to implement, and presently parameters are sent without any encryption.))

[boryspoplawski]

How do you plan on verifying that the use actually talks to a real enclave, not some malicious emulated server? And the other way round, how server verifies the user (this step might not be needed, depending on your scenario).
What you might want to use is remote attestation (so that users can verify who they are talking to) and as part of it deploy some secret key, which then can be used for Encrypted Files. I would recommend reading our documentation on other examples e.g. secret provisioning via RA-TLS. Since the last example was recently reworked, I would recommend documentation for the current master branch: https://gramine.readthedocs.io/en/latest/. Note that you would then need to manually build Gramine (but we will soon have a release, so soon these changes will be visible in stable version).

[jogi343]

Thank you for the pointers. Actually I am not really familiar with how the remote attestation works. I will read the documents you mentioned. Maybe I'll ask here again if I get stuck somewhere.

[dimakuv]

In this scenario, the saved function file never changes. Currently, for every function execution, the TOML file is being parsed which seems to be a time-consuming operation.

What actually happens is that there is a new Gramine enclave created every time you perform "function execution". This is clearly suboptimal, as there seems to be no reason to create a new enclave for each invocation. How exactly does your Node.js perform this "function execution"? There seems to be some fork involved.

Parsing TOML manifest file, this may take some time...

Actually, this was a confusing message for many users. After release v1.2, we changed this line to a more meaningful: Gramine is starting. Parsing TOML manifest file, this may take some time.... Notice the "Gramine is starting" prefix. Basically, we want to say that Gramine is starting, and the first thing Gramine is doing is parsing the TOML manifest file (which may or may not take considerable amount of time). But after parsing TOML, Gramine continues with SGX enclave creation and initialization, which does not have its own log message but it typically takes a lot of time.

You could actually see that TOML parsing is not taking that much time, by enabling loader.log_level = "debug" in the manifest file. This will print the Gramine parsed TOML manifest file successfully log message quite quickly, and you'll notice that the actual slow part is SGX-enclave initialization.

The user calls our web-service to call this function (i.e. the .js file) multiple times within a short period ...

Your use case makes sense. But as I mentioned earlier, for some reason your Node.js application creates a new SGX enclave for each getKey() invocation. This is extremely slow. You should analyze why Node.js does this to you, and rewrite your application to not use fork(). For analysis, I would recommend strace -ff <your native node.js app> for a start.

[jogi343]

Thank you @dimakuv for the suggestions.

I ran strace -ff node myfunction.js, it didn't show any fork() calls.

So currently what our server is doing (seems naive) is: as soon as the .js file is received, it creates a directory exactly like this: https://github.com/gramineproject/examples/tree/master/nodejs and saves the function there. We also use the same makefile as shown there in the link. Later, for every call to this function, it runs the command make SGX=1 check (exactly as mentioned in the link) in the same directory.

The makefile internally runs the command gramine-sgx ./nodejs myfunction.js. (myfunction.js file contains the getKey function as shown above). And I think this gramine-sgx is creating a new enclave each time.
So probably I should run my service itself inside an enclave in the first place. And make it run the new function calls in the same enclave. The service being outside the enclave and running gramine-sgx every time is creating a new enclave. Do let me know if I understand something wrongly here.

[dimakuv]

So probably I should run my service itself inside an enclave in the first place. And make it run the new function calls in the same enclave. The service being outside the enclave and running gramine-sgx every time is creating a new enclave. Do let me know if I understand something wrongly here.

Your understanding is exactly correct. Because you're running gramine-sgx ./nodejs myfunction.js every time you want to execute a function, it forces creation of a new Gramine SGX enclave.

So you should run your service itself inside the SGX enclave.