Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Protected env #1667

Merged
merged 2 commits into from
Jul 13, 2020
Merged

Protected env #1667

merged 2 commits into from
Jul 13, 2020

Conversation

mkow
Copy link
Member

@mkow mkow commented Jul 10, 2020

Description of the changes

This is similar to #1562, but for environment variables. It implements a way to pass environment variables from a file, with intention to use it in conjunction with either trusted or protected files.

Fixes #508.

How to test this PR?

Two new tests were added for the new functionality.


This change is Reviewable

Copy link
Contributor

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 18 of 18 files at r1.
Reviewable status: all files reviewed, 11 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @mkow)


Documentation/manifest-syntax.rst, line 90 at r1 (raw file):

By default, environment variables from the host will *not* be passed to the app.
This can be overriden by adding ``loader.insecure__use_host_env = 1``, but most

typo: overridden


Documentation/manifest-syntax.rst, line 90 at r1 (raw file):

By default, environment variables from the host will *not* be passed to the app.
This can be overriden by adding ``loader.insecure__use_host_env = 1``, but most

The insecure__use_host_env manifest option is a bit hidden in this description. Could you move it to a list, like for command-line args, so it is more visible to readers of this document?


Documentation/manifest-syntax.rst, line 109 at r1 (raw file):

environment, which can be generated using :file:`Tools/argv_serializer`. This
option is intended to point to either a trusted file or a protected file. The
former allows to securely hardcode environments (in more flexible way than

in more -> in a more


LibOS/shim/test/regression/test_libos.py, line 80 at r1 (raw file):

            manifest = self.get_manifest('env_from_file')
            stdout, _ = self.run_binary([manifest], env=host_envs)
            self.assertIn('# of envs: %d\n' % (len(envs) + len(manifest_envs)), stdout)

Please also add self.assertNotIn(... host_envs)


Pal/src/db_main.c, line 75 at r1 (raw file):

    ssize_t cfgsize_envs = get_config_entries_size(store, "loader.env");
    if (cfgsize_envs < 0)

Why not <= 0? I guess it's impossible to return 0 in this function, but just for sanity.


Pal/src/db_main.c, line 87 at r1 (raw file):

    }

    setenvs = __alloca(sizeof(struct setenv) * setenvs_cnt);

Why alloca() instead of malloc()? A bit dangerous to allocate on stack.

Actually, I guess it's fine. Given that these are loader.env entries from the manifest, they should be small and won't overflow the main-thread stack.


Pal/src/db_main.c, line 113 at r1 (raw file):

    /* TODO: This code appears to rely on the memory buffer being zero-
     * initialized, so we use calloc here to get zeroed memory. We should
     * audit this code to verify that it's correct. */

I suggest to remove this TODO comment.


Pal/src/db_main.c, line 117 at r1 (raw file):

    if (nenvs)
        memcpy(new_envp, *envpp, sizeof(**envpp) * nenvs);
    

Remove trailing whitespace


Pal/src/db_main.c, line 143 at r1 (raw file):

            *ptr = e;
        }
    }

This whole code is contrived, but it seems to do its job. I understand that refactoring it is not in scope of this PR.


Pal/src/db_main.c, line 479 at r1 (raw file):

    // TODO: Envs from file should be able to override ones from the manifest, but current
    // code makes this hard to implement.

Current code uses an opposite approach: envvars from the manifest override the ones from the file. Is that bad? I think it sounds reasonable. Why this TODO then? Do you want to ideally flip the current logic?


Pal/src/host/Linux-SGX/db_main.c, line 112 at r1 (raw file):

{
    if (len > 7 && key[0] == 'l' && key[1] == 'o' && key[2] == 'a' && key[3] == 'd' &&
        key[4] == 'e' && key[5] == 'r' && key[6] == '.')

Cute!


Pal/src/host/Linux-SGX/sgx-driver, line 1 at r1 (raw file):

Subproject commit fd435f02ff526fd4e473de03797b0bbab95fb748

Why is this here?

Copy link
Member Author

@mkow mkow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 15 of 17 files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: ITL), "fixup! " found in commit messages' one-liners (waiting on @dimakuv)


Documentation/manifest-syntax.rst, line 90 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

typo: overridden

Done.


Documentation/manifest-syntax.rst, line 90 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

The insecure__use_host_env manifest option is a bit hidden in this description. Could you move it to a list, like for command-line args, so it is more visible to readers of this document?

Done.


Documentation/manifest-syntax.rst, line 109 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

in more -> in a more

Done.


LibOS/shim/test/regression/test_libos.py, line 80 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Please also add self.assertNotIn(... host_envs)

Hmm, should I? We check the total count and then whether all non-host envs are there, so we have it guaranteed that there are no host envs on the list.


Pal/src/db_main.c, line 75 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why not <= 0? I guess it's impossible to return 0 in this function, but just for sanity.

Done.


Pal/src/db_main.c, line 87 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why alloca() instead of malloc()? A bit dangerous to allocate on stack.

Actually, I guess it's fine. Given that these are loader.env entries from the manifest, they should be small and won't overflow the main-thread stack.

Yup, it should be fine, so I didn't touched this code. But I think we should change it to something more sane at some point to be able to error-out when the user specifies too many/too big envs, instead of corrupting memory.


Pal/src/db_main.c, line 113 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

I suggest to remove this TODO comment.

Done.


Pal/src/db_main.c, line 117 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Remove trailing whitespace

Done.


Pal/src/db_main.c, line 143 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

This whole code is contrived, but it seems to do its job. I understand that refactoring it is not in scope of this PR.

Yeah, I tried but gave up. It's too obfuscated for me.


Pal/src/db_main.c, line 479 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Current code uses an opposite approach: envvars from the manifest override the ones from the file. Is that bad? I think it sounds reasonable. Why this TODO then? Do you want to ideally flip the current logic?

My logic is that manifest envs are hardcoded, so you can't update them later. File envs are provided in runtime, so it would be good to have an option to override the already-shipped settings from the manifest. The other way doesn't make sense, I think.


Pal/src/host/Linux-SGX/sgx-driver, line 1 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Why is this here?

oops, stupid submodules, sorry :/

@mkow mkow marked this pull request as ready for review July 10, 2020 22:19
dimakuv
dimakuv previously approved these changes Jul 10, 2020
Copy link
Contributor

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r2.
Reviewable status: all files reviewed, all discussions resolved, not enough approvals from maintainers (1 more required), "fixup! " found in commit messages' one-liners


LibOS/shim/test/regression/test_libos.py, line 80 at r1 (raw file):

Previously, mkow (Michał Kowalczyk) wrote…

Hmm, should I? We check the total count and then whether all non-host envs are there, so we have it guaranteed that there are no host envs on the list.

Ok, just wanted to be more explicit. But you're right, no need.


Pal/src/db_main.c, line 479 at r1 (raw file):

Previously, mkow (Michał Kowalczyk) wrote…

My logic is that manifest envs are hardcoded, so you can't update them later. File envs are provided in runtime, so it would be good to have an option to override the already-shipped settings from the manifest. The other way doesn't make sense, I think.

OK, logical.

@mkow
Copy link
Member Author

mkow commented Jul 10, 2020

Jenkins, retest Jenkins-SGX-18.04 please (test_libos.TC_02_OpenMP.test_000_simple_for_loop crashed, probably the same bug as the one fixed in #1663)

@mkow
Copy link
Member Author

mkow commented Jul 10, 2020

Jenkins, retest Jenkins-Debug-18.04 please (apps.LTP.select04 from LTP timed out, interesting, but probably unrelated. I'll try to reproduce this locally)

@mkow
Copy link
Member Author

mkow commented Jul 10, 2020

Ok, apps.LTP.select04 takes over 26s on my machine (it sleeps a lot, so it's not a Graphene-related issue), so no wonder it sometimes times out. I'll submit a PR raising the timeout for this test.

yamahata
yamahata previously approved these changes Jul 13, 2020
Copy link
Contributor

@yamahata yamahata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 15 of 18 files at r1, 2 of 3 files at r2.
Reviewable status: all files reviewed, all discussions resolved, "fixup! " found in commit messages' one-liners

@mkow mkow dismissed stale reviews from yamahata and dimakuv via 7b0c160 July 13, 2020 20:06
Copy link
Contributor

@yamahata yamahata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r3.
Reviewable status: all files reviewed, all discussions resolved, not enough approvals from maintainers (1 more required)

Copy link
Contributor

@dimakuv dimakuv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@mkow mkow merged commit 7b0c160 into master Jul 13, 2020
@mkow mkow deleted the mkow/protected-env branch July 13, 2020 21:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a way to pass verified argv and environments
3 participants