Reorganize Graphene components when moving to new GitHub organization

[dimakuv]

1. Repository split

When moving to a new GitHub organization, we should split Graphene into several different repositories. Repositories should be self-sufficient because we don't want to use git submodules (we had bad experiences with maintaining submodules). The proposed repositories are:

1. graphene -- contains the core; this is the main repository, successor to the current repo.
2. examples -- contains all examples not tested per commit (in our Jenkins CI) but tested per release.
3. contrib -- contains all other examples (contributed but not necessarily periodically tested), scripts, tutorials, etc.
4. gsc -- contains Graphene Shielded Containers (GSC) plus all GSC examples and corresponding scripts.

Rationale (open for discussion):

• Our examples are divided into three categories: the "fast and useful" ones, the "important but slow" ones, and "everything else". The first-category examples land in the core graphene repo and must be tested in our Jenkins CI (with every PR, with every commit). The second-category examples land in the examples repo and must be tested with each new release (maybe manually, maybe with some other CI). The "everything else" examples land in the contrib repo -- maintainers may review and test them once, but these examples may get stale or unstable. See the current split of examples below.

• Documentation is split among these repositories. The core graphene repo keeps the bulk of our current documentation (with its own readthedocs website). The examples repo doesn't need documentation, so no separate website (all documentation for each example is stored in its README). The contrib repo keeps all the contributed documentation -- mainly tutorials, blogs, how-tos, etc. (with its own readthedocs website). The gsc repo keeps the documentation on GSC (with its own readthedocs website -- open for discussion).

• The GSC project is moved into its own repo. GSC is not a part of Graphene per-se. On the contrary, GSC uses Graphene. GSC doesn't need to track the latest Graphene version. Indeed, GSC development is largely independent from core Graphene development (except for breaking changes in the manifest syntax). Also, GSC is big enough and important enough to merit its own repo.

• Small tools, optional libraries and utilities (is_sgx_available, pf_crypt, ra-tls, argv_serializer) stay in the core graphene repo. Most of these tools are tied to the internals of Graphene (e.g., pf_crypt, ra-tls, argv_serializer) and thus must be updated and tested together with core Graphene. Some of these tools are rather independent and SGX-specific (ias-request, quote-dump), but they use internals of Graphene (they share code with e.g. ra-tls) and thus must be updated/tested together with core Graphene. Finally, there is one tool (is_sgx_available) that is completely independent of everything and could be moved to a separate repository, but it sounds like an overkill to create a whole new repository just for one small utility (open for discussion).

2. Split of examples

(See discussion in #2493.)

The following examples stay in the core graphene repo:

• Bash
• Blender
• Busybox
• Google Go (when added)
• lighttpd
• memcached
• nginx
• python (but with venv?)
• ra-tls-mbedtls
• ra-tls-secret-prov
• redis
• SQLite

The following examples are moved into the examples repo:

• apache (currently not working due to missing Sys-V in Graphene)
• Curl
• GCC
• Java (when added)
• Node.js
• OpenVINO
• PyTorch
• R
• TensorFlow-Lite
• TensorFlow

The following examples are moved into the contrib repo:

• [ nothing here ]

The following examples are completely removed as not used by anyone:

• Capnproto
• nodejs-express-server

3. Repo graphene

The core graphene repo is the successor to our current repo and keeps the same structure. Below is the repo layout with some proposed changes:

• .ci
• Documentation
  • Remove man page for GSC from here; move to GSC repo
  • Remove oldwiki
• Examples (changes described in the previous section)
• LibOS
• Pal
  • Move RA-TLS from under Pal/src/host/Linux-SGX/tools/ to Pal/src/host/Linux-SGX/ (because there will be no tools/, see below)
• Runtime (this will probably get removed completely after Meson switch)
• Scripts
• Tools
  • Remove gsc; move to GSC repo
  • benchmark-http.sh (move from Examples/common_tools/) -- open for discussion
  • argv_serialize (move from current root dir under this new dir)
  • ias-request (move from Pal/src/host/Linux-SGX/tools/)
  • is-sgx-available (move from Pal/src/host/Linux-SGX/tools/)
  • pf_crypt (move from Pal/src/host/Linux-SGX/tools/)
  • pf_tamper (move from Pal/src/host/Linux-SGX/tools/)
  • quote-dump (move from Pal/src/host/Linux-SGX/tools/)
  • verify-ias-report (move from Pal/src/host/Linux-SGX/tools/)

This repo must be created with new, rebased commit history. The new commit history is exactly the same but doesn't contain the commits before the cutoff commit below. Rationale: commit history before the cutoff commit (10. June 2019) looks ad-hoc and unhelpful (contains many merge commits and meaningless ones like Update my email). Additional rationale: before the cutoff commit, Graphene shipped binary blobs for GCC, Apache, etc. which bloated Graphene repository size to ~300MB.

The proposed cutoff commit: 7262d60.

Note that some of the layout changes may be done right now in the current repo, or after the migration in the new repo. The exact timing for these changes is not important.

4. Repo examples

This repo is created from scratch. Can immediately contain the example apps mentioned in section 2. Below is the repo layout:

• apache
• curl
• gcc
• etc.

5. Repo contrib

This repo is migrated from our current https://github.com/oscarlab/graphene-contrib. Its corresponding documentation website is migrated from our current https://graphene-contrib.readthedocs.io.

6. Repo gsc

This repo contains GSC that is moved out of the current core repo. Below is the repo layout:

• .ci (would be nice to have a CI for GSC working again!)
• Documentation (contains the current man page for GSC and probably a couple other pages with QuickStart and Usage)
• Examples (contains examples that are reworked from the current https://github.com/oscarlab/graphene/tree/master/Tools/gsc/test; the layout is similar to our Examples in the core repo)
• Everything that is currently under https://github.com/oscarlab/graphene/tree/master/Tools/gsc (other than test)

It would be extremely hard to cherry-pick the GSC-related commits from the core repo, so we simply create a repo from scratch and the initial commit should already contain the current snapshot of GSC.

Obviously, the new gsc repo must be created after the core graphene repo because it will contain links to this core graphene repo.

7. Moving current issues and PRs

The hope is to be able to migrate all open PRs and issues as-is to the new core repo (some PRs and issues will be closed in our old repo and must be re-created from scratch in examples, contrib, gsc repos). Open for discussion.

[mkow]

which bloated Graphene repository size to ~30MB.

You missed one zero here :)

Ad. the squash: We should credit all the contributors whose commits we squashed in the new commit message. Not sure how to exactly do this, but we could generate some stats from git or something similar.

Overall the plan looks good, thanks for writing it down here!

[dimakuv]

Change: Having graphene/Examples and a separate examples repo is confusing to outsiders. We should rename them as follows (decided during the meeting): graphene/CI-Examples and examples repo.

[woju]

#895

[woju]

7. Moving current issues and PRs

You can only transfer issues to another repo owned by the same user/orga (https://docs.github.com/en/issues/tracking-your-work-with-issues/transferring-an-issue-to-another-repository). So this need to be done in such a way:

1. create new repo in oscarlab/
2. move issues to the repo
3. move repo to new orga

And I might have missed something, I didn't test this.

[dimakuv]

We have ./Documentation/graphene_logo.svg (Graphene logo). We need a new one to finalize the renaming from Graphene to Gramine. (The rest of the renaming is done in the PR.)

[dimakuv]

We now have three public repos:

• https://github.com/gramineproject/gramine/
• https://github.com/gramineproject/gsc
• https://github.com/gramineproject/examples/

The order of reviewing and merging PRs:

1. Add README and gitignore gsc#6 -- Add README and gitignore
2. Add documentation gsc#8 -- Add documentation
3. [GSC] Remove GSC tool and its documentation from this repository gramine#3 -- [GSC] Remove GSC tool and its documentation from this repository
4. Add README and gitignore examples#3 -- Add README and gitignore
5. [Examples] Remove big examples from this repository gramine#4 -- [Examples] Remove big examples from this repository

This brings us to the state of "old codebase" but split across three new repos. Then we need to clean up GSC:

1. Remove stale tests and corresponding Makefile gsc#7 -- Remove stale tests and corresponding Makefile
2. Remove stale AKS-image Dockerfile and corresponding Makefile gsc#9 -- Remove stale AKS-image Dockerfile and corresponding Makefile

At this point we are ready to rename everything from Graphene to Gramine:

1. Rename Graphene to Gramine gramine#17
2. Replace Graphene with Gramine everywhere gsc#10
3. Replace Graphene with Gramine everywhere examples#4