[RFC] "Confidential Model Loader" for model protection with no/low code change

[kenplusplus]

Propose to create an initContainer as "Confidential Model Loader" to archive the model protection with no/low code on existing GenAI microservice.

[hshen14]

1. Will confidential model loader be an option for OPEA, or are you going to make it as default?
2. Do we need to consider the model backward compatible in case the encryption/decryption algorithm might be upgraded?

Overall, the idea looks good, but I guess it would take considerable time/efforts to publish/maintain encrypted in-store models.

[kenplusplus]

1. Will confidential model loader be an option for OPEA, or are you going to make it as default?
   The "confidential model loader" is type of initContainer, following Kubernetes design patten at https://kubernetes.io/docs/concepts/workloads/pods/init-containers/, "initContainer" is like a class's constructor functionality.

I think we should not make it as default. When need deploy the OPEA microservice like meeting summary use case which requiring security considerations, we need add following code in deployment.yaml to enable it.

--- example.yaml using initContainer as the contrustor of microservice's deployment.
spec:
initContainers:
- name: confidential-model-loader
image: confidential-model-loader.image:latest
containers:

• name: myapp-container
  image: busybox:1.28
  command: ['sh', '-c', 'echo The app is running! && sleep 3600']

2. Do we need to consider the model backward compatible in case the encryption/decryption algorithm might be upgraded?

Very nice recommendation! Most AES encryption algorithm will only provide confidentiality protection. AES-GCM (Galois Counter Mode) internally uses AES in CTR mode that can provide confidentiality and at most Ind-CPA security. AES-GCM also uses GHASH to provide integrity and authentication like any secure MAC.

AES-GCM has been widely used in FDE (Full Disk Encryption). But there still are some issue for random IV, so it recommend to use AES-GCM-SIV which is safe from IV-reuse problems. AWS is proposing a new alogirhtm for cloud encryption in NIST at https://csrc.nist.gov/csrc/media/Events/2023/third-workshop-on-block-cipher-modes-of-operation/documents/accepted-papers/Practical%20Challenges%20with%20AES-GCM.pdf

So:

• if the use case's threat model requires "confidentiality", "intregrity" and "authentication", then can choice AES-GCM-SIV
• if the use case's threat model only requires "confidentility", may choice any AES altogirhtms.

Overall, the idea looks good, but I guess it would take considerable time/efforts to publish/maintain encrypted in-store models.

Good suggestion! Will add more worfklow and related utilities design for publishing/maintaining in-store model.

[likebreath]

@kenplusplus Thank you for starting this work.

Would you please convert the PR to a draft for now? The RFC still needs quite some work to be ready, say finishing empty sections, polishing written, and improving diagrams, etc.

We should also quash all changes to a single commit (of course with sign-offs from all contributors), before making it ready for reviews from the OPEA community.

[kenplusplus]

@kenplusplus Thank you for starting this work.

Would you please convert the PR to a draft for now? The RFC still needs quite some work to be ready, say finishing empty sections, polishing written, and improving diagrams, etc.

We should also quash all changes to a single commit (of course with sign-offs from all contributors), before making it ready for reviews from the OPEA community.

Good suggestion! Convert to draft, but please provide detail feedback as soon as possible. @likebreath

[likebreath]

please provide detail feedback as soon as possible. @likebreath

@kenplusplus I just sent a patch to improve the readability and clarity of the RFC. PTAL: kenplusplus#3

Meanwhile, I feel we should further clarify what changes are needed to use the proposed "Confidential Model Loader", both at development time (say code changes) and deployment time (say configuration changes). My personal feel is that readers won't be able to have a tangible idea about how "big" the changes would be based on what we presented here.

[wenhuizhang]

please provide detail feedback as soon as possible. @likebreath

@kenplusplus I just sent a patch to improve the readability and clarity of the RFC. PTAL: kenplusplus#3

Meanwhile, I feel we should further clarify what changes are needed to use the proposed "Confidential Model Loader", both at development time (say code changes) and deployment time (say configuration changes). My personal feel is that readers won't be able to have a tangible idea about how "big" the changes would be based on what we presented here.

IMHO, there will be changes in (1) the model store micro-service, where layers of models are stored in encrypted format instead of plaintext format (2)add a micro-service called Key management service (KMS), where customers and model vendors saves their keys there, preferably, it be a SPIFFEE compatible KMS (3) enhance the agent in model loader with decryption and encryption functions, and also with communication with KMS functions. cc/ @kenplusplus please correct me if something is missing.

[likebreath]

please provide detail feedback as soon as possible. @likebreath

@kenplusplus I just sent a patch to improve the readability and clarity of the RFC. PTAL: kenplusplus#3
Meanwhile, I feel we should further clarify what changes are needed to use the proposed "Confidential Model Loader", both at development time (say code changes) and deployment time (say configuration changes). My personal feel is that readers won't be able to have a tangible idea about how "big" the changes would be based on what we presented here.

IMHO, there will be changes in (1) the model store micro-service, where layers of models are stored in encrypted format instead of plaintext format (2)add a micro-service called Key management service (KMS), where customers and model vendors saves their keys there, preferably, it be a SPIFFEE compatible KMS (3) enhance the agent in model loader with decryption and encryption functions, and also with communication with KMS functions. cc/ @kenplusplus please correct me if something is missing.

@wenhuizhang That's a very good clarification. We should include it in the RFC. Also, we may want to clarify what is in-scope (say helper functions for model decryption using remote attestation) and out-of-scope of this RFC (say encrypted model registry and the KMS micro-service mentioned above).

/cc @kenplusplus