-
Couldn't load subscription status.
- Fork 5
Implement VmConfidentialClient class
#138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement VmConfidentialClient class
#138
Conversation
Solution: Implement `VmConfidentialClient` class to be able to initialize and interact with confidential VMs.
|
Summary: The PR modifies a significant number of files across multiple directories. It includes a refactoring of the 'vmclient.py' file into 'vm_client.py', adds a new 'vm_confidential_client.py' file, and renames several other files. This indicates a high level of complexity and potential impact on the codebase. Highlight: The diff shows the renaming of 'vmclient.py' to 'vm_client.py' and the addition of a new 'vm_confidential_client.py' file. This suggests a significant refactoring and new feature implementation. diff --git a/pyproject.toml b/pyproject.toml
index b52efe66..1070a7f7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -32,6 +32,7 @@ dependencies = [
"python-magic",
"typer",
"typing_extensions",
+ "aioresponses>=0.7.6"
]
[project.optional-dependencies]
diff --git a/src/aleph/sdk/client/vmclient.py b/src/aleph/sdk/client/vm_client.py
similarity index 100%
rename from src/aleph/sdk/client/vmclient.py
rename to src/aleph/sdk/client/vm_client.py
diff --git a/src/aleph/sdk/client/vm_confidential_client.py b/src/aleph/sdk/client/vm_confidential_client.py
new file mode 100644
index 00000000..305ff8ef
--- /dev/null
+++ b/src/aleph/sdk/client/vm_confidential_client.py
@@ -0,0 +1,155 @@
+import json
+import logging
+import tempfile
+from pathlib import Path
+from typing import Any, Dict, Optional, Tuple
+
+import aiohttp
+from aleph_message.models import ItemHash
+
+from aleph.sdk.client.vm_client import VmClient
+from aleph.sdk.types import Account
+from aleph.sdk.utils import run_in_subprocess
+
+logger = logging.getLogger(__name__)
+
+
+class VmConfidentialClient(VmClient):
+ sevctl_path: Path
+
+ def __init__(
+ self,
+ account: Account,
+ sevctl_path: Path,
+ node_url: str = "",
+ session: Optional[aiohttp.ClientSession] = None,
+ ):
+ super().__init__(account, node_url, session)
+ self.sevctl_path = sevctl_path
+
+ async def get_certificates(self) -> Tuple[Optional[int], str]:
+ url = f"{self.node_url}/about/certificates"
+ try:
+ async with self.session.get(url) as response:
+ data = await response.read()
+ with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
+ tmp_file.write(data)
+ return response.status, tmp_file.name
+
+ except aiohttp.ClientError as e:
+ logger.error(
+ f"HTTP error getting node certificates on {self.node_url}: {str(e)}"
+ )
+ return None, str(e)
+
+ async def create_session(
+ self, vm_id: ItemHash, certificate_path: Path, policy: int
+ ):
+ args = [
+ "session",
+ "--name",
+ vm_id,
+ str(certificate_path),
+ str(policy),
+ ]
+ try:
+ await self.sevctl_cmd(args)
+ except Exception as e:
+ raise ValueError(f"Session creation have failed, reason: {str(e)}")
+
+ async def initialize(
+ self, vm_id: ItemHash, session: Path, godh: Path
+ ) -> Tuple[Optional[int], str]:
+ session_file = session.read_bytes()
+ godh_file = godh.read_bytes()
+ params = {
+ "session": session_file,
+ "godh": godh_file,
+ }
+ return await self.perform_confidential_operation(
+ vm_id, "confidential/initialize", params=params
+ )
+
+ async def measurement(self, vm_id: ItemHash) -> Tuple[Optional[int], str]:
+ status, text = await self.perform_confidential_operation(
+ vm_id, "confidential/measurement"
+ )
+ if status:
+ response = json.loads(text)
+ return status, response
+
+ return status, text
+
+ async def validate_measurement(self, vm_id: ItemHash) -> bool:
+ return True
+
+ async def build_secret(
+ self, tek_path: Path, tik_path: Path, measurement: str, secret: str
+ ) -> Tuple[Path, Path]:
+ current_path = Path().cwd()
+ secret_header_path = current_path / "secret_header.bin"
+ secret_payload_path = current_path / "secret_payload.bin"
+ args = [
+ "secret",
+ "build",
+ "--tik",
+ str(tik_path),
+ "--tek",
+ str(tek_path),
+ "--launch-measure-blob",
+ measurement,
+ "--secret",
+ secret,
+ str(secret_header_path),
+ str(secret_payload_path),
+ ]
+ try:
+ await self.sevctl_cmd(args)
+ return secret_header_path, secret_payload_path
+ except Exception as e:
+ raise ValueError(f"Secret building have failed, reason: {str(e)}")
+
+ async def inject_secret(
+ self, vm_id: ItemHash, packed_header: str, secret: str
+ ) -> Tuple[Optional[int], str]:
+ params = {
+ "packed_header": packed_header,
+ "secret": secret,
+ }
+ status, text = await self.perform_confidential_operation(
+ vm_id, "confidential/inject_secret", params=params
+ )
+
+ if status:
+ response = json.loads(text)
+ return status, response
+
+ return status, text
+
+ async def perform_confidential_operation(
+ self, vm_id: ItemHash, operation: str, params: Optional[Dict[str, Any]] = None
+ ) -> Tuple[Optional[int], str]:
+ if not self.pubkey_signature_header:
+ self.pubkey_signature_header = await self._generate_pubkey_signature_header()
+
+ url, header = await self._generate_header(vm_id=vm_id, operation=operation)
+
+ try:
+ async with self.session.post(url, headers=header, data=params) as response:
+ response_text = await response.text()
+ return response.status, response_text
+
+ except aiohttp.ClientError as e:
+ logger.error(f"HTTP error during operation {operation}: {str(e)}")
+ return None, str(e)
+
+ async def sevctl_cmd(self, *args) -> bytes:
+ return await run_in_subprocess(
+ ["sevctl", *args],
+ check=True,
+ )
diff --git a/src/aleph/sdk/utils.py b/src/aleph/sdk/utils.py
index 2d1b30c7..130edc38 100644
--- a/src/aleph/sdk/utils.py
+++ b/src/aleph/sdk/utils.py
@@ -1,8 +1,10 @@
import errno
import hashlib
import json
import logging
import os
+import subprocess
from datetime import date, datetime, time
from enum import Enum
from pathlib import Path
@@ -11,6 +13,7 @@ def sign_vm_control_payload(payload: Dict[str, str], ephemeral_key) -> str:
}
)
return signed_operation
+
+
+async def run_in_subprocess(
+ command: List[str], check: bool = True, stdin_input: Optional[bytes] = None
+) -> bytes:
+ logger.debug(f"command: {' '.join(command)}")
+
+ process = await asyncio.create_subprocess_exec(
+ *command,
+ stdin=asyncio.subprocess.PIPE,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.PIPE,
+ )
+ stdout, stderr = await process.communicate(input=stdin_input)
+
+ if check and process.returncode:
+ logger.error(
+ f"Command failed with error code {process.returncode}:\n"
+ f" stdin = {stdin_input}\n"
+ f" command = {command}\n"
+ f" stdout = {stderr}"
+ )
+ raise subprocess.CalledProcessError(
+ process.returncode, str(command), stderr.decode()
+ )
+
+ return stdoutThe PR introduces significant changes to the codebase that require deep understanding of the project architecture. As such, a 'BLACK' rating is recommended. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
else looks good to me. @Psycojoker what do you thinks ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Else LGTM
Corrections: * Measurement type returned was missing field needed for validation of measurements * Port number was not handled correctly in authentifaction * Adapt to new auth protocol where domain is moved to the operation field (While keeping compat with the old format) * Get measurement was not working since signed with the wrong method * inject_secret was not sending a json * Websocked auth was sending a twice serialized json
c73c82d to
fba8bc1
Compare
fba8bc1 to
dd2639a
Compare
80b3520 to
196fd92
Compare
…ure-implement_confidential_vm_initialization
* Feature: VmClient
* Fix: Protocol (http/https) should not be hardcoded.
One place hardcoded `http://`, the other one `https://`.
* Fix: There was no test for `notify_allocation()`.
* WIP: Copy authentication functions from aleph-vm
* Fix: vm client sessions wasn't close + authentifications for test will use localhost as domain
* Add: Unit test for {perform_operation, stop, reboot, erase, expire}
* Refactor: logs didn't need to generate full header
Fix: extracts domain from node url instead of sending url
Fix: using vmclient sessions in get_logs instead of creating new one
* Add: get_logs test
* Fix: black in aleph_vm_authentification.py
fix: isort issue
Fix: mypy issue
Fix: black
Fix: isort
* Fix: fully remove _generate_header call in get_logs
Fix
Fix: using real path server instead of fake server for test
Fix: create playload
* Fix: black issue
* Fix: test fix workflow
* feat(vm_client): add missing types annotations
* refactor(vm_client): remove duplicated types annotations
* refactor(vm_client): avoid using single letter variable names
* feat(vm_client): increase test_notify_allocation precision
* refactor(vm_client): add empty lines for code readability
* style: run linting:fmt
* Fix: Required an old version of `aleph-message`
* Fix: Newer aleph-message requires InstanceEnvironment
Else tests were breaking.
* Fix: Qemu was not the default hypervisor for instances.
* Fix: Pythom 3.12 fails setup libsecp256k1
When "using bundled libsecp256k1", the setup using `/tmp/venv/bin/hatch run testing:test` fails to proceed on Python 3.12.
That library `secp256k1` has been unmaintained for more than 2 years now (0.14.0, Nov 6, 2021), and seems to not support Python 3.12.
The error in the logs:
```
File "/tmp/pip-build-env-ye8d6ort/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 862, in get_command_obj
cmd_obj = self.command_obj[command] = klass(self)
^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
[end of output]
```
See failing CI run:
https://github.com/aleph-im/aleph-sdk-python/actions/runs/9613634583/job/26516767722
* doc(README): command to launch tests was incorrect
* Refactor: create and sign playload goes to utils and some fix
* Fix: linting issue
* Fix: mypy issue
* fix: black
* feat: use bytes_from_hex where it makes sens
* chore: use ruff new CLI api
* feat: add unit tests for authentication mechanisms of VmClient
* fix: debug code remove
* Update vmclient.py
Co-authored-by: Olivier Le Thanh Duong <[email protected]>
* Fix: update unit test to use stream_logs endpoint instead of logs
* Implement `VmConfidentialClient` class (#138)
* Problem: A user cannot initialize an already created confidential VM.
Solution: Implement `VmConfidentialClient` class to be able to initialize and interact with confidential VMs.
* Problem: Auth was not working
Corrections:
* Measurement type returned was missing field needed for validation of measurements
* Port number was not handled correctly in authentifaction
* Adapt to new auth protocol where domain is moved to the operation field (While keeping compat with the old format)
* Get measurement was not working since signed with the wrong method
* inject_secret was not sending a json
* Websocked auth was sending a twice serialized json
* update 'vendorized' aleph-vm auth file from source
Co-authored-by: Hugo Herter <[email protected]>
Co-authored-by: Laurent Peuch <[email protected]>
Co-authored-by: Olivier Le Thanh Duong <[email protected]>
Co-authored-by: nesitor <[email protected]>
Problem: A user cannot initialize an already created confidential VM.
Solution: Implement
VmConfidentialClientclass to be able to initialize and interact with confidential VMs.