Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
289 changes: 275 additions & 14 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ rustix = { version = "1.1", features = ["process"] }
socket2 = "0.6"

# Serialization
container-device-interface = { git = "https://github.com/cncf-tags/container-device-interface-rs", rev = "b8a056e92dbd159423c129cd306c93e410651b18" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_yml = "0.0.12"
Expand Down
12 changes: 12 additions & 0 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,18 @@ Resource requirements enter the driver layer through `SandboxSpec.resource_requi
can request a specific number of GPUs or the driver-specific default behaviour.
For all in-tree drivers, this is equivalent to selecting a single GPU.

For Docker GPU sandboxes, the driver treats CDI specs as runtime metadata for
both outer injection and inner sandbox policy. It selects opaque CDI device IDs,
passes them to Docker, mounts daemon-reported CDI spec directories into
supervisor-only paths, and bind-mounts a gateway-owned versioned CDI context
read-only before creating the container. The supervisor resolves that context
inside the sandbox and derives Landlock paths and supplemental groups from CDI
`containerEdits`. Host-side CDI spec paths are diagnostic only and are never
treated as sandbox policy paths.
Kubernetes must not infer CDI device IDs from the `nvidia.com/gpu` resource
request; it needs a node-local selected-device handoff before using the same
supervisor resolver.

VM runtime state paths are derived only from driver-validated sandbox IDs
matching `[A-Za-z0-9._-]{1,128}`. The gateway-owned VM driver socket uses a
private `run/` directory plus Unix peer UID/PID checks. Standalone
Expand Down
18 changes: 14 additions & 4 deletions architecture/security-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,20 @@ dynamic and can be hot-reloaded when the new policy validates successfully.

Before applying Landlock, the supervisor enriches baseline filesystem paths that
the runtime needs. Missing baseline paths are skipped so one absent runtime path
does not weaken the whole ruleset. When GPU devices are present, GPU baseline
enrichment adds existing GPU device nodes as read-write paths and promotes
`/proc` to read-write because CUDA workloads write thread metadata under
`/proc/<pid>/task/<tid>/comm`.
does not weaken the whole ruleset. When GPU devices are present without a CDI
context, GPU baseline enrichment adds existing GPU device nodes as read-write
paths. GPU sandboxes with CDI context use CDI-derived paths instead of the
hard-coded GPU baseline. Both paths promote `/proc` to read-write because CUDA
workloads write thread metadata under `/proc/<pid>/task/<tid>/comm`.

GPU/CDI sandboxes can also carry a supervisor-only CDI context from the compute
driver. The supervisor resolves selected CDI IDs from mounted CDI specs and
adds derived device nodes, library mount destinations, and supplemental GIDs
before agent exec. CDI host paths are ignored for policy. Derived mount
destinations default to read-only; writable CDI single-file mounts require an
exact `filesystem_policy.read_write` opt-in, and writable CDI directory mounts
fail closed. CDI resolution errors are security-relevant startup failures and
emit OCSF findings.

Landlock rules are tailored to the inode type reported by the already-opened
path descriptor. Directories retain the requested directory and file rights;
Expand Down
3 changes: 3 additions & 0 deletions crates/openshell-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ tempfile = { version = "3", optional = true }
[target.'cfg(unix)'.dependencies]
nix = { workspace = true }

[target.'cfg(target_os = "linux")'.dependencies]
container-device-interface = { workspace = true }

[features]
default = ["telemetry"]
## Compile in anonymous telemetry emission support. On by default; disable with
Expand Down
146 changes: 146 additions & 0 deletions crates/openshell-core/src/cdi.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Shared CDI context schema and resolver helpers.

use std::path::{Path, PathBuf};

use serde::{Deserialize, Serialize};

pub const CDI_CONTEXT_VERSION: u32 = 1;

/// File name used for the serialized CDI context.
pub const CDI_CONTEXT_FILE_NAME: &str = "cdi-context.json";

/// Absolute supervisor path for the CDI context file mounted by a compute driver.
pub const CDI_CONTEXT_PATH: &str = "/run/openshell/supervisor/cdi-context.json";

/// Base supervisor path under which compute drivers mount CDI specification directories.
pub const CDI_SPEC_DIR_BASE: &str = "/run/openshell/supervisor/cdi-specs";

/// Return the supervisor path used for a CDI specification directory.
#[must_use]
pub fn cdi_spec_mount_path(index: usize) -> String {
format!("{CDI_SPEC_DIR_BASE}/{index}")
}

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct CdiContext {
pub version: u32,
pub selected_devices: Vec<String>,
pub spec_dirs: Vec<CdiSpecDirectory>,
}

impl CdiContext {
#[must_use]
pub fn new(selected_devices: Vec<String>, spec_dirs: Vec<CdiSpecDirectory>) -> Self {
Self {
version: CDI_CONTEXT_VERSION,
selected_devices,
spec_dirs,
}
}
}

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct CdiSpecDirectory {
pub path: String,
pub source: String,
}

impl CdiSpecDirectory {
#[must_use]
pub fn new(path: impl Into<String>, source: impl Into<String>) -> Self {
Self {
path: path.into(),
source: source.into(),
}
}
}

#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct CdiDerivedRequirements {
pub device_node_paths: Vec<String>,
pub read_only_mount_paths: Vec<String>,
pub read_write_mount_paths: Vec<String>,
pub additional_gids: Vec<u32>,
}

#[derive(Debug, thiserror::Error)]
pub enum CdiError {
#[error("CDI policy resolution is unavailable on this platform")]
UnsupportedPlatform,
#[error("failed to read CDI context '{}': {source}", path.display())]
ContextRead {
path: PathBuf,
source: std::io::Error,
},
#[error("failed to parse CDI context '{}': {source}", path.display())]
ContextParse {
path: PathBuf,
source: serde_json::Error,
},
#[error("unsupported CDI context version {0}")]
UnsupportedContextVersion(u32),
#[error("CDI spec dir '{path}' from source '{diagnostic_source}' is unsafe: {reason}")]
UnsafeSpecDir {
path: String,
diagnostic_source: String,
reason: &'static str,
},
#[error("selected CDI device '{0}' was not found in mounted CDI specs")]
MissingDevice(String),
#[error(
"selected CDI device '{device}' was not found in mounted CDI specs after CDI spec refresh reported: {refresh_error}"
)]
MissingDeviceAfterRefresh {
device: String,
refresh_error: String,
},
#[error("failed to merge CDI edits for '{device}': {error}")]
EditMerge { device: String, error: String },
#[error("failed to encode resolved CDI edits: {source}")]
EditEncode { source: serde_json::Error },
#[error("failed to decode resolved CDI edits: {source}")]
EditDecode { source: serde_json::Error },
#[error("CDI-derived path '{path}' is unsafe: {reason}")]
UnsafePolicyPath { path: String, reason: &'static str },
#[error("CDI path '{path}' requested conflicting access modes")]
ConflictingAccess { path: String },
#[error(
"CDI writable mount '{path}' is not explicitly listed in the sandbox policy read_write paths"
)]
WritableMountNotAllowed { path: String },
#[error("CDI writable mount '{path}' must target a single file, found {kind}")]
WritableMountNotFile { path: String, kind: String },
#[error("CDI device node '{path}' must target a character or block device, found {kind}")]
DeviceNodeNotDevice { path: String, kind: String },
#[error("CDI additionalGids must not contain root GID 0")]
RootAdditionalGid,
#[error("CDI mount '{path}' has conflicting ro/rw options")]
ConflictingMountOptions { path: String },
}

pub fn read_context(path: impl AsRef<Path>) -> Result<CdiContext, CdiError> {
let path = path.as_ref();
let json = std::fs::read_to_string(path).map_err(|source| CdiError::ContextRead {
path: path.to_path_buf(),
source,
})?;
serde_json::from_str(&json).map_err(|source| CdiError::ContextParse {
path: path.to_path_buf(),
source,
})
}

#[cfg(target_os = "linux")]
#[path = "cdi_linux.rs"]
mod cdi_linux;
#[cfg(target_os = "linux")]
pub use cdi_linux::resolve_cdi_context;

#[cfg(not(target_os = "linux"))]
#[path = "cdi_stub.rs"]
mod cdi_stub;
#[cfg(not(target_os = "linux"))]
pub use cdi_stub::resolve_cdi_context;
Loading
Loading