Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@
- 🗂 **Ontology-based organizing** — files classified into an OWL taxonomy you can edit
- 📊 **Disk inventory** — "what is on my disk?", aggregated by category, unknowns surfaced
- 🧠 **On-device LLM advisor** — embedded llama.cpp model judges delete-safety, fully offline
- ☁️ **Metadata-first cloud archive** — detects iCloud Drive, OneDrive, and Google Drive; inspects embedded file metadata, bounded dataset schemas, Rust-parsed ZIP indexes, and incomplete-download archive fragments without extracting payloads; verifies macOS iCloud quota through Apple's read-only native account client and revalidates authoritative OneDrive/Google account capacity through read-only OAuth with a conservative reserve; performs gated copy-plus-hash verification; verifies macOS File Provider status first with native PKCE OAuth checksum plus exact OneDrive path or Google My Drive parent-chain fallback; and distinguishes normal provider-confirmation waits from overdue unconfirmed copies while retaining the source
- ☁️ **Metadata-first cloud archive** — detects iCloud Drive, OneDrive, and Google Drive; inspects embedded file metadata, bounded dataset schemas, Rust-parsed ZIP indexes, and incomplete-download archive fragments without extracting payloads; exports a bounded path-free pre-copy preview contract for semantic-data-portal; verifies macOS iCloud quota through Apple's read-only native account client and revalidates authoritative OneDrive/Google account capacity through read-only OAuth with a conservative reserve; performs gated copy-plus-hash verification; verifies macOS File Provider status first with native PKCE OAuth checksum plus exact OneDrive path or Google My Drive parent-chain fallback; and distinguishes normal provider-confirmation waits from overdue unconfirmed copies while retaining the source

## Safety first

Every destructive action goes through explicit review and the OS trash — DiskSage has **no permanent-delete code path**. Cloud archiving currently exposes copy and evidence only: even a successful provider attestation returns a local-eviction permit without deleting the source. All destructive operations are journaled and undoable.

## Status

🚧 Early development. See the [base design](docs/superpowers/specs/2026-07-10-disksage-design.md), [dataset metadata profile design](docs/superpowers/specs/2026-07-16-dataset-metadata-profile-design.md), [cloud OAuth security design](docs/superpowers/specs/2026-07-16-cloud-provider-oauth-pkce-design.md), [cloud capacity evidence design](docs/superpowers/specs/2026-07-21-cloud-capacity-evidence-design.md), and [redacted Naruon capacity export design](docs/superpowers/specs/2026-07-29-naruon-cloud-capacity-export-design.md).
🚧 Early development. See the [base design](docs/superpowers/specs/2026-07-10-disksage-design.md), [dataset metadata profile design](docs/superpowers/specs/2026-07-16-dataset-metadata-profile-design.md), [cloud OAuth security design](docs/superpowers/specs/2026-07-16-cloud-provider-oauth-pkce-design.md), [cloud capacity evidence design](docs/superpowers/specs/2026-07-21-cloud-capacity-evidence-design.md), [redacted Naruon capacity export design](docs/superpowers/specs/2026-07-29-naruon-cloud-capacity-export-design.md), and [semantic catalog pre-copy export design](docs/superpowers/specs/2026-07-29-semantic-catalog-export-design.md).

## Tech

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Semantic catalog pre-copy export

## Context

DiskSage has rich metadata before a cloud copy and a content-addressed Naruon
lineage envelope after a verified copy. The standards-based file ontology in
semantic-data-portal requires a content SHA-256 and a verified distribution, so
using its persistent `FileAsset` contract before copy would misstate both file
identity and storage evidence.

The integration therefore uses the non-persisting
`POST /file-assets/preview/disksage` contract introduced by
semantic-data-portal PR #31, stacked on its standards-based file ontology PR
#28.

## Contract

`export_semantic_catalog_candidate_batch` emits
`disksage.file-catalog-candidate-batch` version 1. It is bounded to 200
candidates and a 2 MiB pretty-printed JSON body. The Rust output type has no
field for:

- absolute source or destination path;
- source root, relative path, source context, or filename;
- cloud-root path or label;
- provider account, drive, permission, object, or locator identifier.

It retains the stable candidate and review fingerprints, destination provider
and account scope, archive kind, byte count, filesystem timestamps, selected
production time, review state, embedded content metadata, bounded dataset
profile, and metadata evidence. Content title, authors, context, column names,
and evidence values are private metadata: stdout must be sent only to an
approved portal endpoint and must not be committed or copied into public
diagnostic artifacts.

## Metadata lineage invariant

Rust and semantic-data-portal enforce the same fixed production-time
precedence:

1. `embedded_metadata`
2. `explicit_filename_date`
3. `filesystem_created`
4. `filesystem_modified`

The selected timestamp must have matching calendar-date evidence and the exact
source binding. If evidence from a higher-priority class exists, selecting a
lower-priority value is rejected. Every non-embedded selection remains
low-confidence and review-required. A filename date is therefore auxiliary
evidence, never a trusted replacement for embedded metadata.

## CLI

The mutually exclusive read-only output mode is:

```text
disksage-cloud-plan --root /absolute/source \
--cloud-root /absolute/detected/cloud/root \
--min-size-mib 1 \
--min-age-days 0 \
--limit 50 \
--export-semantic-catalog
```

The command creates a fresh single-destination dry-run plan and prints the
bounded batch to stdout. It does not call semantic-data-portal itself, write a
catalog record, call an LLM, copy a file, create a receipt, upload a provider
object, approve a review, hydrate or evict a cloud item, move a source, or
delete anything.

## Portal semantics

The portal validates the structure and returns deterministic proposed
`hasArtifactType` projections. It does not echo private content metadata, call
an LLM, or persist graph/file assets. Its response explicitly keeps
`copy_authorized`, `eviction_authorized`, and `persistable_as_file_asset`
false. A content SHA-256 and verified distribution are still required before
the persistent file-asset contract can be used.

The endpoint records its normal authenticated create-file policy decision as
governance evidence. That evidence is not catalog persistence and not operator
approval.

## Integration choice

The exporter and every selection/size check remain deterministic Rust. Noema,
Gemma, contextual-orchestrator, fast-mlsirm, pg-erd-cloud, and Figma are not
needed for this structural adapter. semantic-data-portal is used because an
ontology/catalog boundary is now real; its PR #28 already carries the
standards and research grounding reused by this stacked integration.
60 changes: 58 additions & 2 deletions src-tauri/src/bin/disksage-cloud-plan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ use disksage_lib::provider_evidence::{self, ProviderSyncEvidenceRecord};
use disksage_lib::provider_oauth;
#[cfg(not(coverage))]
use disksage_lib::provider_sync;
#[cfg(not(coverage))]
use disksage_lib::semantic_catalog;
#[cfg(all(not(coverage), unix))]
use sha2::{Digest, Sha256};

Expand Down Expand Up @@ -81,6 +83,7 @@ struct Args {
export_naruon_lineage: Option<PathBuf>,
naruon_sync_evidence: Option<PathBuf>,
export_naruon_capacity: bool,
export_semantic_catalog: bool,
}

#[cfg(not(coverage))]
Expand Down Expand Up @@ -199,6 +202,7 @@ fn parse_args(args: &[String], home: &Path) -> Result<Args, String> {
export_naruon_lineage: None,
naruon_sync_evidence: None,
export_naruon_capacity: false,
export_semantic_catalog: false,
};
let mut index = 0;
while index < args.len() {
Expand Down Expand Up @@ -391,9 +395,10 @@ fn parse_args(args: &[String], home: &Path) -> Result<Args, String> {
)?))
}
"--export-naruon-capacity" => parsed.export_naruon_capacity = true,
"--export-semantic-catalog" => parsed.export_semantic_catalog = true,
"--help" | "-h" => {
return Err(
"usage: disksage-cloud-plan [--list-roots | --inspect-roots] [--root PATH] [--cloud-root PATH | --provider icloud|onedrive|google-drive | --all-readable-roots --decision-summary] [--min-size-mib N] [--min-age-days N] [--limit N] [--decision-summary [--review-reason-set REASON|REASON [--private-review-output ABSOLUTE_NEW_FILE.json]] | --exact-duplicate-review-prefix DIR_PREFIX --exact-duplicate-kind document|media|archive|dataset|backup|creative|incomplete-download] [--verify-capacity [--oauth-connections ABSOLUTE_PATH] [--export-naruon-capacity]] [--capacity-reserve-mib N] [--copy-fingerprint HEX64 --receipt-dir PATH [--review-dir PATH] [--oauth-connections ABSOLUTE_PATH] | --adopt-existing-fingerprint HEX64 --receipt-dir PATH [--review-dir PATH] | --attest-receipt RECEIPT.json --evidence-dir ABSOLUTE_PATH [--oauth-connections ABSOLUTE_PATH [--provider-object-id GOOGLE_FILE_ID]] | --evict-receipt RECEIPT.json --confirm-receipt-id HEX64 --eviction-dir ABSOLUTE_PATH --eviction-approval-dir ABSOLUTE_PATH --journal-path ABSOLUTE_PATH --evidence-dir ABSOLUTE_PATH --reviewed-by human:ID --review-rationale TEXT [--oauth-connections ABSOLUTE_PATH [--provider-object-id GOOGLE_FILE_ID]] | --review-candidate-fingerprint HEX64 --review-fingerprint HEX64 --review-disposition approved|held --reviewed-by human:ID --review-rationale TEXT --review-dir PATH | --export-naruon-lineage RECEIPT.json [--naruon-sync-evidence EVIDENCE.json]]".into(),
"usage: disksage-cloud-plan [--list-roots | --inspect-roots] [--root PATH] [--cloud-root PATH | --provider icloud|onedrive|google-drive | --all-readable-roots --decision-summary] [--min-size-mib N] [--min-age-days N] [--limit N] [--decision-summary [--review-reason-set REASON|REASON [--private-review-output ABSOLUTE_NEW_FILE.json]] | --exact-duplicate-review-prefix DIR_PREFIX --exact-duplicate-kind document|media|archive|dataset|backup|creative|incomplete-download | --export-semantic-catalog] [--verify-capacity [--oauth-connections ABSOLUTE_PATH] [--export-naruon-capacity]] [--capacity-reserve-mib N] [--copy-fingerprint HEX64 --receipt-dir PATH [--review-dir PATH] [--oauth-connections ABSOLUTE_PATH] | --adopt-existing-fingerprint HEX64 --receipt-dir PATH [--review-dir PATH] | --attest-receipt RECEIPT.json --evidence-dir ABSOLUTE_PATH [--oauth-connections ABSOLUTE_PATH [--provider-object-id GOOGLE_FILE_ID]] | --evict-receipt RECEIPT.json --confirm-receipt-id HEX64 --eviction-dir ABSOLUTE_PATH --eviction-approval-dir ABSOLUTE_PATH --journal-path ABSOLUTE_PATH --evidence-dir ABSOLUTE_PATH --reviewed-by human:ID --review-rationale TEXT [--oauth-connections ABSOLUTE_PATH [--provider-object-id GOOGLE_FILE_ID]] | --review-candidate-fingerprint HEX64 --review-fingerprint HEX64 --review-disposition approved|held --reviewed-by human:ID --review-rationale TEXT --review-dir PATH | --export-naruon-lineage RECEIPT.json [--naruon-sync-evidence EVIDENCE.json]]".into(),
)
}
flag => return Err(format!("알 수 없는 인자: {flag}")),
Expand Down Expand Up @@ -579,7 +584,8 @@ fn validate_action_args(args: &Args) -> Result<(), String> {
+ usize::from(eviction_action)
+ usize::from(review_action)
+ usize::from(args.export_naruon_lineage.is_some())
+ usize::from(args.export_naruon_capacity);
+ usize::from(args.export_naruon_capacity)
+ usize::from(args.export_semantic_catalog);
if args.all_readable_roots && actions > 0 {
return Err(
"--all-readable-roots는 mutation 또는 root inspection과 함께 사용할 수 없음".into(),
Expand Down Expand Up @@ -2022,6 +2028,14 @@ fn run() -> Result<(), String> {
);
return Ok(());
}
if args.export_semantic_catalog {
let batch = semantic_catalog::export_semantic_catalog_candidate_batch(&report)?;
println!(
"{}",
serde_json::to_string_pretty(&batch).map_err(|error| error.to_string())?
);
return Ok(());
}
if let (Some(redundant_prefix), Some(kind)) = (
args.exact_duplicate_review_prefix.as_deref(),
args.exact_duplicate_kind,
Expand Down Expand Up @@ -2250,6 +2264,7 @@ mod tests {
assert!(defaults.review_rationale.is_none());
assert!(defaults.export_naruon_lineage.is_none());
assert!(!defaults.export_naruon_capacity);
assert!(!defaults.export_semantic_catalog);
assert!(defaults.naruon_sync_evidence.is_none());
assert!(!defaults.verify_capacity);
assert!(!defaults.decision_summary);
Expand Down Expand Up @@ -3415,6 +3430,47 @@ mod tests {
assert!(validate_action_args(&multiple).is_err());
}

#[test]
fn semantic_catalog_export_is_single_destination_dry_run_only() {
let export = parse_args(
&[
"--export-semantic-catalog".into(),
"--provider".into(),
"icloud".into(),
],
Path::new("/h"),
)
.unwrap();
assert!(export.export_semantic_catalog);
assert!(validate_action_args(&export).is_ok());

let mut conflicting = export.clone();
conflicting.copy_fingerprint = Some("a".repeat(64));
conflicting.receipt_dir = Some(PathBuf::from("/receipts"));
assert!(validate_action_args(&conflicting).is_err());

let multiple = parse_args(
&[
"--export-semantic-catalog".into(),
"--all-readable-roots".into(),
"--decision-summary".into(),
],
Path::new("/h"),
)
.unwrap();
assert!(validate_action_args(&multiple).is_err());

let summary = parse_args(
&[
"--export-semantic-catalog".into(),
"--decision-summary".into(),
],
Path::new("/h"),
)
.unwrap();
assert!(validate_action_args(&summary).is_err());
}

#[test]
fn action_validation_requires_explicit_complete_eviction_arguments() {
let mut args = parse_args(&[], Path::new("/h")).unwrap();
Expand Down
2 changes: 1 addition & 1 deletion src-tauri/src/cloud.rs
Original file line number Diff line number Diff line change
Expand Up @@ -998,7 +998,7 @@ fn add_filename_publication_month(metadata: &mut ContentMetadata, year: i32, mon
);
}

fn date_value(epoch_ms: u64) -> String {
pub(crate) fn date_value(epoch_ms: u64) -> String {
let (year, month, day) = date_parts(epoch_ms);
format!("{year:04}-{month:02}-{day:02}")
}
Expand Down
1 change: 1 addition & 0 deletions src-tauri/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ pub mod provider_capacity;
pub mod provider_evidence;
pub mod provider_oauth;
pub mod provider_sync;
pub mod semantic_catalog;

// coverage 빌드에서 제외 — GUI 런타임은 헤드리스 테스트로 실행 불가
#[cfg(not(coverage))]
Expand Down
Loading
Loading