feat(clp-tdl-package): Add the Spider TDL package that registers CLP's compression tasks.

[LinZhihao-723]

Description

This PR depends on #2401.

This PR adds clp-tdl-package, the Spider TDL package that a Spider task executor loads to run CLP's compression work. This PR is the skeleton: the package registers itself, sets up everything the tasks need at load time, and declares the two tasks. Both task bodies are intentionally left unimplemented — they follow in a subsequent PR.

Why a separate crate

The package builds as a cdylib, which the task executor dlopens. That is a different artifact from anything else in the workspace, so it gets its own crate rather than a target inside an existing one. It is also built as an rlib so it can be linked normally by tests.

Package initialization

Spider's TDL supports an init hook that runs once when the package is loaded, before any task is dispatched. package_init uses it to prepare four pieces of process-global state, so that a misconfigured deployment fails at load with a clear message rather than at task time:

• the multi-threaded Tokio runtime the tasks use to drive async S3 I/O
• the Spider task executor config, read from the file named by CLP_CONFIG_PATH
• the CLP home directory, from CLP_HOME
• a tracing subscriber

That last one is not optional. The task executor and this dlopened package each statically link their own copy of tracing, so they have independent global dispatchers. The executor's subscriber is invisible from inside the package, and without installing one here every task-side event hits NoSubscriber and is silently dropped. The subscriber mirrors the executor's format — JSON to stderr, RUST_LOG-driven filter — so both streams interleave readably in the same executor log file.

Each piece is exposed through an accessor that panics if the init hook has not run. Initialization is idempotent, so a repeated load is a no-op rather than an error.

Module layout

src/
  lib.rs                       package registration and the init hook
  common.rs                    process-global state and its accessors
  task/
    compression/
      mod.rs                   the `#[task]` wrappers Spider invokes
      compress.rs              (empty) the clp-s compression worker
      commit.rs                (empty) the commit worker

The #[task] wrappers are kept separate from the workers they will call so that the Spider-facing surface — task names, parameter and return types — stays readable independently of the compression logic.

Config type

SpiderTaskExecutorConfig is added to clp-rust-utils as an empty placeholder. The executor config is a narrower view of CLP's config than Config, and its fields are added in the PR that adds the code which reads them.

Build

task rust now also builds this package. cargo build --bins does not build cdylib targets, so the explicit second invocation is required for the .so to be produced.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Ensure all workflows pass.
• E2e-tested in a dev branch.

Summary by CodeRabbit

• New Features

  • Added initial support for submitting CLP S3 compression jobs to Spider.
  • Added compression job status tracking, including succeeded, failed, and cancelled outcomes.
  • Added configuration and data structures for S3 inputs, compression options, task results, and archive metadata.
  • Registered CLP compression tasks for S3 compression and commit workflows.

• Build & Packaging

  • Added the compression task package and coordinator components to the Rust workspace.
  • Updated release builds to include the compression task package.

[coderabbitai]

Walkthrough

Adds shared S3 compression data types, a Spider TDL package with initialization and task entrypoints, and a compression coordinator crate defining job submission and completion APIs.

Changes

Compression workflow

Layer / File(s)	Summary
Shared compression contracts Cargo.toml, components/clp-rust-utils/src/clp_config/package/config.rs, components/clp-rust-utils/src/task_io/*, components/clp-tdl-package/Cargo.toml, components/compression-coordinator/Cargo.toml, taskfile.yaml	Adds workspace crates, compression protocol structs, executor configuration scaffolding, crate manifests, and a release build command for the TDL package.
TDL package initialization components/clp-tdl-package/src/common.rs, components/clp-tdl-package/src/lib.rs	Adds cached runtime, configuration, CLP home, and tracing initialization, then registers the clp TDL package and its task entrypoints.
Compression task entrypoints components/clp-tdl-package/src/task/*	Adds Spider task wrappers for S3 compression and commit operations; both currently log task information and terminate as unimplemented.
Compression coordinator API components/compression-coordinator/src/*	Adds terminal job outcomes, a public error type, and async contracts for submitting and polling S3 compression jobs.
Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• y-scope/clp#2401: Adds the corresponding compression data types and coordinator job-submission API.
• y-scope/clp#2402: Implements the coordinator submission API for SpiderClient.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding the Spider TDL package for CLP compression tasks.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/clp-tdl-package/src/common.rs`:
- Around line 77-91: Update init_stderr_tracing_subscriber to be idempotent by
treating an already-installed global tracing subscriber as a successful
initialization instead of propagating the try_init failure. Preserve propagation
of other initialization errors so package_init retains meaningful failure
behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c4a7eb4b-1286-46ad-a31f-7506db3f9b25

📥 Commits

Reviewing files that changed from the base of the PR and between e60de7d and 3e82a1e.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (18)

• Cargo.toml
• components/clp-rust-utils/src/clp_config/package/config.rs
• components/clp-rust-utils/src/lib.rs
• components/clp-rust-utils/src/task_io.rs
• components/clp-rust-utils/src/task_io/compression.rs
• components/clp-tdl-package/Cargo.toml
• components/clp-tdl-package/src/common.rs
• components/clp-tdl-package/src/lib.rs
• components/clp-tdl-package/src/task/compression/commit.rs
• components/clp-tdl-package/src/task/compression/compress.rs
• components/clp-tdl-package/src/task/compression/mod.rs
• components/clp-tdl-package/src/task/mod.rs
• components/compression-coordinator/Cargo.toml
• components/compression-coordinator/src/compression_job_submitter/mod.rs
• components/compression-coordinator/src/compression_job_submitter/spider.rs
• components/compression-coordinator/src/error.rs
• components/compression-coordinator/src/lib.rs
• taskfile.yaml