This repository was archived by the owner on Jul 30, 2026. It is now read-only.
fix(core): make local source attribution structural - #194
Merged
makoMakoGo merged 2 commits intoJul 25, 2026
Conversation
Store parser and cache messages without client identity, then inject ClientId only at adapter emission. Derive Overview Data Size from the confirmed per-client inventory and validate the invariant in TUI cache boundaries.
Walkthrough本次变更将解析消息、输入适配器和缓存统一迁移到来源中立的 Changes来源中立消息与适配器管线
输入足迹与 TUI 报告
架构与验证
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant SessionParser
participant InputMessageCache
participant AdapterBinding
participant BoundMessageSink
participant ReportAndTUI
SessionParser->>InputMessageCache: 写入来源中立 ParsedMessage
InputMessageCache-->>AdapterBinding: 返回 ParsedMessage 与 DecoderVersion
AdapterBinding->>BoundMessageSink: 绑定 ClientId 并发射 UnifiedMessage
BoundMessageSink->>ReportAndTUI: 汇总消息与 InputFootprint
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
crates/tokscale-core/src/lib_tests.rs (1)
337-362: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win建议保留“每客户端内去重”的回归覆盖。
新测试只断言
health.input_data_bytes()等于client_space总和；而被删除的input_data_size_by_client_deduplicates_within_each_client覆盖的是仍然保留的契约（input_data_bytes仍按ClientId分组去重，HealthReport文档也仍如此声明）。建议补一个针对同一客户端内重复文件 identity 的用例，避免去重语义在后续重构中静默退化。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/lib_tests.rs` around lines 337 - 362, 在现有测试附近恢复一个针对同一客户端重复文件 identity 的回归测试，覆盖 input_data_bytes 按 ClientId 分组去重的契约。可复用被删除的 input_data_size_by_client_deduplicates_within_each_client 测试所使用的 fixture、输入构造和断言方式，确保同一客户端内重复文件不会重复计入，而不同客户端的计数行为保持不变。Source: Learnings
crates/tokscale-core/src/adapters/codex.rs (1)
900-926: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win将
parser_messages对齐到 finalization 测试模式。Codex 的折叠路径在
finalization分支会调用finalize_codex_messages(&mut messages, None),而该助手又调用finalize_token_priced_messages;当前解析预期值只调用refresh_derived_fields()后再.attribute(ClientId::Codex),缺少模型/供应商标识归一化和 token 规格化。建议仿照junie.rs、omp.rs、codebuddy.rs建立 finalization helper，避免未来别名或 pricing 逻辑变化时覆盖不了生产路径。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/codex.rs` around lines 900 - 926, 将测试辅助流程与生产的 finalization 路径对齐：在 Codex 测试中建立类似 junie.rs、omp.rs 和 codebuddy.rs 的 finalization helper，调用 finalize_codex_messages(&mut messages, None)，由其完成模型/供应商归一化及 token 规格化。更新 parser_messages 使用该 helper，保留后续 assert_output_matches_parser 中的 ClientId::Codex attribution。Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@crates/tokscale-core/src/adapters/codex.rs`:
- Around line 900-926: 将测试辅助流程与生产的 finalization 路径对齐：在 Codex 测试中建立类似
junie.rs、omp.rs 和 codebuddy.rs 的 finalization helper，调用
finalize_codex_messages(&mut messages, None)，由其完成模型/供应商归一化及 token 规格化。更新
parser_messages 使用该 helper，保留后续 assert_output_matches_parser 中的 ClientId::Codex
attribution。
In `@crates/tokscale-core/src/lib_tests.rs`:
- Around line 337-362: 在现有测试附近恢复一个针对同一客户端重复文件 identity 的回归测试，覆盖 input_data_bytes
按 ClientId 分组去重的契约。可复用被删除的
input_data_size_by_client_deduplicates_within_each_client 测试所使用的
fixture、输入构造和断言方式，确保同一客户端内重复文件不会重复计入，而不同客户端的计数行为保持不变。
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3889ef87-cfb9-463d-ad07-6fff0d8bfe21
📒 Files selected for processing (55)
crates/tokscale-cli/src/tui/cache.rscrates/tokscale-cli/src/tui/mod.rscrates/tokscale-core/src/adapters/antigravity.rscrates/tokscale-core/src/adapters/cache.rscrates/tokscale-core/src/adapters/claude.rscrates/tokscale-core/src/adapters/codebuddy.rscrates/tokscale-core/src/adapters/codex.rscrates/tokscale-core/src/adapters/file.rscrates/tokscale-core/src/adapters/goose.rscrates/tokscale-core/src/adapters/hermes.rscrates/tokscale-core/src/adapters/junie.rscrates/tokscale-core/src/adapters/kilo.rscrates/tokscale-core/src/adapters/kiro.rscrates/tokscale-core/src/adapters/mod.rscrates/tokscale-core/src/adapters/omp.rscrates/tokscale-core/src/adapters/opencode.rscrates/tokscale-core/src/adapters/pi.rscrates/tokscale-core/src/adapters/warp.rscrates/tokscale-core/src/adapters/zed.rscrates/tokscale-core/src/input_health.rscrates/tokscale-core/src/lib.rscrates/tokscale-core/src/lib_tests.rscrates/tokscale-core/src/message_cache.rscrates/tokscale-core/src/sessions/amp.rscrates/tokscale-core/src/sessions/antigravity_cli.rscrates/tokscale-core/src/sessions/claudecode.rscrates/tokscale-core/src/sessions/cline.rscrates/tokscale-core/src/sessions/codebuddy.rscrates/tokscale-core/src/sessions/codebuff.rscrates/tokscale-core/src/sessions/codex.rscrates/tokscale-core/src/sessions/commandcode.rscrates/tokscale-core/src/sessions/copilot.rscrates/tokscale-core/src/sessions/droid.rscrates/tokscale-core/src/sessions/gemini.rscrates/tokscale-core/src/sessions/goose.rscrates/tokscale-core/src/sessions/grok.rscrates/tokscale-core/src/sessions/hermes.rscrates/tokscale-core/src/sessions/junie.rscrates/tokscale-core/src/sessions/kilo.rscrates/tokscale-core/src/sessions/kimi.rscrates/tokscale-core/src/sessions/kiro.rscrates/tokscale-core/src/sessions/mod.rscrates/tokscale-core/src/sessions/mux.rscrates/tokscale-core/src/sessions/openclaw.rscrates/tokscale-core/src/sessions/opencode.rscrates/tokscale-core/src/sessions/pi.rscrates/tokscale-core/src/sessions/qwen.rscrates/tokscale-core/src/sessions/roocode.rscrates/tokscale-core/src/sessions/warp.rscrates/tokscale-core/src/sessions/zcode.rscrates/tokscale-core/src/sessions/zed.rscrates/tokscale-core/tests/codebuff.rscrates/tokscale-core/tests/hermes.rsdocs/adr/0007-client-identity-catalog.mddocs/adr/0008-single-copy-memory-pipeline.md
💤 Files with no reviewable changes (2)
- crates/tokscale-core/tests/codebuff.rs
- crates/tokscale-core/tests/hermes.rs
Keep inputs, decoder output, cache shards, and pipeline errors source-neutral; attach ClientId only through AdapterBinding. Use typed client identities through aggregation and derive all input-size views from one confirmed InputFootprint.
makoMakoGo
changed the title
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (14)
crates/tokscale-cli/src/tui/cache.rs (1)
130-134: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
health_for_input_footprint已不再依赖入参，建议直接删除该辅助函数。参数被忽略后它等价于
HealthReport::default()，保留反而暗示健康报告仍与输入足迹相关。可将所有调用点替换为HealthReport::default()。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-cli/src/tui/cache.rs` around lines 130 - 134, Remove the unused health_for_input_footprint helper and replace every call to it with tokscale_core::input_health::HealthReport::default(). Ensure no callers or references to the helper remain.crates/tokscale-core/src/lib.rs (1)
324-361: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value输入字节溢出使用
expect会在库路径上 panic，可考虑降级为可诊断错误。
add_bytes的错误类型已建模为InputFootprintOverflow，但这里直接expect。实际溢出不可达（磁盘总量远小于u64::MAX），因此优先级不高；若希望保持库无 panic 语义，可让这两个函数返回Result并映射为LocalReportError::operational。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/lib.rs` around lines 324 - 361, Remove the panic paths in prepared_input_footprint and confirmed_input_footprint by returning a Result and propagating InputFootprintOverflow instead of calling expect on add_bytes. Map the overflow to LocalReportError::operational at the reporting boundary, and update callers to propagate the resulting error while preserving existing footprint calculations.crates/tokscale-cli/src/tui/mod.rs (1)
1132-1151: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value测试夹具仍以字符串客户端键为入口，未跟随本次强类型化。 两处工厂都接收
BTreeMap<String, u64>，再用ClientId::from_str(...).expect("test client must be canonical")反解成InputFootprint；调用点传的全是字面量或ClientId::X.as_str().to_string()，白绕一圈还引入运行期失败点。
crates/tokscale-cli/src/tui/mod.rs#L1132-L1151：把loaded_snapshot的client_bytes参数改为impl IntoIterator<Item = (ClientId, u64)>，直接交给InputFootprint::from_client_bytes，删除from_str+expect。crates/tokscale-cli/src/tui/ui/mod.rs#L277-L292：同样改造install_generation，调用点用[(ClientId::Junie, 0)]代替BTreeMap::from([(ClientId::Junie.as_str().to_string(), 0)])。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-cli/src/tui/mod.rs` around lines 1132 - 1151, Update loaded_snapshot in crates/tokscale-cli/src/tui/mod.rs:1132-1151 to accept impl IntoIterator<Item = (ClientId, u64)> and pass it directly to InputFootprint::from_client_bytes, removing the from_str conversion and expect. Apply the same change to install_generation in crates/tokscale-cli/src/tui/ui/mod.rs:277-292, and update its call sites to use [(ClientId::Junie, 0)] instead of string-keyed BTreeMap values.crates/tokscale-core/build.rs (2)
187-197: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win建议为
Ord的字符串序补一行意图注释。
Ord按as_str()字典序而非repr(usize)判别值排序，这与代码库其他处的sort_by_key(|c| *c as usize)顺序不同。聚合投影（materialize_daily_common/materialize_hourly_common/project_common_selected的sort_by_key）依赖这个顺序来固定浮点累加次序，改成判别值顺序会静默改变累加结果。生成代码里加一句注释可以避免后来者把它“优化”掉。♻️ 建议补充注释
+// Ordering follows the canonical string id, not the `repr(usize)` discriminant: +// aggregation folds rely on it for deterministic floating-point accumulation. impl Ord for ClientId {{ fn cmp(&self, other: &Self) -> std::cmp::Ordering {{ self.as_str().cmp(other.as_str()) }} }}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/build.rs` around lines 187 - 197, 在 ClientId 的 Ord::cmp 实现中补充一行意图注释，明确必须按 as_str() 的字典序排序，而不是按 repr(usize) 判别值排序，因为聚合投影依赖该顺序稳定浮点累加结果。仅添加这项说明，保留现有 cmp 实现不变。
214-223: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value反序列化可避免每个客户端 id 都分配
String。缓存/报告反序列化会为每条按客户端分桶的记录解析一次 id，用
Cow<'de, str>在 JSON 无转义时可零拷贝，行为与错误信息保持不变。♻️ 建议改动
- let id = <String as serde::Deserialize>::deserialize(deserializer)?; + let id = <std::borrow::Cow<'de, str> as serde::Deserialize>::deserialize(deserializer)?; Self::from_str(&id) .ok_or_else(|| serde::de::Error::custom(format!("unknown local client `{{id}}`")))🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/build.rs` around lines 214 - 223, 更新 ClientId 的 Deserialize::deserialize 实现，使用 Cow<'de, str> 反序列化输入，确保未转义的 JSON client id 可借用而无需分配 String。将借用内容传递给 ClientId::from_str，并保持未知 id 的错误信息和现有行为不变。crates/tokscale-core/src/lib_tests.rs (1)
2139-2187: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
decoder_changed与canonical_clients的比较存在混淆变量，未能真正验证 decoder revision 的影响。
canonical_clients使用requested_clients = [Amp, Zed]，而decoder_changed使用requested_clients = [Amp]——两者的客户端列表本身就不同，因此assert_ne!恒为真，与 decoder revision（0 → 999）是否生效无关。如果未来DecoderSpec的 revision 字段不再参与签名计算，这个测试也不会捕获该回归，掩盖了它声称验证的契约。建议让
decoder_changed复用与canonical_clients相同的requested_clients，只改变 decoder revision，以真正隔离该变量。♻️ 建议修复
let decoder_changed = signature_for_test_units( - &[ClientId::Amp], + &[ClientId::Amp, ClientId::Zed], ClientId::Amp, vec![crate::adapters::InputUnit::plain_file( first, crate::adapters::DecoderSpec::plain(message_cache::DecoderId::Amp, 999), )], ); assert_ne!(canonical_clients, decoder_changed);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/lib_tests.rs` around lines 2139 - 2187, Update the decoder_changed signature test to use the same requested_clients list as canonical_clients, changing only the DecoderSpec revision from the baseline value to 999. Keep the existing decoder and input setup otherwise unchanged so the final assert_ne! isolates and verifies the decoder revision’s effect on the signature.crates/tokscale-core/src/adapters/kiro.rs (2)
252-256: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win删除已无用的
_client形参。
InputDiscoveryError::new不再携带 client 后，该参数彻底未被使用，只会让调用方（第 60 行）误以为去重逻辑与客户端相关。♻️ 建议的重构
-fn dedup_units_by_canonical_path( - _client: ClientId, - units: &mut Vec<InputUnit>, -) -> Result<(), InputDiscoveryError> { +fn dedup_units_by_canonical_path(units: &mut Vec<InputUnit>) -> Result<(), InputDiscoveryError> {同步调整第 60 行调用：
- dedup_units_by_canonical_path(client, &mut units)?; + dedup_units_by_canonical_path(&mut units)?;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/kiro.rs` around lines 252 - 256, Remove the unused _client parameter from dedup_units_by_canonical_path and update its call site in the surrounding discovery flow to pass only the units argument. Keep the deduplication behavior unchanged.
68-87: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value合并重复分支并更新陈旧的
meta文案。
KiroFile与KiroGlobalStorage两个分支体完全一致；同时unreachable!文案仍称 "input unit meta"，而判别依据已改为decoder.route()（第 102 行的plan_cache_hit同样如此）。♻️ 建议的重构
.map(|unit| match unit.decoder.route() { - DecoderRoute::KiroFile => adapter_cache::load_or_scan_unit_with( - unit, - ctx, - sessions::kiro::parse_kiro_file, - ), + DecoderRoute::KiroFile | DecoderRoute::KiroGlobalStorage => { + adapter_cache::load_or_scan_unit_with(unit, ctx, sessions::kiro::parse_kiro_file) + } DecoderRoute::KiroSqlite => { adapter_cache::parse_uncached_unit(unit, ctx, sessions::kiro::parse_kiro_sqlite) } - DecoderRoute::KiroGlobalStorage => adapter_cache::load_or_scan_unit_with( - unit, - ctx, - sessions::kiro::parse_kiro_file, - ), DecoderRoute::None | DecoderRoute::AntigravityCliSqlite | DecoderRoute::OpenCodeSqlite | DecoderRoute::CodeBuddyJsonl | DecoderRoute::CodeBuddyExtensionLog { .. } - | DecoderRoute::Codex => unreachable!("unexpected Kiro input unit meta"), + | DecoderRoute::Codex => unreachable!("unexpected Kiro decoder route"), })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/kiro.rs` around lines 68 - 87, 在处理 DecoderRoute 的 match 中合并 KiroFile 与 KiroGlobalStorage 分支，复用同一个 load_or_scan_unit_with 调用；同时更新该分支的 unreachable! 提示文本，将过时的 “input unit meta” 改为反映 decoder.route() 判别依据的文案，并同步检查 plan_cache_hit 相关提示。crates/tokscale-core/src/adapters/claude.rs (1)
242-276: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win建议改用
adapter_cache::fold_units_with_filter承载去重。这里手写的
record_health→write_cache→ 失败/未计划时remove流程与adapter_cache::fold_units_with_filter完全同构，唯一差异是去重过滤。hermes.rs已经用该辅助表达同样的去重语义；保留两份等价的缓存失效流程，后续修改fold_units_with_filter时容易与本处漂移。♻️ 建议的重构
fn fold_claude_units( parsed: Vec<ParsedUnit>, ctx: &mut FoldContext<'_>, sink: &mut BoundMessageSink<'_>, seen_keys: &mut HashSet<u64>, ) -> Result<(), crate::adapters::InputPipelineError> { - for parsed_unit in parsed { - let adapter_cache::ResolvedUnit { - unit, - messages, - cache_write, - invalidate_cache, - status, - rejections, - } = adapter_cache::resolve_unit(parsed_unit, ctx)?; - ctx.record_health(unit.path.clone(), status, rejections); - let path = unit.path.clone(); - let cache_write_outcome = adapter_cache::write_cache(cache_write, ctx, &messages); - if cache_write_outcome.is_err() && invalidate_cache { - ctx.input_cache.remove(&path, unit.decoder.version()); - } - let cache_write_outcome = cache_write_outcome?; - adapter_cache::emit_messages( - messages - .into_iter() - .filter(|message| crate::should_keep_deduped_message(seen_keys, message)), - sink, - ); - - if cache_write_outcome == adapter_cache::CacheWriteOutcome::NotPlanned && invalidate_cache { - ctx.input_cache.remove(&path, unit.decoder.version()); - } - } - Ok(()) + adapter_cache::fold_units_with_filter(parsed, ctx, sink, |_, messages| { + messages + .into_iter() + .filter(|message| crate::should_keep_deduped_message(seen_keys, message)) + .collect() + }) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/claude.rs` around lines 242 - 276, 重构 fold_claude_units，改用 adapter_cache::fold_units_with_filter 承载 record_health、缓存写入及失败或未计划时的失效处理，并将现有去重逻辑作为其过滤器传入。移除该函数中重复的 resolve、缓存处理和 emit_messages 流程，保持 should_keep_deduped_message 基于 seen_keys 的去重语义不变。crates/tokscale-core/src/message_cache.rs (1)
262-349: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win为
stable_name唯一性加一道防护。
stable_name同时参与from_stable_name的反查和shard_key_for_input_key的哈希（第 1849 行）。若将来两个变体不小心写了同一字面量，from_stable_name会静默映射到match中的第一个匹配，且两个 decoder 会共享同一分片键——warm hit 可能返回另一个 decoder 的缓存消息。当前 32 个名称确实互不重复，但宏本身没有任何编译期或测试期约束。建议让宏顺带导出全量名称，并补一条唯一性断言。🛡️ 建议的防护
impl DecoderId { + #[cfg(test)] + pub(crate) const ALL_STABLE_NAMES: &'static [&'static str] = &[$($stable_name),+]; + pub(crate) const fn stable_name(self) -> &'static str {配套测试：
#[test] fn decoder_stable_names_are_unique() { let unique: std::collections::HashSet<_> = DecoderId::ALL_STABLE_NAMES.iter().collect(); assert_eq!( unique.len(), DecoderId::ALL_STABLE_NAMES.len(), "duplicate decoder stable names would collide shard keys" ); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/message_cache.rs` around lines 262 - 349, Extend define_decoder_ids! to expose all configured stable names through DecoderId::ALL_STABLE_NAMES, preserving the existing macro-generated mappings. Add a decoder_stable_names_are_unique test that collects these names into a HashSet and asserts the unique count matches the array length, so duplicate stable_name literals fail tests.crates/tokscale-core/src/adapters/discover.rs (1)
48-49: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win建议移除已失效的
client形参。
scan_roots、push_existing_file、canonical_key的client参数在本次重构后已完全不参与逻辑（错误构造也不再使用它），仅以_client保留，导致所有调用方仍需传递无意义的实参。既然InputUnit已不携带ClientId，这些签名可以一并收敛。♻️ 建议的签名简化
-pub(crate) fn scan_roots<I>( - _client: ClientId, - roots: I, - pattern: &str, -) -> Result<Vec<PathBuf>, InputDiscoveryError> +pub(crate) fn scan_roots<I>(roots: I, pattern: &str) -> Result<Vec<PathBuf>, InputDiscoveryError>-pub(crate) fn push_existing_file( - _client: ClientId, - path: PathBuf, - paths: &mut Vec<PathBuf>, -) -> Result<(), InputDiscoveryError> { +pub(crate) fn push_existing_file( + path: PathBuf, + paths: &mut Vec<PathBuf>, +) -> Result<(), InputDiscoveryError> {-fn canonical_key(_client: ClientId, path: &Path) -> Result<PathBuf, InputDiscoveryError> { +fn canonical_key(path: &Path) -> Result<PathBuf, InputDiscoveryError> {需要同步更新
input_units_from_paths/input_units_from_paths_preserving_order及各适配器调用点，可作为独立的收尾提交。Also applies to: 105-106, 134-134
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/discover.rs` around lines 48 - 49, 移除 scan_roots、push_existing_file 和 canonical_key 中不再使用的 client 参数，并同步调整 input_units_from_paths、input_units_from_paths_preserving_order 及所有适配器调用点，确保调用链不再传递无意义的 ClientId，同时保持现有路径扫描与错误处理行为不变。crates/tokscale-core/src/adapters/codebuddy.rs (1)
62-75: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win建议显式穷举
DecoderRoute，与其他适配器保持一致。这里用
_ =>通配符兜底，而antigravity.rs（50-59 行）与opencode.rs（57-66 行）都显式列出全部变体。将来新增路由时，后两者会编译失败提醒接入，而本文件会静默落到运行时unreachable!panic。♻️ 建议改为穷举分支
- _ => unreachable!("unexpected CodeBuddy input unit meta"), + DecoderRoute::None + | DecoderRoute::OpenCodeSqlite + | DecoderRoute::AntigravityCliSqlite + | DecoderRoute::KiroFile + | DecoderRoute::KiroSqlite + | DecoderRoute::KiroGlobalStorage + | DecoderRoute::Codex => { + unreachable!("CodeBuddy adapter received a non-CodeBuddy decoder route") + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/codebuddy.rs` around lines 62 - 75, Update the route match in the CodeBuddy adapter’s unit-processing flow to explicitly enumerate every current DecoderRoute variant, replacing the wildcard unreachable branch. Keep the existing handling for CodeBuddyJsonl and CodeBuddyExtensionLog, and ensure any newly added DecoderRoute variant causes a compile-time match error until handled.crates/tokscale-core/src/adapters/antigravity.rs (2)
57-59: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value诊断文本与命名仍引用已删除的
meta概念。 本次重构已用DecoderRoute取代InputUnitMeta，但unreachable!的信息与一处测试变量名仍沿用 “input unit meta”，会让后续排查按错误概念检索代码。
crates/tokscale-core/src/adapters/antigravity.rs#L57-L59：将文案改为描述路由失配，例如 “Antigravity adapter received a non-Antigravity decoder route”。crates/tokscale-core/src/adapters/opencode.rs#L64-L66：同样把 “unexpected OpenCode input unit meta” 改为按 decoder route 表述。crates/tokscale-core/src/adapters/codebuddy.rs#L73-L73：把 “unexpected CodeBuddy input unit meta” 改为按 decoder route 表述（与该文件的穷举分支建议一并落地即可）。crates/tokscale-core/src/adapters/codebuddy.rs#L366-L366：测试局部变量metas改名为routes，与unit.decoder.route()的取值一致。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/antigravity.rs` around lines 57 - 59, Replace the obsolete “input unit meta” terminology with decoder-route terminology in the unreachable diagnostics for DecoderRoute::Codex: update crates/tokscale-core/src/adapters/antigravity.rs lines 57-59, crates/tokscale-core/src/adapters/opencode.rs lines 64-66, and crates/tokscale-core/src/adapters/codebuddy.rs line 73 to describe receiving a non-matching decoder route. In crates/tokscale-core/src/adapters/codebuddy.rs line 366, rename the test-local variable metas to routes to match unit.decoder.route().
96-130: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win可复用
fold_units_with_filter消除重复的折叠骨架。
fold_antigravity_units与adapters/cache.rs的fold_units_with_filter（347-381 行）逻辑完全一致，唯一差异是消息过滤；hermes.rs的fold_hermes_units（81-93 行）已用该辅助函数表达同样的去重语义。改用它可避免缓存失效/写入顺序这类易错逻辑在多处各自维护。♻️ 建议的复用写法
fn fold_antigravity_units( parsed: Vec<ParsedUnit>, ctx: &mut FoldContext<'_>, sink: &mut BoundMessageSink<'_>, seen: &mut HashSet<u64>, ) -> Result<(), crate::adapters::InputPipelineError> { - for parsed_unit in parsed { - // ...手工重复 resolve/record_health/write_cache/remove 流程... - } - Ok(()) + adapter_cache::fold_units_with_filter(parsed, ctx, sink, |_, messages| { + messages + .into_iter() + .filter(|message| crate::should_keep_deduped_message(seen, message)) + .collect() + }) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/tokscale-core/src/adapters/antigravity.rs` around lines 96 - 130, Refactor fold_antigravity_units to use the existing adapters/cache.rs helper fold_units_with_filter instead of maintaining its own resolve, cache-write, invalidation, and message-emission loop. Pass a filter closure that preserves the current should_keep_deduped_message(seen, message) behavior, while retaining the existing context, sink, and error propagation semantics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/tokscale-core/src/message_cache.rs`:
- Around line 18-21: Update the prune flow around read_shard_header_for_prune
and prune_input_message_cache so shards with the immediately preceding
CACHE_FORMAT_VERSION are treated as stale entries eligible for deletion rather
than returned as UnsupportedFormat. Preserve hard failures for unsupported
versions that are not this known legacy version, and allow classification to
continue processing and deleting subsequent shards.
---
Nitpick comments:
In `@crates/tokscale-cli/src/tui/cache.rs`:
- Around line 130-134: Remove the unused health_for_input_footprint helper and
replace every call to it with
tokscale_core::input_health::HealthReport::default(). Ensure no callers or
references to the helper remain.
In `@crates/tokscale-cli/src/tui/mod.rs`:
- Around line 1132-1151: Update loaded_snapshot in
crates/tokscale-cli/src/tui/mod.rs:1132-1151 to accept impl IntoIterator<Item =
(ClientId, u64)> and pass it directly to InputFootprint::from_client_bytes,
removing the from_str conversion and expect. Apply the same change to
install_generation in crates/tokscale-cli/src/tui/ui/mod.rs:277-292, and update
its call sites to use [(ClientId::Junie, 0)] instead of string-keyed BTreeMap
values.
In `@crates/tokscale-core/build.rs`:
- Around line 187-197: 在 ClientId 的 Ord::cmp 实现中补充一行意图注释，明确必须按 as_str()
的字典序排序，而不是按 repr(usize) 判别值排序，因为聚合投影依赖该顺序稳定浮点累加结果。仅添加这项说明，保留现有 cmp 实现不变。
- Around line 214-223: 更新 ClientId 的 Deserialize::deserialize 实现，使用 Cow<'de,
str> 反序列化输入，确保未转义的 JSON client id 可借用而无需分配 String。将借用内容传递给
ClientId::from_str，并保持未知 id 的错误信息和现有行为不变。
In `@crates/tokscale-core/src/adapters/antigravity.rs`:
- Around line 57-59: Replace the obsolete “input unit meta” terminology with
decoder-route terminology in the unreachable diagnostics for
DecoderRoute::Codex: update crates/tokscale-core/src/adapters/antigravity.rs
lines 57-59, crates/tokscale-core/src/adapters/opencode.rs lines 64-66, and
crates/tokscale-core/src/adapters/codebuddy.rs line 73 to describe receiving a
non-matching decoder route. In crates/tokscale-core/src/adapters/codebuddy.rs
line 366, rename the test-local variable metas to routes to match
unit.decoder.route().
- Around line 96-130: Refactor fold_antigravity_units to use the existing
adapters/cache.rs helper fold_units_with_filter instead of maintaining its own
resolve, cache-write, invalidation, and message-emission loop. Pass a filter
closure that preserves the current should_keep_deduped_message(seen, message)
behavior, while retaining the existing context, sink, and error propagation
semantics.
In `@crates/tokscale-core/src/adapters/claude.rs`:
- Around line 242-276: 重构 fold_claude_units，改用
adapter_cache::fold_units_with_filter 承载
record_health、缓存写入及失败或未计划时的失效处理，并将现有去重逻辑作为其过滤器传入。移除该函数中重复的 resolve、缓存处理和
emit_messages 流程，保持 should_keep_deduped_message 基于 seen_keys 的去重语义不变。
In `@crates/tokscale-core/src/adapters/codebuddy.rs`:
- Around line 62-75: Update the route match in the CodeBuddy adapter’s
unit-processing flow to explicitly enumerate every current DecoderRoute variant,
replacing the wildcard unreachable branch. Keep the existing handling for
CodeBuddyJsonl and CodeBuddyExtensionLog, and ensure any newly added
DecoderRoute variant causes a compile-time match error until handled.
In `@crates/tokscale-core/src/adapters/discover.rs`:
- Around line 48-49: 移除 scan_roots、push_existing_file 和 canonical_key 中不再使用的
client 参数，并同步调整 input_units_from_paths、input_units_from_paths_preserving_order
及所有适配器调用点，确保调用链不再传递无意义的 ClientId，同时保持现有路径扫描与错误处理行为不变。
In `@crates/tokscale-core/src/adapters/kiro.rs`:
- Around line 252-256: Remove the unused _client parameter from
dedup_units_by_canonical_path and update its call site in the surrounding
discovery flow to pass only the units argument. Keep the deduplication behavior
unchanged.
- Around line 68-87: 在处理 DecoderRoute 的 match 中合并 KiroFile 与 KiroGlobalStorage
分支，复用同一个 load_or_scan_unit_with 调用；同时更新该分支的 unreachable! 提示文本，将过时的 “input unit
meta” 改为反映 decoder.route() 判别依据的文案，并同步检查 plan_cache_hit 相关提示。
In `@crates/tokscale-core/src/lib_tests.rs`:
- Around line 2139-2187: Update the decoder_changed signature test to use the
same requested_clients list as canonical_clients, changing only the DecoderSpec
revision from the baseline value to 999. Keep the existing decoder and input
setup otherwise unchanged so the final assert_ne! isolates and verifies the
decoder revision’s effect on the signature.
In `@crates/tokscale-core/src/lib.rs`:
- Around line 324-361: Remove the panic paths in prepared_input_footprint and
confirmed_input_footprint by returning a Result and propagating
InputFootprintOverflow instead of calling expect on add_bytes. Map the overflow
to LocalReportError::operational at the reporting boundary, and update callers
to propagate the resulting error while preserving existing footprint
calculations.
In `@crates/tokscale-core/src/message_cache.rs`:
- Around line 262-349: Extend define_decoder_ids! to expose all configured
stable names through DecoderId::ALL_STABLE_NAMES, preserving the existing
macro-generated mappings. Add a decoder_stable_names_are_unique test that
collects these names into a HashSet and asserts the unique count matches the
array length, so duplicate stable_name literals fail tests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 190d4cea-e62b-4ce7-9931-7cd19a5103a7
📒 Files selected for processing (56)
crates/tokscale-cli/src/commands/cache.rscrates/tokscale-cli/src/commands/models.rscrates/tokscale-cli/src/commands/shared.rscrates/tokscale-cli/src/tui/actions.rscrates/tokscale-cli/src/tui/app.rscrates/tokscale-cli/src/tui/cache.rscrates/tokscale-cli/src/tui/data/mod.rscrates/tokscale-cli/src/tui/mod.rscrates/tokscale-cli/src/tui/presentation.rscrates/tokscale-cli/src/tui/session_data.rscrates/tokscale-cli/src/tui/ui/mod.rscrates/tokscale-cli/src/tui/ui/models.rscrates/tokscale-cli/src/tui/ui/overview_snapshot.rscrates/tokscale-cli/src/tui/ui/view_footer.rscrates/tokscale-cli/tests/cli_tests.rscrates/tokscale-core/benches/aggregation.rscrates/tokscale-core/build.rscrates/tokscale-core/src/adapters/antigravity.rscrates/tokscale-core/src/adapters/cache.rscrates/tokscale-core/src/adapters/claude.rscrates/tokscale-core/src/adapters/cline.rscrates/tokscale-core/src/adapters/codebuddy.rscrates/tokscale-core/src/adapters/codebuff.rscrates/tokscale-core/src/adapters/codex.rscrates/tokscale-core/src/adapters/decoder.rscrates/tokscale-core/src/adapters/discover.rscrates/tokscale-core/src/adapters/error.rscrates/tokscale-core/src/adapters/file.rscrates/tokscale-core/src/adapters/goose.rscrates/tokscale-core/src/adapters/hermes.rscrates/tokscale-core/src/adapters/junie.rscrates/tokscale-core/src/adapters/kilo.rscrates/tokscale-core/src/adapters/kiro.rscrates/tokscale-core/src/adapters/mod.rscrates/tokscale-core/src/adapters/omp.rscrates/tokscale-core/src/adapters/openclaw.rscrates/tokscale-core/src/adapters/opencode.rscrates/tokscale-core/src/adapters/pi.rscrates/tokscale-core/src/adapters/roocode.rscrates/tokscale-core/src/adapters/runtime.rscrates/tokscale-core/src/adapters/warp.rscrates/tokscale-core/src/adapters/zed.rscrates/tokscale-core/src/aggregate/keys.rscrates/tokscale-core/src/aggregate/tui.rscrates/tokscale-core/src/aggregate/tui_sessions.rscrates/tokscale-core/src/input_footprint.rscrates/tokscale-core/src/input_health.rscrates/tokscale-core/src/lib.rscrates/tokscale-core/src/lib_tests.rscrates/tokscale-core/src/message_cache.rscrates/tokscale-core/src/sessions/mod.rsdocs/adr/0007-client-identity-catalog.mddocs/adr/0008-single-copy-memory-pipeline.mddocs/adr/0010-period-views-derive-from-daily.mddocs/cli.mddocs/configuration.md
🚧 Files skipped from review as they are similar to previous changes (1)
- crates/tokscale-core/src/adapters/codex.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AdapterBindingthe sole production owner of localClientIdInputUnit, decoder output, cache shards, and pipeline errors source-neutral; attach the client only throughBoundMessageSinkDecoderSpecand stable string-serializedDecoderIdClientIdthrough the resolved client universe, aggregation keys, and session accumulationInputFootprintthe single confirmed input-size fact for Sessions, Overview, TUI cache storage, and headless report metadataArchitecture
The registry binds one adapter to one typed client. Discovery runs under that binding, cache recovery reparses through the same binding, and the runtime sink is the only production path that can construct a client-attributed
UnifiedMessage. Session decoders and message-cache bodies use the same source-neutralUsageRecord.DecoderSpecbinds decoder identity, semantic revision, and execution route. Routed decoders require their specialized constructor, so cache identity and dispatch cannot drift independently.InputFootprintstores one checkedBTreeMap<ClientId, u64>. Sessions reads each client value from it, Overview derives Data Size as its checked sum, and Models JSON exposes the same confirmed map asmetadata.inputFootprint. Data Health and the TUI cache no longer persist a second total.The input-message cache format advances to v12, the inventory signature contract advances to v3, and the TUI generation schema advances to v52.
Real-data verification
A full source-built
tokscale cache warm --home /home/travisproduced one atomic schema 52 generation with:sum(clientSpace) = 6,950,968,526 BinputDataBytesfieldThe immediately preceding live schema 50 generation had both
sum(clientSpace)andhealth.inputDataBytesequal to6,945,642,695 B. The5,325,831 Bchange was entirely Codex growth between generations; the real inventory showed no cross-client physical-file discrepancy.Validation
cargo fmt --all --checkcargo check --workspace --all-targetscargo test --workspace— 2108 passed, 5 ignoredcargo clippy --workspace --all-targets -- -D warningscargo build -p tokscale-cligit diff --checkSummary by CodeRabbit
tokscale models --json的metadata现携带inputFootprint，并据此展示数据规模指标。health.inputDataBytes现在不再作为落盘统计项返回。