Skip to content

fix(vendor): recover Windows portability patches #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Nanako0129 merged 1 commit into from
Jul 17, 2026
Merged

fix(vendor): recover Windows portability patches #59

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions vendor/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,14 @@ next sync should treat these as already-present and not re-apply them.
| `#856` (`ae36db5c`) — **M13 selective parser-dependency completion** | Existing Droid settings snapshots can read a same-session fallback JSONL for the model, legacy Kimi wire logs read the shared `~/.kimi/config.json`, and existing Kiro CLI session headers read a same-stem JSONL for estimated tokens, prompt time, and duration. Those dependencies were parser-only: both cache lanes, the live-tail token, and pruning still observed just the scanned primary. **TokenBar adaptation:** parser-owned path helpers now drive specialized monolithic fingerprints, materialized and shipping streaming lanes, latest-mtime probing, and sibling-aware fail-open pruning. Hermetic regressions cover absent-then-created and rewritten dependencies, independently warmed materialized/streaming caches, all three change-token lanes, fresh-dependency retention, missing-dependency pruning, and stat failure. The upstream shard cache, `CacheIdentity`, `parser_version`, Kimi Code, Kiro IDE/globalStorage/`.chat`/Windows sources, and unrelated client changes remain excluded. Related-file fingerprint storage already exists, so cache schema remains 28. | `src/sessions/droid.rs`, `src/sessions/kimi.rs`, `src/sessions/kiro.rs`, `src/message_cache.rs`, `src/lib.rs` |
| `#878` (`3587f745`) — **M14** | Codex token snapshots now attribute `duration_ms` to non-overlapping intervals from the previous accepted snapshot rather than repeatedly measuring from the turn start. Invalid, equal, backward, replayed, duplicate, regressive, and zero-token rows do not advance the cursor; a new `turn_context` resets it. **TokenBar adaptation:** the cursor is serialized in the existing incremental state, the local monolithic cache schema bumps **28→29** so unchanged sources cannot replay overlapping durations or resume without the cursor, and hermetic regressions cover parser edge cases, incremental append parity, same-fingerprint schema-28 rebuild, materialized warm-cache parity, and shipping streaming cold/warm performance (`7,000 ms / 170 timed tokens / 3 samples`). Upstream's shard-cache `parser_version` bump is deliberately excluded; the existing Rust → FFI → Swift `ModelPerformance` shape is unchanged. | `src/sessions/codex.rs`, `src/message_cache.rs`, `src/lib.rs` (tests), `tests/fixtures/codex_duration_timing.jsonl` |

## Recovered Windows downstream commits

The Windows port started from TokenBar [`2ed256ee`](https://github.com/Nanako0129/TokenBar/commit/2ed256eea7f6761e85198e3bc584e08a8d3d8ac1), then accumulated and validated the vendor-only sequence recorded in the [issue #45 handoff](https://github.com/Nanako0129/TokenBar/issues/45#issuecomment-5002865759). These commits are now recovered into this repository so it remains the canonical source for the next Windows sync; the detailed Windows build, hostile-environment, profile-manifest, and FFI evidence stays in that handoff rather than being duplicated here.

| Commits | What | Files | Cache / upstream status |
|---|---|---|---|
| `e5200634` → `db2a96a3` → `15f418ee` → `e807f333` → `0979cdb0` → `26b892a6` → `6ac77c03` → `e91fb2c6` → `fbecb99c` → `3c8bfc52` → `aec5bd88` | Makes core tests hermetic with panic-safe environment/current-directory guards, serial coordination, isolated cache/XDG roots, platform-safe JSON/path fixtures, and explicit-home parser/scanner fixtures while retaining dedicated positive environment cases. Production fixes release the temporary cache writer before Windows atomic replacement, reopen the final cache read/write for the durability sync, keep Windows Zed/config fallbacks behind `use_env_roots`, and compare extra-path warnings against the supplied scan home. | `src/clients.rs`, `src/lib.rs`, `src/message_cache.rs`, `src/pricing/cache.rs`, `src/scanner.rs`, `src/sessions/claudecode.rs`, `src/sessions/opencode.rs` | No serialized parser output, cache layout, FFI, or public API change; `CACHE_SCHEMA_VERSION` remains 29. Upstream follow-up is deliberately split into cache runtime, explicit-home scanner, and test-only hermetic contributions; none is reported yet. |

## Upstream fixes reported but not yet vendored

Bugs we found *while* vendoring, reported upstream, and that were fixed there —
Expand Down
178 changes: 87 additions & 91 deletions vendor/tokscale-core/src/clients.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,10 @@ impl PathRoot {
if let Ok(xdg_config_home) = std::env::var("XDG_CONFIG_HOME") {
return format!("{xdg_config_home}/tokscale");
}
}

// Match paths::get_config_dir() platform branches so the
// scanner reads from the same root the writer (e.g.
// get_antigravity_cache_dir) targets. Hardcoding
// `{home}/.config/tokscale` everywhere would diverge from
// dirs::config_dir() on Windows (where it resolves to
// %APPDATA%\tokscale), causing synced data to land in
// %APPDATA% while the scanner looks in %USERPROFILE%.
#[cfg(target_os = "windows")]
{
// Match paths::get_config_dir() so default Windows scans
// read the same %APPDATA% root used by cache writers.
#[cfg(target_os = "windows")]
if let Some(dir) = dirs::config_dir() {
return dir.join("tokscale").to_string_lossy().into_owned();
}
Expand Down Expand Up @@ -535,20 +528,57 @@ impl Default for ClientCounts {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Mutex, OnceLock};
use serial_test::serial;
use std::ffi::OsStr;

struct EnvGuard(Vec<(&'static str, Option<std::ffi::OsString>)>);

impl EnvGuard {
fn capture(keys: &[&'static str]) -> Self {
Self(
keys.iter()
.map(|key| (*key, std::env::var_os(key)))
.collect(),
)
}

fn set(&mut self, key: &'static str, value: impl AsRef<OsStr>) {
unsafe { std::env::set_var(key, value) };
}

fn env_lock() -> &'static Mutex<()> {
static LOCK: OnceLock<Mutex<()>> = OnceLock::new();
LOCK.get_or_init(|| Mutex::new(()))
fn remove(&mut self, key: &'static str) {
unsafe { std::env::remove_var(key) };
}
}

fn restore_env(var: &str, previous: Option<String>) {
match previous {
Some(value) => unsafe { std::env::set_var(var, value) },
None => unsafe { std::env::remove_var(var) },
impl Drop for EnvGuard {
fn drop(&mut self) {
unsafe {
for (key, previous) in self.0.drain(..) {
match previous {
Some(value) => std::env::set_var(key, value),
None => std::env::remove_var(key),
}
}
}
}
}

#[test]
#[serial]
fn test_env_guard_restores_after_unwind() {
const KEY: &str = "TOKSCALE_CLIENTS_ENV_GUARD_SELF_CHECK";
let mut outer = EnvGuard::capture(&[KEY]);
outer.set(KEY, "before");
let result = std::panic::catch_unwind(|| {
let mut inner = EnvGuard::capture(&[KEY]);
inner.set(KEY, "during");
panic!("exercise EnvGuard unwinding");
});
assert!(result.is_err());
assert_eq!(std::env::var_os(KEY), Some("before".into()));
}

#[test]
fn test_client_id_count() {
assert_eq!(ClientId::COUNT, 31);
Expand Down Expand Up @@ -583,89 +613,69 @@ mod tests {
}

#[test]
#[serial]
fn test_path_root_xdg_data_uses_env_var_when_set() {
let _guard = env_lock().lock().unwrap();
let previous = std::env::var("XDG_DATA_HOME").ok();
unsafe { std::env::set_var("XDG_DATA_HOME", "/tmp/xdg-data-home") };
let mut _env = EnvGuard::capture(&["XDG_DATA_HOME"]);
_env.set("XDG_DATA_HOME", "/tmp/xdg-data-home");

let resolved = PathRoot::XdgData.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/xdg-data-home");

restore_env("XDG_DATA_HOME", previous);
}

#[test]
#[serial]
fn test_path_root_xdg_data_falls_back_when_unset() {
let _guard = env_lock().lock().unwrap();
let previous = std::env::var("XDG_DATA_HOME").ok();
unsafe { std::env::remove_var("XDG_DATA_HOME") };
let mut _env = EnvGuard::capture(&["XDG_DATA_HOME"]);
_env.remove("XDG_DATA_HOME");

let resolved = PathRoot::XdgData.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/home/.local/share");

restore_env("XDG_DATA_HOME", previous);
}

#[test]
#[serial]
fn test_path_root_xdg_data_ignores_env_when_disabled() {
let _guard = env_lock().lock().unwrap();
let previous = std::env::var("XDG_DATA_HOME").ok();
unsafe { std::env::set_var("XDG_DATA_HOME", "/tmp/xdg-data-home") };
let mut _env = EnvGuard::capture(&["XDG_DATA_HOME"]);
_env.set("XDG_DATA_HOME", "/tmp/xdg-data-home");

let resolved = PathRoot::XdgData.resolve_with_env_strategy("/tmp/home", false);
assert_eq!(resolved, "/tmp/home/.local/share");

restore_env("XDG_DATA_HOME", previous);
}

#[test]
#[serial]
fn test_path_root_config_uses_override_when_set() {
let _guard = env_lock().lock().unwrap();
let previous_override = std::env::var("TOKSCALE_CONFIG_DIR").ok();
let previous_xdg = std::env::var("XDG_CONFIG_HOME").ok();
unsafe {
std::env::set_var("TOKSCALE_CONFIG_DIR", "/tmp/custom-config-root");
std::env::set_var("XDG_CONFIG_HOME", "/tmp/xdg-config-home");
}
let mut _env = EnvGuard::capture(&["TOKSCALE_CONFIG_DIR", "XDG_CONFIG_HOME"]);
_env.set("TOKSCALE_CONFIG_DIR", "/tmp/custom-config-root");
_env.set("XDG_CONFIG_HOME", "/tmp/xdg-config-home");

let resolved = PathRoot::Config.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/custom-config-root");

restore_env("TOKSCALE_CONFIG_DIR", previous_override);
restore_env("XDG_CONFIG_HOME", previous_xdg);
}

#[test]
#[serial]
#[cfg(target_os = "linux")]
fn test_path_root_config_uses_xdg_config_home_when_override_unset() {
let _guard = env_lock().lock().unwrap();
let previous_override = std::env::var("TOKSCALE_CONFIG_DIR").ok();
let previous_xdg = std::env::var("XDG_CONFIG_HOME").ok();
unsafe {
std::env::remove_var("TOKSCALE_CONFIG_DIR");
std::env::set_var("XDG_CONFIG_HOME", "/tmp/xdg-config-home");
}
let mut _env = EnvGuard::capture(&["TOKSCALE_CONFIG_DIR", "XDG_CONFIG_HOME"]);
_env.remove("TOKSCALE_CONFIG_DIR");
_env.set("XDG_CONFIG_HOME", "/tmp/xdg-config-home");

let resolved = PathRoot::Config.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/xdg-config-home/tokscale");

restore_env("TOKSCALE_CONFIG_DIR", previous_override);
restore_env("XDG_CONFIG_HOME", previous_xdg);
}

#[test]
#[serial]
#[cfg(target_os = "windows")]
fn test_path_root_config_uses_dirs_config_dir_on_windows() {
// Windows must resolve PathRoot::Config to the same root that
// paths::get_config_dir() and get_antigravity_cache_dir() use,
// i.e. dirs::config_dir() (= %APPDATA%\tokscale). Hardcoding
// {home}/.config/tokscale would diverge from the writer side
// and silently hide synced Antigravity data from reports.
let _guard = env_lock().lock().unwrap();
let previous_override = std::env::var("TOKSCALE_CONFIG_DIR").ok();
unsafe {
std::env::remove_var("TOKSCALE_CONFIG_DIR");
}
let mut _env = EnvGuard::capture(&["TOKSCALE_CONFIG_DIR"]);
_env.remove("TOKSCALE_CONFIG_DIR");

let resolved = PathRoot::Config.resolve("C:\\fake-home");
let expected = dirs::config_dir()
Expand All @@ -677,76 +687,62 @@ mod tests {
resolved, expected,
"PathRoot::Config on Windows must match dirs::config_dir().join('tokscale') so the scanner agrees with the writer"
);

restore_env("TOKSCALE_CONFIG_DIR", previous_override);
}

#[test]
#[serial]
fn test_path_root_config_ignores_env_when_disabled() {
let _guard = env_lock().lock().unwrap();
let previous_override = std::env::var("TOKSCALE_CONFIG_DIR").ok();
let previous_xdg = std::env::var("XDG_CONFIG_HOME").ok();
unsafe {
std::env::set_var("TOKSCALE_CONFIG_DIR", "/tmp/custom-config-root");
std::env::set_var("XDG_CONFIG_HOME", "/tmp/xdg-config-home");
}
let mut _env = EnvGuard::capture(&["TOKSCALE_CONFIG_DIR", "XDG_CONFIG_HOME"]);
_env.set("TOKSCALE_CONFIG_DIR", "/tmp/custom-config-root");
_env.set("XDG_CONFIG_HOME", "/tmp/xdg-config-home");

let resolved = PathRoot::Config.resolve_with_env_strategy("/tmp/home", false);
assert_eq!(resolved, "/tmp/home/.config/tokscale");

restore_env("TOKSCALE_CONFIG_DIR", previous_override);
restore_env("XDG_CONFIG_HOME", previous_xdg);
}

#[test]
#[serial]
fn test_path_root_env_var_uses_env_when_set() {
let _guard = env_lock().lock().unwrap();
let var = "TOKSCALE_TEST_PATH_ROOT";
let previous = std::env::var(var).ok();
unsafe { std::env::set_var(var, "/tmp/custom-root") };
let mut _env = EnvGuard::capture(&[var]);
_env.set(var, "/tmp/custom-root");

let root = PathRoot::EnvVar {
var,
fallback_relative: ".fallback",
};
let resolved = root.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/custom-root");

restore_env(var, previous);
}

#[test]
#[serial]
fn test_path_root_env_var_falls_back_when_unset() {
let _guard = env_lock().lock().unwrap();
let var = "TOKSCALE_TEST_PATH_ROOT";
let previous = std::env::var(var).ok();
unsafe { std::env::remove_var(var) };
let mut _env = EnvGuard::capture(&[var]);
_env.remove(var);

let root = PathRoot::EnvVar {
var,
fallback_relative: ".fallback",
};
let resolved = root.resolve("/tmp/home");
assert_eq!(resolved, "/tmp/home/.fallback");

restore_env(var, previous);
}

#[test]
#[serial]
fn test_path_root_env_var_ignores_env_when_disabled() {
let _guard = env_lock().lock().unwrap();
let var = "TOKSCALE_TEST_PATH_ROOT";
let previous = std::env::var(var).ok();
unsafe { std::env::set_var(var, "/tmp/custom-root") };
let mut _env = EnvGuard::capture(&[var]);
_env.set(var, "/tmp/custom-root");

let root = PathRoot::EnvVar {
var,
fallback_relative: ".fallback",
};
let resolved = root.resolve_with_env_strategy("/tmp/home", false);
assert_eq!(resolved, "/tmp/home/.fallback");

restore_env(var, previous);
}

#[test]
Expand Down Expand Up @@ -837,17 +833,15 @@ mod tests {
}

#[test]
#[serial]
fn test_zed_data_dir_path() {
let _guard = env_lock().lock().unwrap();
let previous = std::env::var("XDG_DATA_HOME").ok();
unsafe { std::env::remove_var("XDG_DATA_HOME") };
let mut _env = EnvGuard::capture(&["XDG_DATA_HOME"]);
_env.remove("XDG_DATA_HOME");

assert_eq!(
ClientId::Zed.data().resolve_path("/tmp/home"),
"/tmp/home/.local/share/zed/threads/threads.db"
);

restore_env("XDG_DATA_HOME", previous);
}

#[test]
Expand Down Expand Up @@ -875,7 +869,9 @@ mod tests {
assert!(client.data().parse_local);
assert!(client.data().submit_default);
assert_eq!(
client.data().resolve_path("/tmp/home"),
client
.data()
.resolve_path_with_env_strategy("/tmp/home", false),
"/tmp/home/.grok/sessions"
);
}
Expand Down
Loading