fix(memory): use slim Hindsight stack on Intel macOS instead of bare hindsight-all

[Enough1122]

Fixes #81421

Root cause

hermes_cli/memory_setup.py::_provider_pip_dependencies appends a bare hindsight-all spec for every Hindsight local/local_embedded install. This was added by #72363 (commit 5645169c8f1) to restore local Hindsight after a venv rebuild stripped hindsight-embed (#70636).

On Intel macOS, the current full local-ML dependency set includes MLX packages that have no x86_64 wheels. The resolver backtracks instead of failing: it installs ancient hindsight-all / hindsight-api releases whose overlapping hindsight_api files override the working slim API. The daemon then crashes with:

ValueError: Unknown embeddings provider: onnx.
Supported: 'local', 'tei', 'openai', 'cohere', 'litellm'

while the Hindsight web UI stays up and reports HTTP 500/502 — exactly the mixed-stack evidence in the issue (hindsight-all 0.4.17 + hindsight-api 0.3.0 installed over slim 0.8.4).

Fix

_provider_pip_dependencies now detects Darwin + x86_64 (_is_intel_macos()) and installs the thin slim stack for local/local_embedded mode on that platform:

• hindsight-all-slim
• hindsight-api-slim[local-onnx]

This matches the known-working Intel stack from the issue (slim 0.8.4 + hindsight-all installed --no-deps as a thin wrapper; the explicit slim specs are the portable declaration). Apple Silicon, Linux, and every other OS keep the full hindsight-all bundle — the #70636 heal path is unchanged. Non-local modes and missing config fall back to the declared bridge deps exactly as before.

Regression tests

6 tests in tests/hermes_cli/test_memory_setup_intel_hindsight.py:

1. test_detects_intel_macos — Darwin + x86_64 detection.
2. test_arm64_macos_is_not_intel — Apple Silicon keeps the full bundle.
3. test_linux_is_not_intel_macos — Linux x86_64 keeps the full bundle.
4. test_intel_macos_uses_slim_stack_not_bare_bundle — the issue's exact scenario: bare hindsight-all must NOT be in the dependency plan on Intel macOS.
5. test_non_local_modes_unaffected — remote-mode config never gets local deps.
6. test_missing_config_falls_back_to_declared — no config.json → declared deps only.

The issue scenario is RED on pre-fix code (verified: old code appends bare hindsight-all unconditionally for local_embedded) and GREEN after. Existing test_memory_setup.py (4 tests) continues to pass.

Scope

Two small additions to memory_setup.py (_is_intel_macos helper + one branch in _provider_pip_dependencies) plus tests. No change to the installer, the resolver, or non-Hindsight providers.

[Enough1122]

Fix follow-up: the original fix only covered _provider_pip_dependencies (the update-heal / --force resolver path). The interactive hermes memory setup wizard runs HindsightProvider.post_setup which independently hardcoded local_dep = 'hindsight-all', so on Intel macOS the wizard installed the slim stack first and then immediately reinstalled the bare full bundle on top — undoing the fix.

This follow-up:

• Imports _is_intel_macos into the plugin's post_setup and selects the slim specs on Intel macOS, matching _provider_pip_dependencies.
• Adds hindsight-all-slim / hindsight-api-slim to the import-name maps in hermes_cli/memory_setup.py and hermes_cli/web_server.py so the already-installed check doesn't reinstall the slim packages every run.
• Adds regression tests for the plugin path's dep selection (9/9 pass locally).

[Enough1122]

Update — third commit added (964ee66): smoke-check the slim runtime, per @ijevin's superseded #81559.

I've integrated the post-install smoke validation that @ijevin's #81559 (closed in favor of this PR) flagged as its unique value, so nothing needs cherry-picking from that branch:

1. Post-install/refresh smoke check. After a Hindsight install or refresh on Intel macOS + local/local_embedded, a clean-subprocess probe imports hindsight_api / hindsight_embed and confirms hindsight_api.LocalSTEmbeddings resolves — the exact fix(update): Hindsight dependency heal backtracks to ancient full packages on Intel macOS #81421 backtrack failure signature (importable-but-ancient API). On failure it prints an actionable banner and raises RuntimeError, so hermes update can no longer report a "healed" install that leaves the daemon crashing with Unknown embeddings provider: onnx.
   • Runs on both paths: after a successful pip install, and on the non-force early-return refresh path (all slim deps already importable) so pre-existing stale runtimes are caught too.
   • Gated to Intel macOS + local mode; every other platform/mode is untouched.
2. Import-map correction. The slim meta-packages don't expose a top-level import at their pip name — hindsight-all-slim / hindsight-api-slim now map to the hindsight_api module they ship (both memory_setup.py and web_server.py), otherwise the missing-dep probe would reinstall them on every refresh.
3. hindsight-embed added to the slim-stack spec in _provider_pip_dependencies and the plugin post_setup wizard, so the embed manager that drives the ONNX provider is explicitly declared (same class of bug as hermes update downgrades/removes third-party packages installed into the managed venv (e.g. hindsight-embed) #70636).

About the failing CI check: the only red is Build&Test Docker image / build (arm64), which fails with "The operation was aborted due to timeout" — an infrastructure timeout in the Docker build, not a code failure. All 12 Python test slices pass. Happy to rebase/retry if useful.

New commits: d19216d8c → 69aed443d (rebase) → 964ee6676 (this). 15 tests added in total across the three commits (9 CLI/plugin-path + 6 smoke-check); all pass locally.

[ijevin]

Thanks for integrating the smoke check + hindsight-embed + import-map fixes into #81530 — glad the unique value from #81559 made it across. Nothing to cherry-pick from my branch on my side either; I'll close out the local fork branch so we don't leave stale assets sitting around.

Quick observation from looking at your three-commit evolution: forcing the smoke check on both the install path AND the non-force early-return refresh path is a stronger design than what I had (mine was install-only). The non-force path is exactly where stale runtimes hide — your fix catches the failure mode I'd have left behind.

On the Docker arm64 timeout: that's a known GitHub Actions infrastructure flake on this repo, not your code. If it persists after a rebase-retry, it's worth pinging a maintainer — they sometimes have a workaround for the heavier Docker build path. (Not blocking the merge in any case.)

Closing out from my side. 🫡

[spfcraze]

This was generated by AI during triage.

Summary:
The smoke check runs even when the dependency install failed — it sits after the whole install try/except, so on Intel macOS a failed hindsight install ends in a RuntimeError that claims pip reported success.

Problems:

• In hermes_cli/memory_setup.py, the second _maybe_run_intel_macos_local_embedded_smoke_check() call sits after the install try/except block, outside the install-success branch — a blocked or failed install runs the probe too, and its failures raise.
• The raised message ("pip (or an existing importable release) reports success but the configured local runtime is not usable") contradicts that path, where pip just reported failure.
• cmd_setup_provider (hermes_cli/memory_setup.py:265) and the picker (hermes_cli/memory_setup.py:323) call _install_dependencies(name) with no guard, so on Intel macOS the RuntimeError propagates out of _install_dependencies into the wizard after the "Run manually" guidance was already printed.

Solution:
Call the smoke check only in the install-success branch; the early-return refresh path already runs it for the stale-runtime case.

---

Checked against b63ccb9 — the PR head when this was written — and 72eda94, main at the same moment.

[Enough1122]

Follow-up commit pushed on fix/81421-hindsight-intel-heal (9dad6fce4) — addressed @spfcraze's triage review.

What changed: the post-install smoke check now lives inside the outcome.ok branch only. The previous flow ran it after the install try/except, so a failed or blocked install printed "Run manually: …" guidance and raised a RuntimeError whose message ("pip (or an existing importable release) reports success") directly contradicted the same branch's failure printout.

Behavior preserved: the early-return refresh path (hermes_cli/memory_setup.py:210) keeps its own pre-existing smoke-check call — it already covers the stale-runtime case for installs where every slim dep is already importable.

tests/hermes_cli/test_memory_setup*.py + test_canonical_custom_identity.py (6 files, 6 tests) pass locally.

[Enough1122]

Thanks for the detailed verification and for flagging the walkthrough — and yes, the observation about ordering is exactly right: the smoke check now runs ONLY after a confirmed successful install (commit 9dad6fce4), with failure propagation out of the install try/except (79ed32370), so a failed/blocked install can never produce a RuntimeError that claims pip succeeded. The refresh path (deps already importable) still smoke-checks the configured runtime, since that is the #81421 failure mode the check exists for.

Branch rebased onto current main; tests/hermes_cli/test_memory_setup_intel_hindsight.py 21/21 pass.

[Enough1122]

Rebased onto current main to pick up #85970 (cost-guard fixture fix that landed 29 minutes after the previous CI run).

The slice 2/12 failures were pre-existing on main at the time (test_model_cost_guard.py, untouched by this PR), fixed upstream by #85970. Verified locally on the rebased head: 35 passed (memory setup + cost guard).

[Enough1122]

Rebased onto 16b54e2a0 (the #85970 merge point) instead of latest main.

The previous rebase onto latest main picked up 56a41715d (fix: persist provider on model switch, merged 09:09 UTC), which breaks tests/tui_gateway/test_slash_worker_profile_home.py on main itself — the test stubs hermes_constants with a MagicMock returning a bare str, and the changed import chain now hits DEFAULT_DB_PATH = get_hermes_home() / "state.db" (str / str TypeError). Reproduced locally on main HEAD (fails), passes on 16b54e2a0 (12/12).

This PR's own changes are unaffected: memory-setup + cost-guard + profile-home tests all pass (36/36) on the rebased head. One commit behind latest main (56a4171 only) — happy to rebase again once the main-side test is fixed upstream.