Skip to content

perf(memory): store holographic vectors as float32 (#30499 salvage) - #76917

Merged
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/30499-hrr-float32
Aug 2, 2026
Merged

perf(memory): store holographic vectors as float32 (#30499 salvage)#76917
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/30499-hrr-float32

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Salvage of #30499 by @JabberELF — both commits cherry-picked to preserve authorship; one conflict resolved against current main.

Context — what this changes for users

The holographic memory plugin stores one HRR phase vector per fact as raw float64 bytes — 8KB per fact at the default dim=1024. Phase angles don't need float64: float32's ~1e-7 relative error is far below the similarity noise floor. This PR halves vector storage (4KB/fact; a 10k-fact store shrinks ~40MB) with a versioned format: new blobs carry an HRR1 magic prefix, and the decoder promotes everything to float64 on read so downstream math is unchanged.

Migration story (the risky part — independently audited)

  • Existing float64 blobs (no prefix) still load: decoder detects HRR1 + exact float32 size, falls back to raw float64 by length. Traced end-to-end with a real legacy blob written into an existing DB — reads fine, no crash, no garbage.
  • dim=1 size-collision ambiguity (8 bytes = 1 float64 = prefix+1 float32) explicitly handled: writer falls back to raw float64 when sizes collide; reader prefers the legacy interpretation. Both directions test-pinned, including the adversarial HRR1-prefix-collision case (second commit).
  • Mixed-precision safety: decode always promotes to float64, so similarity() never compares mixed dtypes; all 8 production call sites pass dim=self.hrr_dim.
  • Round-trip parity tests at atol=1e-6 (the float32 quantization bound).

Salvage notes

Verification

  • 245 tests green (new storage suite + full tests/plugins/memory/), ruff clean.
  • Mutation check: production reverted → 9/11 new tests fail (the 2 legacy-read tests correctly still pass); restored → 11/11.

Closes #30499.

When hrr_dim=1 the prefixed float32 blob (4+4=8 bytes) collides in
size with a raw float64 blob (1×8=8 bytes), making the format
discriminator in bytes_to_phases ambiguous — a legacy blob starting
with HRR1 would be misread as a prefixed float32 vector.

- phases_to_bytes now accepts an optional dim and falls back to
  writing raw float64 when the two blob sizes are equal.
- bytes_to_phases prefers the legacy float64 interpretation when
  sizes collide and dim is provided, since phases_to_bytes never
  writes a prefixed float32 blob at dim=1.
- Three regression tests cover dim=1 write, round-trip, and the
  legacy-prefix collision case.

Addresses hermes-sweeper review on PR NousResearch#30499.
@teknium1 teknium1 added area/memory Memory subsystem: store, providers, sync, background reviews type/perf Performance improvement or optimization labels Aug 2, 2026
@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) August 2, 2026 17:03
@kshitijk4poor
kshitijk4poor merged commit 7a450ca into NousResearch:main Aug 2, 2026
40 checks passed
@alt-glitch alt-glitch added P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 2, 2026
@kshitijk4poor
kshitijk4poor deleted the salvage/30499-hrr-float32 branch August 5, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/memory Memory tool and memory providers type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants