Fix dependencies and docker - #1
Merged
Merged
Conversation
benfrank241
added a commit
that referenced
this pull request
Apr 2, 2026
Hindsight #1 on BEAM at 10M tokens — 64.1% vs 40.6% next-best. Includes full tier comparison table, context rot section, and AMB manifesto cross-links. Image is a placeholder pending final asset. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
benfrank241
added a commit
that referenced
this pull request
Apr 2, 2026
…M Tokens (#851) * Add BEAM SOTA blog post
3 tasks
This was referenced May 25, 2026
7 tasks
nicoloboschi
added a commit
that referenced
this pull request
Jun 3, 2026
… always shown The residual near-duplicate observations were recall misses, not LLM errors: the existing near-identical observation (the merge 'twin') is semantic rank #1 for the new fact, but shares no source-fact graph link and little lexical overlap, so RRF averaged it below the per-fact recall budget (512 tokens, ~8 observations) and the LLM never saw it -> created a duplicate. The cross-encoder demoted it the same way (semantic #1 -> reranked #37). Add round-robin interleave fusion: take each retrieval arm's #1, then each #2, ... (semantic, bm25, graph, temporal), de-duplicating, until the budget fills. This guarantees every arm's top hit a slot, so the semantic-#1 twin is always shown and the LLM UPDATEs instead of creating a duplicate. Replace the recall 'rerank: bool' / ad-hoc passthrough with a single named 'reranking' strategy: 'cross_encoder' (default), 'rrf', 'interleave'. Consolidation dedup recall uses 'interleave'. For interleave the fusion order is authoritative — combined scoring's recency/temporal re-sort is skipped (that re-sort is what buried the twin). Measured on an English translation of the hermes transcript (removes the cross-lingual embedding confound): near-dup observations 4% (@0.97) -> 0% at both 1/10 and 1/4 cuts, coverage 89% -> 94%, no false merges (heavy observations are coherent single themes; distinct facets stay separate).
nicoloboschi
added a commit
that referenced
this pull request
Jun 3, 2026
… always shown The residual near-duplicate observations were recall misses, not LLM errors: the existing near-identical observation (the merge 'twin') is semantic rank #1 for the new fact, but shares no source-fact graph link and little lexical overlap, so RRF averaged it below the per-fact recall budget (512 tokens, ~8 observations) and the LLM never saw it -> created a duplicate. The cross-encoder demoted it the same way (semantic #1 -> reranked #37). Add round-robin interleave fusion: take each retrieval arm's #1, then each #2, ... (semantic, bm25, graph, temporal), de-duplicating, until the budget fills. This guarantees every arm's top hit a slot, so the semantic-#1 twin is always shown and the LLM UPDATEs instead of creating a duplicate. Replace the recall 'rerank: bool' / ad-hoc passthrough with a single named 'reranking' strategy: 'cross_encoder' (default), 'rrf', 'interleave'. Consolidation dedup recall uses 'interleave'. For interleave the fusion order is authoritative — combined scoring's recency/temporal re-sort is skipped (that re-sort is what buried the twin). Measured on an English translation of the hermes transcript (removes the cross-lingual embedding confound): near-dup observations 4% (@0.97) -> 0% at both 1/10 and 1/4 cuts, coverage 89% -> 94%, no false merges (heavy observations are coherent single themes; distinct facets stay separate).
nicoloboschi
added a commit
that referenced
this pull request
Jun 3, 2026
…edup recall (#1907) Round-robin interleave fusion for consolidation dedup recall (guarantees the semantic-#1 'twin' a slot so the LLM updates instead of duplicating), unified 'reranking' strategy param (cross_encoder/rrf/interleave), case-sensitive exact-dup guard, obs-dedup tool + benchmark wired into the perf dashboard (English dataset). Near-dup observation rate 4% -> 0% on the English hermes transcript (1/10 and 1/4), coverage 89% -> 94%, no false merges.
This was referenced Jun 4, 2026
cdbartholomew
added a commit
that referenced
this pull request
Jun 9, 2026
…ever (#2092) * blog: Hindsight is the fastest-growing open-source AI memory project ever Equal-age GitHub star analysis (per-star timestamps) plus third-party validation from OSSCAR (#10 fastest-growing OSS org, ahead of Mem0) and dope.security (#1 MCP server in enterprise traffic). Adds cdbartholomew to blog authors. * blog: add truncate marker, featured image, fix Slack invite link - Add <!-- truncate --> after the lead (fixes the build warning addressed repo-wide in #2065) - Add featured/social image and hero image - Replace workspace login URL with the canonical join.slack.com invite * blog: clean up featured image (remove curve overlapping the headline) * blog: add captured star-history chart (Hindsight steepest slope); align featured image to brand palette - Embed a static capture of the overlaid star-history graph in the 'still accelerating' section; Hindsight shows the steepest slope of any project. Replaces the unreliable live-URL embed (rate-limited). - Recolor the featured/OG card to the Hindsight brand palette (#0074d9 -> #009296 gradient, #09090b background) instead of off-palette mint. * blog: add star-history chart to featured image (text left, chart right)
3 tasks
This was referenced Jun 26, 2026
This was referenced Jul 10, 2026
nicoloboschi
added a commit
that referenced
this pull request
Jul 24, 2026
…e hang)
Retain hung forever on the Oracle backend: every retain test burned its 120s
client timeout while the server sat idle, so test-python-client-oracle and
test-typescript-client-oracle only ever reached ~5% of the suite before the
30-minute job limit.
The server was not slow — it was deadlocked. flush_pending_stats() acquires
its own connection, but it was being called while the enclosing
acquire_with_retry(...) block still held one:
async with acquire_with_retry(pool) as conn: # conn checked out
async with conn.transaction(): # SAVEPOINT only
...write facts/entities...
await entity_resolver.flush_pending_stats() # takes a 2nd connection
oracledb does not autocommit and OracleConnection.transaction() is only a
SAVEPOINT, so the write is committed by OracleBackend.acquire() when its block
exits. Connection #2's `UPDATE entities ...` therefore waits on row locks held
by the still-open connection #1, which cannot commit until the call returns —
a circular wait. Oracle never reports ORA-00060 because session #1 is blocked
in Python, not on the database, so it hangs indefinitely instead of erroring.
Move the flush after the acquire block in all three call sites (streaming
retain, delta retain, transfer importer), which is what its own docstring
already required ("must be called AFTER the retain transaction commits") and
which PostgreSQL satisfied only by accident via asyncpg autocommit.
Guarded with an AST lint test rather than a behavioural one: the deadlock
cannot be reproduced against PostgreSQL, which is what the suite runs on.
nicoloboschi
added a commit
that referenced
this pull request
Jul 24, 2026
…ain deadlock (#2948) * ci(oracle): free runner disk space before Oracle jobs The three Oracle jobs run the Oracle 23ai `free` service image, which together with the Python ML deps (torch) exhausts the runner's ~14 GB root disk. Two symptoms, one cause: - uv fails to extract a wheel with "No space left on device (os error 28)" (fast ~2 min failure), and - a near-full disk starves I/O badly enough to trip the 30-minute job timeout. test-python-client-oracle and test-typescript-client-oracle have been red on every open PR (#2941, #2942, #2943) from this, independent of the code under test. Reclaim ~20 GB of preinstalled tooling (the same jlumbroso action the Docker build job already uses) before the Oracle setup step. docker-images stays false here: unlike the Docker build job, the Oracle service container is already running by the time steps execute, so pruning images could disrupt it. The savings come from the tool cache, Android SDK, .NET, Haskell, large apt packages and swap. * ci(oracle): trim disk reclaim to the fast, high-yield options The first pass enabled every reclaim, which cost ~4 minutes of job time — counterproductive on jobs that are already fighting a 30-minute limit. android + dotnet + haskell + swap are a few rm -rf's worth ~16-21 GB, which is ample headroom for the Oracle image plus torch. Dropped: - large-packages: apt-get remove, costs minutes for little extra space; - tool-cache: deletes the preinstalled Python that actions/setup-python then re-downloads, making the job slower rather than faster. * fix(retain): flush entity stats after releasing the connection (Oracle hang) Retain hung forever on the Oracle backend: every retain test burned its 120s client timeout while the server sat idle, so test-python-client-oracle and test-typescript-client-oracle only ever reached ~5% of the suite before the 30-minute job limit. The server was not slow — it was deadlocked. flush_pending_stats() acquires its own connection, but it was being called while the enclosing acquire_with_retry(...) block still held one: async with acquire_with_retry(pool) as conn: # conn checked out async with conn.transaction(): # SAVEPOINT only ...write facts/entities... await entity_resolver.flush_pending_stats() # takes a 2nd connection oracledb does not autocommit and OracleConnection.transaction() is only a SAVEPOINT, so the write is committed by OracleBackend.acquire() when its block exits. Connection #2's `UPDATE entities ...` therefore waits on row locks held by the still-open connection #1, which cannot commit until the call returns — a circular wait. Oracle never reports ORA-00060 because session #1 is blocked in Python, not on the database, so it hangs indefinitely instead of erroring. Move the flush after the acquire block in all three call sites (streaming retain, delta retain, transfer importer), which is what its own docstring already required ("must be called AFTER the retain transaction commits") and which PostgreSQL satisfied only by accident via asyncpg autocommit. Guarded with an AST lint test rather than a behavioural one: the deadlock cannot be reproduced against PostgreSQL, which is what the suite runs on. * test(repair): retry the concurrent index drop on deadlock test_dry_run_creates_nothing still flaked in test-api shard 3. CONCURRENTLY avoids ACCESS EXCLUSIVE but still takes ShareUpdateExclusive, which conflicts with the ShareLock a fresh bank's plain CREATE INDEX holds — and that one cannot be made concurrent, since it runs inside the bank-create transaction. So _drop_bank_indexes can still be picked as the deadlock victim while another xdist worker seeds a bank: Process A waits for ShareUpdateExclusiveLock on memory_units; blocked by B. Process B waits for ShareLock on virtual transaction; blocked by A. The bank-create side already retries (#2943); give the drop the same treatment. The drop is idempotent, so retrying is safe.
This was referenced Jul 27, 2026
nepenth
pushed a commit
to nepenth/hindsight
that referenced
this pull request
Aug 7, 2026
Hindsight vectorize-io#1 on BEAM at 10M tokens — 64.1% vs 40.6% next-best. Includes full tier comparison table, context rot section, and AMB manifesto cross-links. Image is a placeholder pending final asset. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nepenth
pushed a commit
to nepenth/hindsight
that referenced
this pull request
Aug 7, 2026
Hindsight vectorize-io#1 on BEAM at 10M tokens — 64.1% vs 40.6% next-best. Includes full tier comparison table, context rot section, and AMB manifesto cross-links. Image is a placeholder pending final asset. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.