You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reflect operations had no overall timeout—only per-LLM-call timeouts. With high budget (20 iterations × 120s LLM timeout), a single call could hang for ~40 minutes.
Core change
Wrap run_reflect_agent() in asyncio.wait_for() inside reflect_async(), enforcing a configurable wall-clock timeout (default 300s). Both HTTP and MCP callers are protected.
HINDSIGHT_API_REFLECT_WALL_TIMEOUT env var (default 300), documented in configuration.md
reflect_wall_timeout field on HindsightConfig
HTTP layer
Catch TimeoutError in reflect endpoint → HTTP 504
Added responses={504: ...} to FastAPI decorator for OpenAPI spec correctness
Bug fix: main.py startup crash
Replaced the 170-line manual HindsightConfig(...) reconstruction with dataclasses.replace(). The old code would TypeError on any new required field addition (including reflect_wall_timeout).
# Before: TypeError when new fields addedconfig=HindsightConfig(field1=config.field1, field2=config.field2, ...) # 170 lines# After: forward-compatibleconfig=dataclasses.replace(config, host=args.host, port=args.port, log_level=args.log_level)
Code quality
Removed redundant except (asyncio.TimeoutError, TimeoutError) — same class since Python 3.11
Switched new logging calls to lazy % formatting
Added type hints to new test code
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
openaipublic.blob.core.windows.net
Triggering command: /home/REDACTED/work/hindsight/hindsight/.venv/bin/pytest pytest tests/test_reflect_agent.py -v -k TestReflectAgentMocked or TestCleanAnswerText or TestCleanDoneAnswer or TestToolNameNormalization --timeout=60 (dns block)
Triggering command: /home/REDACTED/work/hindsight/hindsight/.venv/bin/pytest pytest tests/test_reflect_agent.py -v -k test_wall_clock_timeout or test_clean_text_with_done_call or test_normalize_standard_name --timeout=60 -p no:randomly (dns block)
releases.astral.sh
Triggering command: /home/REDACTED/.local/bin/uv uv run pytest tests/test_reflect_agent.py -v -k TestReflectAgentMocked or TestCleanAnswerText or TestCleanDoneAnswer or TestToolNameNormalization --timeout=60 (dns block)
If you need me to access, download, or install something from one of these locations, you can either:
Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)
…o#642)
Add a configurable wall-clock timeout (default: 300s / 5 minutes) for
the entire reflect operation. This prevents reflect calls from hanging
for up to 40 minutes when LLM calls are slow or iteration counts are
high.
Changes:
- Add DEFAULT_REFLECT_WALL_TIMEOUT (300s) config constant
- Add HINDSIGHT_API_REFLECT_WALL_TIMEOUT env variable support
- Wrap run_reflect_agent() with asyncio.wait_for() in reflect_async()
- Return HTTP 504 on timeout in the reflect HTTP endpoint
- Add unit test for wall-clock timeout enforcement
Co-authored-by: ThePlenkov <6381507+ThePlenkov@users.noreply.github.com>
Agent-Logs-Url: https://github.com/ThePlenkov/hindsight/sessions/a123d68b-aca1-4040-8bba-8c4f0fab2e2c
CopilotAI
changed the title
[WIP] Fix issue 642 in upstream project
Add wall-clock timeout to reflect operations
Mar 21, 2026
• Add configurable wall-clock timeout (default 300s) for reflect operations
• Wrap run_reflect_agent() with asyncio.wait_for() to enforce timeout
• Return HTTP 504 status on timeout in reflect endpoint
• Add unit test verifying timeout enforcement with asyncio.wait_for()
Diagram
flowchart LR
A["Config: reflect_wall_timeout"] -->|"env: HINDSIGHT_API_REFLECT_WALL_TIMEOUT"| B["HindsightConfig"]
B -->|"timeout value"| C["reflect_async()"]
C -->|"asyncio.wait_for()"| D["run_reflect_agent()"]
D -->|"timeout exceeded"| E["TimeoutError"]
E -->|"caught in HTTP endpoint"| F["HTTP 504 Response"]
Enforce wall-clock timeout on reflect agent execution
• Extract wall_timeout from config in reflect_async() method
• Wrap run_reflect_agent() call with asyncio.wait_for() to enforce timeout
• Catch asyncio.TimeoutError and convert to TimeoutError with descriptive message
• Log timeout event with elapsed time and query preview for debugging
• Add exception handler for asyncio.TimeoutError and TimeoutError in reflect endpoint
• Return HTTP 504 status code on timeout with user-friendly error message
• Log timeout errors for monitoring and debugging
View more (1) 4. hindsight-api-slim/tests/test_reflect_agent.py
🧪 Tests +28/-0
Add unit test for wall-clock timeout enforcement
• Add asyncio import for timeout testing
• Add test_wall_clock_timeout() test method verifying asyncio.wait_for() enforcement
• Test simulates slow LLM call that exceeds short timeout threshold
• Verify asyncio.TimeoutError is raised when timeout is exceeded
The /v1/default/banks/{bank_id}/reflect endpoint now returns 504 on timeout, but the tracked
OpenAPI specs still only document 200 and 422. This can cause client generation and API docs to
be incorrect.
+ except (asyncio.TimeoutError, TimeoutError) as e:+ logger.error(f"Timeout in /v1/default/banks/{bank_id}/reflect: {e}")+ raise HTTPException(+ status_code=504,+ detail=str(e) or "Reflect operation timed out. Consider reducing the budget or simplifying the query.",+ )
Evidence
PR Compliance ID 155340 requires regenerating/updating OpenAPI artifacts when an HTTP endpoint
changes. The PR adds a new 504 timeout response in the handler, but both
hindsight-docs/static/openapi.json and hindsight-clients/go/api/openapi.yaml still define only
200 and 422 responses for the reflect path.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
The reflect endpoint now returns HTTP `504` on timeout, but the committed OpenAPI specs still only document `200` and `422`.
## Issue Context
This PR added timeout handling in the reflect HTTP handler. OpenAPI artifacts (used for docs and client generation) must be regenerated/updated to include the new `504` response.
## Fix Focus Areas
- hindsight-api-slim/hindsight_api/api/http.py[2624-2629]
- hindsight-docs/static/openapi.json[520-587]
- hindsight-clients/go/api/openapi.yaml[350-398]
- scripts/generate-openapi.sh[1-18]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
A new configuration env var HINDSIGHT_API_REFLECT_WALL_TIMEOUT was added, but it is missing from
hindsight-docs/docs/developer/configuration.md in the Reflect section. This violates the
requirement to document new configuration fields (including default and hierarchical/static
classification).
PR Compliance ID 155352 requires new config fields to be documented in the configuration reference.
The code introduces ENV_REFLECT_WALL_TIMEOUT / DEFAULT_REFLECT_WALL_TIMEOUT, but the Reflect
configuration table in the docs lists other reflect variables and does not include
HINDSIGHT_API_REFLECT_WALL_TIMEOUT.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
The new env var `HINDSIGHT_API_REFLECT_WALL_TIMEOUT` is not documented in the configuration reference.
## Issue Context
This PR adds a new Reflect wall-clock timeout setting with default `300` seconds. The docs' Reflect section must include the variable name, description, default value, and whether it is hierarchical.
## Fix Focus Areas
- hindsight-api-slim/hindsight_api/config.py[338-342]
- hindsight-api-slim/hindsight_api/config.py[500-504]
- hindsight-docs/docs/developer/configuration.md[889-896]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
3. New test lacks type hints 📘 Rule violation⚙ Maintainability
Description
The newly added async test test_wall_clock_timeout (and helper slow_llm_call) has no parameter
or return type annotations. This violates the requirement that all Python function definitions in
touched files include type hints.
+ @pytest.mark.asyncio+ async def test_wall_clock_timeout(self, mock_llm, mock_functions):+ """Test that asyncio.wait_for can enforce a wall-clock timeout on run_reflect_agent."""++ async def slow_llm_call(*args, **kwargs):+ await asyncio.sleep(10) # Simulate a slow LLM call+ return LLMToolCallResult(+ tool_calls=[LLMToolCall(id="1", name="recall", arguments={"query": "test"})],+ finish_reason="tool_calls",+ )++ mock_llm.call_with_tools.side_effect = slow_llm_call++ with pytest.raises(asyncio.TimeoutError):+ await asyncio.wait_for(+ run_reflect_agent(+ llm_config=mock_llm,+ bank_id="test-bank",+ query="test query",+ bank_profile={"name": "Test", "mission": "Testing"},+ max_iterations=5,+ **mock_functions,+ ),+ timeout=0.1, # Very short timeout to trigger quickly+ )+
Evidence
PR Compliance ID 155358 requires explicit type annotations for all function definitions in modified
Python files. The new async def test_wall_clock_timeout(self, mock_llm, mock_functions): and
nested async def slow_llm_call(*args, **kwargs): do not provide parameter or return type
annotations.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
Newly added async test functions are missing required parameter and return type annotations.
## Issue Context
The repo compliance rule requires type hints on all `def`/`async def` in touched Python files (except potentially `self`/`cls`). The new test and its nested helper should be annotated (e.g., `-> None`, and `*args: Any, **kwargs: Any`).
## Fix Focus Areas
- hindsight-api-slim/tests/test_reflect_agent.py[421-446]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
View more (1) 4. CLI startup TypeError 🐞 Bug✓ Correctness
Description
Adding the required reflect_wall_timeout field to HindsightConfig breaks the
hindsight_api/main.py path that reconstructs HindsightConfig when --log-level overrides the
env, because that constructor call does not pass reflect_wall_timeout. As a result, starting the
server with a log-level override will raise `TypeError: __init__() missing 1 required positional
argument: 'reflect_wall_timeout'` and exit.
# Reflect agent settings
reflect_max_iterations: int
reflect_max_context_tokens: int
+ reflect_wall_timeout: int
Evidence
reflect_wall_timeout is a required dataclass field (no default on the dataclass field itself), so
any direct HindsightConfig(...) call must include it. main.py reconstructs
HindsightConfig(...) when CLI log-level differs from config, but that kwargs list includes
reflect_max_iterations, reflect_max_context_tokens, and reflect_mission while omitting
reflect_wall_timeout, which will crash at runtime on that code path.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
### Issue description
`HindsightConfig` now requires `reflect_wall_timeout`, but `hindsight_api/main.py` reconstructs a new `HindsightConfig(...)` when `--log-level` overrides the env config without passing `reflect_wall_timeout`, causing a startup-time `TypeError`.
### Issue Context
This regression is introduced by adding a new required dataclass field. Any call sites that manually rebuild `HindsightConfig` must be updated, or (preferably) refactored to avoid enumerating all fields.
### Fix Focus Areas
- hindsight-api-slim/hindsight_api/main.py[152-326]
- hindsight-api-slim/hindsight_api/config.py[803-807]
### Implementation guidance
- Minimal fix: add `reflect_wall_timeout=config.reflect_wall_timeout` to the `HindsightConfig(...)` kwargs in `main.py`.
- More robust fix: replace the manual reconstruction with `dataclasses.replace(config, host=args.host, port=args.port, log_level=args.log_level)` (or equivalent) to prevent future field-addition breakages.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
ⓘ The new review experience is currently in Beta. Learn more
The /v1/default/banks/{bank_id}/reflect endpoint now returns 504 on timeout, but the tracked
OpenAPI specs still only document 200 and 422. This can cause client generation and API docs to
be incorrect.
Agent Prompt
## Issue description
The reflect endpoint now returns HTTP `504` on timeout, but the committed OpenAPI specs still only document `200` and `422`.
## Issue Context
This PR added timeout handling in the reflect HTTP handler. OpenAPI artifacts (used for docs and client generation) must be regenerated/updated to include the new `504` response.
## Fix Focus Areas
- hindsight-api-slim/hindsight_api/api/http.py[2624-2629]
- hindsight-docs/static/openapi.json[520-587]
- hindsight-clients/go/api/openapi.yaml[350-398]
- scripts/generate-openapi.sh[1-18]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
A new configuration env var HINDSIGHT_API_REFLECT_WALL_TIMEOUT was added, but it is missing from
hindsight-docs/docs/developer/configuration.md in the Reflect section. This violates the
requirement to document new configuration fields (including default and hierarchical/static
classification).
Agent Prompt
## Issue description
The new env var `HINDSIGHT_API_REFLECT_WALL_TIMEOUT` is not documented in the configuration reference.
## Issue Context
This PR adds a new Reflect wall-clock timeout setting with default `300` seconds. The docs' Reflect section must include the variable name, description, default value, and whether it is hierarchical.
## Fix Focus Areas
- hindsight-api-slim/hindsight_api/config.py[338-342]
- hindsight-api-slim/hindsight_api/config.py[500-504]
- hindsight-docs/docs/developer/configuration.md[889-896]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
The reason will be displayed to describe this comment to others. Learn more.
3. New test lacks type hints 📘 Rule violation⚙ Maintainability
The newly added async test test_wall_clock_timeout (and helper slow_llm_call) has no parameter
or return type annotations. This violates the requirement that all Python function definitions in
touched files include type hints.
Agent Prompt
## Issue description
Newly added async test functions are missing required parameter and return type annotations.
## Issue Context
The repo compliance rule requires type hints on all `def`/`async def` in touched Python files (except potentially `self`/`cls`). The new test and its nested helper should be annotated (e.g., `-> None`, and `*args: Any, **kwargs: Any`).
## Fix Focus Areas
- hindsight-api-slim/tests/test_reflect_agent.py[421-446]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
The reason will be displayed to describe this comment to others. Learn more.
4. Cli startup typeerror 🐞 Bug✓ Correctness
Adding the required reflect_wall_timeout field to HindsightConfig breaks the
hindsight_api/main.py path that reconstructs HindsightConfig when --log-level overrides the
env, because that constructor call does not pass reflect_wall_timeout. As a result, starting the
server with a log-level override will raise `TypeError: __init__() missing 1 required positional
argument: 'reflect_wall_timeout'` and exit.
Agent Prompt
### Issue description
`HindsightConfig` now requires `reflect_wall_timeout`, but `hindsight_api/main.py` reconstructs a new `HindsightConfig(...)` when `--log-level` overrides the env config without passing `reflect_wall_timeout`, causing a startup-time `TypeError`.
### Issue Context
This regression is introduced by adding a new required dataclass field. Any call sites that manually rebuild `HindsightConfig` must be updated, or (preferably) refactored to avoid enumerating all fields.
### Fix Focus Areas
- hindsight-api-slim/hindsight_api/main.py[152-326]
- hindsight-api-slim/hindsight_api/config.py[803-807]
### Implementation guidance
- Minimal fix: add `reflect_wall_timeout=config.reflect_wall_timeout` to the `HindsightConfig(...)` kwargs in `main.py`.
- More robust fix: replace the manual reconstruction with `dataclasses.replace(config, host=args.host, port=args.port, log_level=args.log_level)` (or equivalent) to prevent future field-addition breakages.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
We reviewed changes in b7abf85...c0d5144 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.
Some issues found as part of this review are outside of the diff in this pull request and aren't shown in the inline review comments due to GitHub's API limitations. You can see those issues on the DeepSource dashboard.
The reason will be displayed to describe this comment to others. Learn more.
Overlapping exceptions (asyncio.TimeoutError and TimeoutError are the same)
Two or more exceptions in the same exception handler clause that are identical or parts of the same inheritance hierarchy (i.e. overlapping). It is recommmend to refactor this.
The reason will be displayed to describe this comment to others. Learn more.
Use lazy % formatting in logging functions
Formatting the message manually before passing it to a logging call does unnecessary work if logging is disabled. Consider using the logging module's built-in formatting features to avoid that.
The reason will be displayed to describe this comment to others. Learn more.
Use lazy % formatting in logging functions
Formatting the message manually before passing it to a logging call does unnecessary work if logging is disabled. Consider using the logging module's built-in formatting features to avoid that.
The reason will be displayed to describe this comment to others. Learn more.
Use lazy % formatting in logging functions
Formatting the message manually before passing it to a logging call does unnecessary work if logging is disabled. Consider using the logging module's built-in formatting features to avoid that.
…edup recall (vectorize-io#1907)
Round-robin interleave fusion for consolidation dedup recall (guarantees the semantic-#1 'twin' a slot so the LLM updates instead of duplicating), unified 'reranking' strategy param (cross_encoder/rrf/interleave), case-sensitive exact-dup guard, obs-dedup tool + benchmark wired into the perf dashboard (English dataset). Near-dup observation rate 4% -> 0% on the English hermes transcript (1/10 and 1/4), coverage 89% -> 94%, no false merges.
…ever (vectorize-io#2092)
* blog: Hindsight is the fastest-growing open-source AI memory project ever
Equal-age GitHub star analysis (per-star timestamps) plus third-party
validation from OSSCAR (#10 fastest-growing OSS org, ahead of Mem0) and
dope.security (#1 MCP server in enterprise traffic). Adds cdbartholomew
to blog authors.
* blog: add truncate marker, featured image, fix Slack invite link
- Add <!-- truncate --> after the lead (fixes the build warning addressed
repo-wide in vectorize-io#2065)
- Add featured/social image and hero image
- Replace workspace login URL with the canonical join.slack.com invite
* blog: clean up featured image (remove curve overlapping the headline)
* blog: add captured star-history chart (Hindsight steepest slope); align featured image to brand palette
- Embed a static capture of the overlaid star-history graph in the
'still accelerating' section; Hindsight shows the steepest slope of
any project. Replaces the unreliable live-URL embed (rate-limited).
- Recolor the featured/OG card to the Hindsight brand palette
(#0074d9 -> #009296 gradient, #09090b background) instead of off-palette mint.
* blog: add star-history chart to featured image (text left, chart right)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
![deepsource-io[bot]](https://avatars.githubusercontent.com/in/16372?s=60&v=4){kind=small}
Reflect operations had no overall timeout—only per-LLM-call timeouts. With high budget (20 iterations × 120s LLM timeout), a single call could hang for ~40 minutes.
Core change
Wrap
run_reflect_agent()inasyncio.wait_for()insidereflect_async(), enforcing a configurable wall-clock timeout (default 300s). Both HTTP and MCP callers are protected.Config & docs
HINDSIGHT_API_REFLECT_WALL_TIMEOUTenv var (default300), documented inconfiguration.mdreflect_wall_timeoutfield onHindsightConfigHTTP layer
TimeoutErrorin reflect endpoint → HTTP 504responses={504: ...}to FastAPI decorator for OpenAPI spec correctnessBug fix:
main.pystartup crashReplaced the 170-line manual
HindsightConfig(...)reconstruction withdataclasses.replace(). The old code wouldTypeErroron any new required field addition (includingreflect_wall_timeout).Code quality
except (asyncio.TimeoutError, TimeoutError)— same class since Python 3.11%formattingWarning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
openaipublic.blob.core.windows.net/home/REDACTED/work/hindsight/hindsight/.venv/bin/pytest pytest tests/test_reflect_agent.py -v -k TestReflectAgentMocked or TestCleanAnswerText or TestCleanDoneAnswer or TestToolNameNormalization --timeout=60(dns block)/home/REDACTED/work/hindsight/hindsight/.venv/bin/pytest pytest tests/test_reflect_agent.py -v -k test_wall_clock_timeout or test_clean_text_with_done_call or test_normalize_standard_name --timeout=60 -p no:randomly(dns block)releases.astral.sh/home/REDACTED/.local/bin/uv uv run pytest tests/test_reflect_agent.py -v -k TestReflectAgentMocked or TestCleanAnswerText or TestCleanDoneAnswer or TestToolNameNormalization --timeout=60(dns block)If you need me to access, download, or install something from one of these locations, you can either:
⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.