fix: guard json.loads() and use atomic writes for persistent state - #29019
fix: guard json.loads() and use atomic writes for persistent state#29019annguyenNous wants to merge 1 commit into
Conversation
JSON decode guards (5 files): - weixin.py: wrap _api_post/_api_get json.loads in try/except - browser_cdp_tool.py: skip malformed CDP WebSocket frames - osv_check.py: return [] on non-JSON OSV API responses - api_server.py: return None on corrupted SQLite cache entries Atomic writes (6 files): - skills_hub.py: use atomic_json_write for lock/taps files - model_metadata.py: use atomic_yaml_write for context cache - delivery.py: use tempfile+fsync+os.replace for delivery output - run.py: use atomic_json_write for voice mode preferences - environments/base.py: use atomic_json_write for JSON store - checkpoint_manager.py: use atomic_json_write for project metadata Resource management (1 file): - vision_tools.py: close PIL Image in finally block to prevent fd leak
teknium1
left a comment
There was a problem hiding this comment.
Thanks for collecting these hardening fixes in one place. Several of the underlying problems are still present on current main, but this branch needs salvage work before it is safe to carry forward.
Problems
tools/vision_tools.pyis stale against current main. Currentorigin/mainhas_resize_image_for_vision(..., max_dimension=...)and requires both the byte cap and_dims_ok(...)before returning (tools/vision_tools.py:372,tools/vision_tools.py:502onorigin/main). The PR branch drops that parameter/guard and returns onlen(candidate) <= max_base64_bytesat PR linetools/vision_tools.py:391, which would regress the newer dimension-cap fix.- The OSV 502/503 behavior is incomplete. In the PR branch, the new guard starts after
urllib.request.urlopen(...)(tools/osv_check.py:150-154), but non-2xx HTTP responses commonly raise beforeresp.read()is reached. - No behavioral tests were added; the PR body only reports py_compile coverage.
Suggested changes
- Reapply the vision fd-close fix on top of current main while preserving
max_dimensionand_dims_ok. - Wrap the OSV
urlopencall itself if transient HTML/non-JSON failures should return[]. - Add focused tests for the malformed JSON and atomic-write paths touched here.
Automated hermes-sweeper review.
| img.save(buf, **save_kwargs) | ||
| encoded = base64.b64encode(buf.getvalue()).decode("ascii") | ||
| candidate = f"data:{out_mime};base64,{encoded}" | ||
| if len(candidate) <= max_base64_bytes: |
There was a problem hiding this comment.
This branch is stale against current main: the current helper also gates success on max_dimension via _dims_ok(...). Salvage should keep that dimension check while adding the image close/finally behavior.
| with urllib.request.urlopen(req, timeout=_TIMEOUT) as resp: | ||
| result = json.loads(resp.read()) | ||
| try: | ||
| result = json.loads(resp.read()) |
There was a problem hiding this comment.
This catches malformed JSON after a successful urlopen, but 502/503 responses often raise HTTPError at urlopen before this line. If the goal is to return [] for transient HTML service failures, wrap urlopen too.
Problem
Multiple files have unguarded
json.loads()calls that crash on malformed/non-JSON responses, and several persistent state files use non-atomicwrite_text()that can corrupt on crash.Fix
JSON decode guards (5 files):
_api_post()and_api_get()now catchjson.JSONDecodeErrorand raise descriptiveRuntimeError[]instead of crashingNoneinstead of propagatingJSONDecodeErrorAtomic writes (6 files):
atomic_json_write()(temp+fsync+replace)atomic_yaml_write()atomic_json_write()atomic_json_write()atomic_json_write()Resource management (1 file):
Image.open()handle now closed infinallyblock to prevent fd leak during resize loopsBefore vs After
json.JSONDecodeErrorcrash on non-200 HTMLRuntimeErrorwith contextTests
py_compilesyntax check