fix(file_ops): Korean/CJK UTF-8 files falsely detected as binary

[copybara-agent]

Problem

head -c 1000 samples by byte count. A Korean/CJK character (3 bytes each) split at byte 999 causes the terminal decoder (errors='replace') to inject a spurious U+FFFD. _is_likely_binary() previously treated any U+FFFD as proof of non-UTF-8 content, causing valid Korean markdown files to be rejected with Binary file — cannot display as text.

High-byte ratio for Korean UTF-8 is ~71%, which also ruled out the non-printable ratio path.

Fix

When U+FFFD is detected in the sample, verify the real file via python3 with strict UTF-8 decoding before declaring it binary. Valid UTF-8 files (Korean, CJK, etc.) now pass through correctly.

Tests

Three regression tests added to TestReadNonUtf8IsBinary:

• test_cjk_utf8_byte_boundary_not_flagged_binary — 400×'가' (1200 bytes); 1000 mod 3=1 guarantees a split → spurious FFFD → must still read as text
• test_real_latin1_file_still_flagged_binary — real latin-1 file must still be caught
• All 4 tests pass (2 existing + 2 new)

[spfcraze]

This was generated by AI during triage.

Summary:
The strict-UTF-8 re-check reads the first 100 characters of the file — 300 bytes for the 3-byte CJK characters in this PR's own test — while the sampled window is 1000 bytes, so the boundary-split U+FFFD it is meant to disambiguate sits beyond the region it verifies.

Problems:

• tools/file_operations.py:916 in this branch runs open(path, encoding='utf-8', errors='strict').read(100): 100 characters, i.e. 300 bytes for the PR's '가' fixture. The spurious U+FFFD from a byte-boundary cut is the sample's last character — for the PR's own 400×'가' file (1200 bytes), head -c 1000 orphans the start byte at byte 999, so the region the re-check reads (bytes 0–299) does not contain it.
• A file whose first ~300 bytes are valid UTF-8 and whose bytes 300–999 are not now returns text, where the pre-fix branch returned binary. The guard comment this PR extends (tools/file_operations.py:896-902 on main) warns that returning lossy text lets a read→edit→write round-trip overwrite the original bytes with mojibake; that hazard is back for exactly this class.

Solution:
Strict-decode the window the sample came from instead of the first 100 characters: open(path, 'rb').read(1000).decode('utf-8', errors='strict') — the boundary-cut case then passes, and non-UTF-8 bytes anywhere in the 1000-byte window still fail the check.

---

Checked against 6099282 — the tip of fix/korean-utf8-binary-detection when this was written — and e7667e5, main at the same moment.