test(evidence): Pixel 6a TTFT/prefill/thermal/4b-tier rows for #11734 — Refs #11734

[lalalune]

What

Captures the four adb-measurable benchmark rows of #11734 on the physical Pixel 6a (serial 27051JEGR10034, policy build 35f61d1f — the same install the #11908 acceptance soak passed on), eliza-1-2b Q4 through the in-process bionic Vulkan host, driven over adb forward tcp:31337. The issue parked these rows as hardware-lab-gated; only battery/power draw (tethered = charging; discharge unmeasurable) and iOS genuinely are.

Full methodology + raw data: .github/issue-evidence/11734-pixel6a-adb-rows/ (per-turn JSONL, session logcat, thermal TSV, memory timelines, analyzers). Non-gating measuredBaselines in packages/benchmarks/mobile-resource/budgets.json + BASELINE.md updated; gating deviceClasses budgets stay null per the multi-run-stability rule.

The four rows

1. TTFT distribution (was: single warm run)

12 identical-length turns: p50 54.3 s / p90 57.7 s (min 52.9, max 58.8). Warm-only p50 54.3 / p90 54.5; 3/12 turns were transparent post-memory-pressure-release reloads (57.3–58.8 s) — deliberately included, that's what this RAM class delivers. Client-side overhead vs the native window is only ~136 ms. Client TTFT = full-turn latency on this build: the fast path emits the entire reply as one SSE chunk because the bionic stream_next decodes its full 256-token buffer in a single native call (maxTokens=20 never engages; every ~9-token reply pays a ~46 s 256-token decode).

2. Isolated prefill tok/s (was: combined-decode only)

8-rung exact-token prompt ladder (92→219 Qwen3.5 tokens), regressing the device-logged in-lock work window (queue- and load-immune; verified against an unqueued turn's wall to 30 ms): W = 32.4 s + 0.1945 s/tok × P (r² 0.78) → marginal prefill ≈ 5.1 tok/s, effective decode ≤ 7.9 tok/s. Prefill is batch-quantized (ELIZA_LLM_N_BATCH=128): flat ≤127 prompt tokens (~53.7 s, decode-dominated), then ~+23 s per extra 128-token batch. Independent cross-check on the identical GGUF + Vulkan lib family with the app stopped: llama-bench pp128 3.88 / pp512 8.86 / tg128 6.78 tok/s. The historical "4.8 tok/s warm decode" baseline is a combined-window rate, now documented as such.

3. Thermal timeline (was: point snapshot)

91.4 min continuous session, 15 s cadence, 363 samples (dumpsys thermalservice HAL + battery; /sys/class/thermal is permission-blocked on this user build). Thermal Status stayed 0 the whole session — the OS throttling ladder never engaged under sustained ~55 s/turn inference from a cool start. Peaks: skin_therm1 42.9 °C, VIRTUAL-SKIN 38.1 °C, charger_skin 47.9 °C, TPU 68 °C, battery 29.2 °C. Bonus bound for the battery row: level fell 76%→70% while USB-powered (sustained inference outdraws the adb supply), then recharged.

4. eliza-1-4b tier (was: not captured) — the kill IS the measurement

Staged the published device-class 4b (bundles/4b/text/eliza-1-4b-256k.gguf, 2.95 GB Q4_K_M, sha256-verified), switched registry+assignments, relaunched, drove one turn with 5 s memory sampling. Result, exactly as the E2B-4.9GB finding predicted: lowmemorykiller killed the foreground app during the load — ApplicationExitInfo reason=3 (LOW_MEMORY), rss=3.4GB, importance=100; GL mtrack 3.22 GB / PSS 3.51 GB at the last sample; MemAvailable 1670→206 MB in 22 s; kill 24 s after boot; Android auto-restart would retry the load (kill loop). The #11760 release policy cannot fire mid-load (the load holds the resident lock). The catalog's minRamGb: 6 floor for this tier is confirmed real on this 5.59 GB device. The 2b was restored and verified with a real turn; the device ends in its original state.

Additional findings (documented in the evidence README + budgets findings)

• The shipped device-tier eliza-1 text GGUFs are Qwen3.5, not Gemma-4: on-device eliza-1-2b-q4.gguf reports general.architecture=qwen35 (base Qwen/Qwen3.5-2B-Base); same for the published 2b/4b -256k/-64k artifacts — only the 8 GB -128k files carry the Gemma-4 cutover. The fast path's Gemma-format template tags are plain text to this model, so decode always fills the 256-token native buffer.
• An autonomous ~5-min background agent job (11 k-char prompt, maxTokens=8192) takes longer than its own period at device prefill speed, so it self-queues and holds the resident-model lock indefinitely, starving chat turns — this poisoned two sweep attempts (kept in evidence) and forced the per-rung relaunch methodology.

Still genuinely lab-gated (issue stays open for these)

• Battery / power draw (idle / chat / voice / background) — needs a bench power meter at the battery/USB terminals.
• iOS rows for both tiers — needs a physical iOS device.
• Multi-run stability (≥3 quiet, cool runs) before promoting anything into gating deviceClasses budgets, and the workflow-lane gate flip.

Refs #11734

🤖 Generated with Claude Code

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c56bb432-e6a7-4295-b3bc-a265e2459631

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch evidence/11734-adb-measurable-rows

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.