Skip to content

fix(stm32wl): make HardFault UART reporting resilient to a frozen tick count - #11196

Closed
ndoo wants to merge 4 commits into
meshtastic:developfrom
meshmy:fix/stm32wl-hardfault-uart-timeout
Closed

fix(stm32wl): make HardFault UART reporting resilient to a frozen tick count#11196
ndoo wants to merge 4 commits into
meshtastic:developfrom
meshmy:fix/stm32wl-hardfault-uart-timeout

Conversation

@ndoo

@ndoo ndoo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Problem

On STM32WL (reproduced on wio-e5 hardware, under a burst of incoming DMs), a HardFault can leave the device completely locked up with no diagnostic output at all — not even the HardFault! register dump HardFault_Handler_C is supposed to print over serial before blinking SOS.

debug_printf() (src/platform/stm32wl/main-stm32wl.cpp) calls SrcWrapper's uart_debug_write(), whose "wait for the UART to go idle" loop — and HAL_UART_Transmit()'s own internal timeout — are both bounded using HAL_GetTick(), which only advances via the SysTick interrupt. HardFault is fixed at NVIC priority -1 (always the highest priority in the system), so SysTick can never preempt it. If the UART happens to be genuinely mid-transmission when the fault lands (e.g. a LOG_HEAP line was still draining out over serial, which is exactly what was observed), HAL_GetTick() is frozen for as long as we're inside the fault handler — the timeout condition can never become true, and HardFault_Handler_C spins forever inside HAL_UART_GetState() instead of ever printing anything or reaching the SOS blink loop. Confirmed with GDB: the device was found parked in HAL_UART_GetState <- serial_tx_active <- uart_debug_write <- debug_printf <- HardFault_Handler_C, mid-fault, having never printed the "HardFault!\r\n" already sitting in its own message buffer.

This doesn't fix whatever originally triggers the HardFault (that's a separate, harder problem under active investigation) — it fixes the fact that when one does happen, the device hangs silently instead of giving us anything to debug from.

Fix

Bypass the framework's UART TX path entirely for the fault handler's own diagnostic output. faultSafeUartWrite() writes straight to the debug UART peripheral registers (Serial.getHandle()->Instance, which resolves to whatever DEBUG_UART/SERIAL_UART_INSTANCE the variant is actually configured for), polling USART_ISR_TXE_TXFNF/USART_ISR_TC directly and bounding the wait with the DWT cycle counter (DWT->CYCCNT) instead of HAL_GetTick(). The DWT cycle counter is a free-running hardware counter that keeps incrementing regardless of interrupt/exception state, so it's safe to use as a deadline from inside a fault handler where nothing else in the system is running.

Test plan

  • Builds clean (pio run) on all three affected STM32WL environments with different SERIAL_UART_INSTANCE configs: wio-e5 (instance 1), rak3172 (instance 1, default), CDEBYTE_E77-MBL (instance 2) — confirms the Serial-derived peripheral lookup generalizes correctly.
  • Not yet verified on physical hardware (no wio-e5 attached to the machine this was developed on) — opening as draft pending confirmation that a forced HardFault now actually prints its register dump instead of hanging. @ndoo, this is the piece I'd want you to confirm on your board before merging.

🤝 Attestations

  • I have tested that my proposed changes behave as described.
  • I have tested that my proposed changes do not cause any obvious regressions on the following devices:
    • Heltec (Lora32) V3
    • LilyGo T-Deck
    • LilyGo T-Beam
    • RAK WisBlock 4631
    • Seeed Studio T-1000E tracker card
    • Other (please specify below): wio-e5, rak3172, CDEBYTE_E77-MBL — build-verified only, not yet flashed to hardware

Summary by CodeRabbit

  • Bug Fixes
    • Improved HardFault diagnostic output reliability.
    • Prevented diagnostic messages from hanging when system timing interrupts are unavailable during fault handling.
    • Ensured fault messages finish transmitting within a finite timeout.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 52ef0679-de2f-4735-9287-7d81228eba22

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

HardFault diagnostic output on STM32WL now bypasses the HAL UART path, uses direct USART register access with DWT cycle-counter timeouts, and waits for final transmission completion.

Changes

Fault-safe UART diagnostics

Layer / File(s) Summary
Direct timed UART diagnostic output
src/platform/stm32wl/main-stm32wl.cpp
Adds faultSafeUartWrite() for bounded, register-level USART transmission and updates debug_printf to use it for hardfault messages.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: making STM32WL HardFault UART reporting tolerate a frozen tick count.
Description check ✅ Passed The description covers the problem, fix, test plan, and attestations, and includes the required testing caveat for hardware verification.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…k count

uart_debug_write() (STM32duino SrcWrapper) times its "wait for UART idle"
loop, and HAL_UART_Transmit()'s own internal timeout, off HAL_GetTick(),
which only advances via the SysTick interrupt. HardFault is fixed at NVIC
priority -1 (always the highest in the system), so SysTick can never
preempt it: if the UART happens to be genuinely mid-transmission when the
fault lands (e.g. a log line was still draining out), HAL_GetTick() is
frozen for as long as we're inside the fault handler, the timeout can
never trip, and HardFault_Handler_C hangs forever inside HAL_UART_GetState
instead of printing the register dump or reaching the SOS blink loop -
reproduced on wio-e5 hardware under a burst of incoming DMs.

Bypass the framework TX path entirely for the fault handler's own output:
write straight to the debug UART peripheral registers, using the DWT
cycle counter (a free-running hardware counter that keeps incrementing
regardless of interrupt state) to bound the wait instead of HAL_GetTick().

Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Andrew Yong <me@ndoo.sg>
@ndoo
ndoo force-pushed the fix/stm32wl-hardfault-uart-timeout branch from ecdf166 to f1952df Compare July 24, 2026 17:08
@ndoo

ndoo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/platform/stm32wl/main-stm32wl.cpp`:
- Around line 223-225: Shorten the comment above the direct UART debug write to
two lines or fewer, retaining only that HAL_GetTick-based timeout is unsafe in
the fault handler and DWT->CYCCNT provides timing that continues during faults.
Remove the extra explanatory detail while preserving the implementation
unchanged.
- Around line 235-250: Update faultSafeUartWrite so it initializes the DWT cycle
counter deadline once before the byte loop and reuses that single timeout budget
for both TXE/TXFNF waits and the final TC wait. Remove the per-byte start reset,
and return when the elapsed cycles from the shared start reach timeoutCycles,
preserving the existing bounded-failure behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5444b2aa-0e9b-438a-b7f9-79d3331bd969

📥 Commits

Reviewing files that changed from the base of the PR and between 88e287f and f1952df.

📒 Files selected for processing (1)
  • src/platform/stm32wl/main-stm32wl.cpp

Comment thread src/platform/stm32wl/main-stm32wl.cpp Outdated
Comment thread src/platform/stm32wl/main-stm32wl.cpp
@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

⚡ Try this PR in the Web Flasher

Flash this PR in the Web Flasher

firmware commit boards expires

Warning

This is an automated, unreviewed CI test build. Back up your device configuration
before flashing, and only flash devices you are able to recover.

Supported boards built by this PR (31)
Device Board Platform
Crowpanel Adv 3.5 TFT elecrow-adv-35-tft esp32-s3
Heltec HT62 heltec-ht62-esp32c3-sx1262 esp32-c3
Heltec Mesh Node 096 heltec-mesh-node-t096 nrf52840
Heltec Mesh Node T1 heltec-mesh-node-t1 nrf52840
Heltec Mesh Node T114 heltec-mesh-node-t114 nrf52840
Heltec V3 heltec-v3 esp32-s3
Heltec V4 heltec-v4 esp32-s3
Meshnology W10 meshnology_w10 esp32-s3
Meshnology W12 meshnology_w12 esp32-s3
Raspberry Pi Pico pico rp2040
Raspberry Pi Pico W picow rp2040
RAK WisMesh Pocket V3 rak_wismesh_pocket nrf52840
RAK WisMesh Pod rak_wismesh_pod nrf52840
RAK WisMesh Repeater Mini V2 rak_wismesh_repeater_mini nrf52840
RAK WisMesh Tag rak_wismeshtag nrf52840
RAK WisBlock 11200 rak11200 esp32
RAK WisBlock 11310 rak11310 rp2040
RAK3312 rak3312 esp32-s3
RAK WisBlock 4631 rak4631 nrf52840
Seeed SenseCAP Mesh-Tracker-X1 seeed_mesh_tracker_X1 nrf52840
Seeed Wio Tracker L1 seeed_wio_tracker_L1 nrf52840
Seeed Xiao NRF52840 Kit seeed_xiao_nrf52840_kit nrf52840
Seeed Xiao ESP32-S3 seeed-xiao-s3 esp32-s3
Station G2 station-g2 esp32-s3
Station G3 station-g3 esp32-s3
LILYGO T-Deck t-deck-tft esp32-s3
LILYGO T-Echo t-echo nrf52840
LILYGO T-Echo Plus t-echo-plus nrf52840
LILYGO T-Impulse Plus t-impulse-plus nrf52840
LilyGo T3-C6 tlora-c6 esp32-c6
Seeed SenseCAP T1000-E tracker-t1000-e nrf52840

Build artifacts expire on 2026-08-25. Updated for 88ce6f0.

Per-byte timer reset meant a wedged-but-slowly-progressing UART could
stretch the total write out to size * timeoutCycles instead of being
bounded by one deadline. Share a single start point across the byte loop
and the final TC wait instead.

Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Andrew Yong <me@ndoo.sg>
@ndoo

ndoo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai addressed feedback, please review again

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@ndoo I’ll review the updated changes in #11196.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@ndoo

ndoo commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

@t-miura reports that this fixes the hardfault crash report, it can now print fully:

Before:

HEAP | 13:30:55 2579 [RadioIf] [heap 1764] ------ Thread RadioIf leaked heap 2244 -> 1824 (-420) ------
HEAP

After:

HardFault!
r0: e99de361
r1: 87532b73
r2: e99de729
r3: 00008c70
r12: 0000000a
lr: 08031cd1
pc[return address]: 0803138c
xpsr: 41000000

@ndoo
ndoo marked this pull request as ready for review July 25, 2026 06:39
@thebentern thebentern added the bugfix Pull request that fixes bugs label Jul 26, 2026
@ndoo

ndoo commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Closing this - I will raise this as an upstream bug report in STM32duino if it continues to trouble us during hardfault debugging, but currently SWD is a more reliable way to see what is going on during a hardfault anyway.

@ndoo ndoo closed this Aug 10, 2026
@ndoo
ndoo deleted the fix/stm32wl-hardfault-uart-timeout branch August 28, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants