Remove references to deprecated MyNodeInfo fields by thebentern · Pull Request #2552 · meshtastic/firmware

thebentern · 2023-06-07T18:22:06Z

No description provided.

github-actions · 2023-06-07T19:02:43Z

🤖 Pull request artifacts

file	commit
`pr2552-firmware-2.1.16.747c674.zip`	`747c674`

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@6e4f51f

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@3efefff

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@747c674

thebentern · 2023-06-08T13:06:38Z

Smoke tested on T-Beam with update and pairing to app. Seems to be in good shape

This reverts commit 81f8054

* Skadoosh * Removing deprecated fields * Remove remaining deprecations * Macro * Macro

None of this was reachable: air_period_tx / air_period_rx file-scope mirrors of airtimes.periodTX/RX, accumulated, rotated and memset in lockstep with them but never read out or serialised. Orphaned when meshtastic#2552 re-pointed the writes at bare globals instead of deleting them. lastUtilPeriod, lastUtilPeriodTX written on every sync, read nowhere airtimes.lastPeriodIndex written on every rotation, read nowhere currentPeriodIndex() computes (secs / 3600) % 8 - a modular-ring index for the one array that is shift-ordered rather than a ring. Its only two uses were the dead field above and a log line. It is the fossil of the same confusion that makes getSilentMinutes() wrong. UtilizationPercentTX() declared, never defined free logAirtime()/airtimeReport() declared, never defined; the latter still carried the array-returning signature the previous commit removed, so it actively misled Also fixes the rotation log line, which read currentPeriodIndex() from inside the loop although the index is advanced before it - on a multi-hour wake it printed the same final value once per rotation. It now reports which of the crossed hours is being rotated. airtimeRotatePeriod() is kept: it has no caller in the tree either, but unlike the above it is a defined public method, so out-of-tree callers are plausible. Measured, not estimated: sizeof(AirTime) 464 -> 456 B, plus 64 B of globals, so -72 B of static RAM. Padding accounts for the difference from the 66 B the plan predicted by counting declared bytes. The whole point of writing the tests first: the suite is green here with zero test changes.

…1362) * Copy airtime reports into a caller buffer instead of exposing the array airtimeReport() returned a pointer into the rotating bucket arrays, so the caller held a handle to state that logAirtime() and every accessor mutate underneath it. Copy into a caller-supplied buffer instead, and report failure for a null buffer, a count past the log depth, or an unknown report type. ContentHandler owns its buffer and hoists getPeriodsToLog() out of the three calls that repeated it. * Cover the AirTime report API and log-dispatch contract Half of AirTime's surface had no tests: which store each report type feeds, what airtimeReport() does when misused, how the first sync seeds itself, and whether calling several entry points in one interval compounds the rotation. Eighteen tests, asserted through the public API rather than the public bucket arrays - those arrays are meant to become private, and a test that reads them would have to be rewritten rather than pinning a contract. Two of them state a convention that was never written down: the report arrays are shift-ordered with slot 0 newest, and slot 0 covers only the time since the last rotation. channelUtilization and utilizationTX use the opposite convention - a modular ring indexed by uptime phase - and reading one as if it were the other is a defect that has already happened once. * Characterise AirTime window decay, TX gates, and sleep behaviour Thirty-three tests in three kinds. Invariants must hold forever; boundaries pin off-by-ones a refactor would move; five characterisations encode today's wrong numbers, each tagged with the phase that will flip it. Readings are asserted against an event-log oracle - airtime physically on air inside (now - window, now], computed from a list of completed packets - rather than against hand-worked constants, so a test states "this matches the definition" instead of "this looked right when I wrote it". The characterisations, all measured rather than assumed: - the window covers (N-1)p + phase but divides by Np, so a steady 10% load reads 8.33% right after a bucket boundary -> phase 5 - the same load sweeps across bucket phase instead of holding -> phase 5 - the hour window carries the same defect, 10x smaller -> phase 5 - a packet longer than its bucket is credited whole to the bucket it completed in, so a saturated LONG_SLOW channel reads >100% -> phase 4b - getSilentMinutes() reads a modular ring as if the index were an age, so identical airtime gives different answers by phase -> phase 6 Two tests needed correcting during the write, both my expectations rather than the code: a six-bucket ring sheds whole buckets, so a 30s gap drops three of five survivors and not "half"; and the oracle sees 59 completions in a 60s window, not 60, because the one on the lower edge is outside it. Not written: the planned RX_LOG/RX_ALL_LOG disjointness test. That is a property of the two radio drivers, which choose one or the other per packet - it is not observable from AirTime, which records what it is told. The AirTime-side half is already covered by the routing tests. * Drop write-only and undefined AirTime members None of this was reachable: air_period_tx / air_period_rx file-scope mirrors of airtimes.periodTX/RX, accumulated, rotated and memset in lockstep with them but never read out or serialised. Orphaned when #2552 re-pointed the writes at bare globals instead of deleting them. lastUtilPeriod, lastUtilPeriodTX written on every sync, read nowhere airtimes.lastPeriodIndex written on every rotation, read nowhere currentPeriodIndex() computes (secs / 3600) % 8 - a modular-ring index for the one array that is shift-ordered rather than a ring. Its only two uses were the dead field above and a log line. It is the fossil of the same confusion that makes getSilentMinutes() wrong. UtilizationPercentTX() declared, never defined free logAirtime()/airtimeReport() declared, never defined; the latter still carried the array-returning signature the previous commit removed, so it actively misled Also fixes the rotation log line, which read currentPeriodIndex() from inside the loop although the index is advanced before it - on a multi-hour wake it printed the same final value once per rotation. It now reports which of the crossed hours is being rotated. airtimeRotatePeriod() is kept: it has no caller in the tree either, but unlike the above it is a defined public method, so out-of-tree callers are plausible. Measured, not estimated: sizeof(AirTime) 464 -> 456 B, plus 64 B of globals, so -72 B of static RAM. Padding accounts for the difference from the 66 B the plan predicted by counting declared bytes. The whole point of writing the tests first: the suite is green here with zero test changes. * Document what the AirTime figures measure and how they are stored Comments only, but four of the things they replace were false. The header's example analytics claimed RX_ALL_LOG was "all received lora packets" and offered "RX_ALL_LOG - RX_LOG = other lora radios". Both radio drivers pick exactly one of the two per packet, so they are disjoint: RX_ALL_LOG is airtime we could not parse, the subtraction can go negative, and the total is TX + RX + RX_ALL. Replaced with the actual contract - four inputs, eight outputs, the window each spans, and the fact that the three thresholds are hard-coded members rather than the settings they look like. Names the two storage conventions on their declarations, because mixing them up is what makes getSilentMinutes() wrong: channelUtilization and utilizationTX are modular rings indexed by uptime phase, where the oldest bucket is (current + 1) % N; airtimes.period* is shift-ordered with slot 0 newest, where the index IS an age and slot 0 is a partial hour. Defines the measurement as wall time rather than awake time, and says why: a sleeping node still hears traffic, and per-node redefinition would make two broadcast readings incomparable. Records that the 60s figure is published to the mesh at >= 1h cadence, so what other nodes see is a snapshot - at LONG_FAST and 1% occupancy it reads exactly 0 in about 44% of reports - and that the contention window it feeds moves in 20-percentage-point steps, so small errors never reach the backoff. Finally, states that rotation happens on access rather than on the scheduler tick, names the test that enforces it, and leaves a TODO pointing at the plan phases that fix the characterised accuracy defects. * Serialise AirTime behind a lock proven by a private token Two mechanisms solving different halves. A lock-free inner core (Windows) holds all state and all logic; it has no lock and no way to reach one, so nesting is impossible by construction. A private Held token takes the lock in its own constructor and is the only thing that can be passed where a core method demands one, so the lock cannot be forgotten either. The rule is now uniform with no exceptions to remember: every public method takes the lock once and delegates. In particular isTxAllowed*() lock like everything else - before the split they could not, because they called the public accessors and the lock is not recursive. That asymmetry was the foot-gun the previous design documented in prose and hoped nobody would trip. getPeriodsToLog()/getSecondsPerPeriod() still take no lock; they return compile-time constants and touch no state. channelUtilization[] and utilizationTX[] were public, so the lock was bypassable at compile time. They move into the private core. Four test sites reached in; all four now use logAirtime() plus the virtual clock, and no new test seam was needed. Nothing in src/ was affected. The re-entry assert is guarded on PIO_UNIT_TESTING, so it exists in test builds only. The design sketched #ifdef DEBUG, but nothing in this tree defines DEBUG or NDEBUG, so either spelling ships the assert to every board - and nrf52_promicro_diy_tcxo has ~128 bytes of headroom under its 0xEA000 warm-store cap, which the assert's strings and abort path overrun. It would have worked on hardware, since the check runs in Held's owner initialiser and so precedes the blocking take; the objection is that abort()ing a live mesh node is a poor trade for a bug never seen in the field. Native tests are where it earns its keep anyway: Portduino compiles Lock::lock() to an empty body, so a nested take there succeeds silently and nothing else would notice. Also comments out ScopedBusyAirTime in test_traffic_management. It is inert twice over: the module holds no reference to airTime at all since hop exhaustion was shelved, and the fixture never worked anyway - writing the buckets on a fresh AirTime is undone by the first accessor call, which takes the firstTime branch and memsets them. It reported 0%, not the 100% it claimed. Left in place, commented, with both reasons recorded. Cost on the tightest board in the tree, nrf52_promicro_diy_tcxo: the six phases together add 96 bytes of flash, leaving it 32 bytes clear of the warm-store guard. RAM is 72 bytes lower from the dead-state removal. Suite green at 47/47, with test_airtime unedited apart from the added nesting test. * Count rotations with the loop variable, not a separate tally LOG_DEBUG compiles to nothing under DEBUG_MUTE, so the counter's only read disappeared with it and the tally became write-only. It does not warn today - this build has -Wunused-but-set-variable on, and it fires for other locals, but not for one that is only initialised and never read - so it was latent rather than broken: a stricter flag or -Werror would have failed muted builds only. Using the loop variable removes the class of problem, since the loop condition reads it, and drops the elapsedAirtimePeriods-- mutation as a side benefit. Same iteration count, same output. Found by compiling nrf52_promicro_diy_tcxo with -D DEBUG_MUTE, which is worth recording for its own sake: muting logs takes that image from 802 784 to 673 416 bytes, 98.5% to 82.6% of flash. Logging is 16% of the largest nrf52 image, and its 32 bytes of warm-store headroom are a logging-verbosity question rather than a code-size one. * Tighten the comments added by this branch Comment-only: with comments stripped, all five files are byte-identical to the previous commit. Removed the references to the planning notes. Those documents are working material and will go stale; the code should not depend on them. The five CHARACTERISATION tags now describe the defect they pin and stop there, and the accuracy TODO names the four defects and points at the tests instead of a plan file. Also removed, as noise rather than information: - comparisons against pre-#11291 behaviour, which nobody reading this needs - a comment describing the lock restructure as future work, written before it landed - speculation ("plausible", "worth pinning so a future...") - an aside arguing with an arithmetic slip made while writing the test Kept the mechanical facts that are slow to re-derive: the two storage orderings and which array uses which, RX_LOG/RX_ALL_LOG disjointness, the locking rule and the addSpanned() constraint that protects it, why the re-entry assert is test-only, and the concrete numbers - (N-1)p + phase, 14 164 ms, the 20 pp contention-window steps. Net 16 comment lines out of src/, 33 out of test/. * Gate the AirTime re-entry check on the host, not on testing PIO_UNIT_TESTING is injected by PlatformIO purely on BUILD_TYPE, with no platform check, so it is defined on an on-target `pio test` run too. The check arms before the lock is taken - a nested take blocks forever, so a later check would never run - which under preemption false-positives on legitimate contention and races on its own write. Derive AIRTIME_REENTRY_CHECK once from PIO_UNIT_TESTING && !HAS_FREE_RTOS and use it at all three sites. Had the three conditions ever diverged, an on-target test build would fail to compile on a member the header no longer declares. * Log AirTime outside the lock it serialises DEBUG_PORT.log() blocks on a UART write, and `lock` is a plain binary semaphore with no priority inheritance, so holding it across a log call lets the main thread stall the radio thread in getTxDelayMsec(). Move logAirtime()'s LOG_DEBUG into the shell, after the Held scope closes; the shell already has both arguments, so nothing has to be passed back out of the core. isTxAllowed{ChannelUtil,AirUtil} read into a local under the lock and warn after it. The log bodies are braced because LOG_DEBUG compiles away under DEBUG_MUTE and a bare `if (x) ;` trips -Wempty-body. Fold the two doubled index calls into `+=` while touching the lines. * Give each airtime report its own buffer handleReport() reused one array across the three airtimeReport() calls and ignored the bool. A failed report would have left the previous type's data in place and emitted it under the next type's key. Build each through a lambda whose buffer is zeroed per call, so a failure emits zeros. Unreachable today - the count is always PERIODS_TO_LOG and the type is always valid - but the old shape only read as correct by accident. * Drop a stray semicolon from the inert-guard comment * Address external review: name the race, tighten the claims and the tests The header sold the lock as mechanism without naming a second thread, which invites the reasonable objection that this is a cooperative OSThread codebase. There is a real race and it is nRF52-only: NRF52Bluetooth registers its ToRadio write callback with defer == false, so a phone's packet runs handleToRadio -> sendToMesh -> Router::send on the Bluefruit BLE task, reading utilizationTXPercent() and getSilentMinutes() while loopTask may be inside logAirtime(). ESP32 hands BLE work to the main task and does not have it. Three claims in the header were wrong or overstated: - "nesting is impossible by construction" - Windows is a nested class with an enclosing class's access rights, and `extern AirTime *airTime` is in the same header, so airTime->anyPublicMethod() from inside it is well-formed and would hang. Nothing does it; the assert is the backstop. Say that instead, because the comment below instructs contributors to add helpers to Windows on the strength of the guarantee. - "every public method takes the lock exactly once" - two constant accessors take none and isTxAllowedAirUtil() takes it zero or one times. State the exceptions where the invariant is stated, not only at the definitions. - "both radio drivers pick exactly one per packet" - five drop paths log neither. At most one. Recorded against plan4 rather than fixed here: it changes a telemetry value. getPeriodsToLog()/getSecondsPerPeriod() become static constexpr, which removes them from the locking claim structurally and lets ContentHandler size its buffer and its count from one constant. Tests: - C14's saturated AirTime is installed by a helper and restored in tearDown. Unity's TEST_ABORT() is longjmp and does not run destructors of automatic objects, so the scoped guard it replaces would leave airTime dangling into an abandoned frame on any assertion failure - and the same commit that added it removed the tearDown reset that did cover that. - test_getSilentMinutes_counts_minutes_until_enough_ages_out asserted only `mins <= 60`, which neither return path can violate. The answer is 59. - test_backwards_uptime_degrades_safely stepped 600s -> 60s, which leaves elapsedAirtimePeriods at 0, so it never reached the hourly-report branch its own comment describes. Step by the wrap instead and assert the exact figures. - test_airtime leaked EU_868 out of the duty-cycle case into every later one, and the reentry test's isTxAllowedAirUtil() coverage depended on it. Restore the region in tearDown and set it explicitly where it is wanted. - Rename that test to what it can actually check: no single method takes the lock twice. The calls are sequential, so it cannot catch two methods nesting. * trunk: suppress trufflehog/Lob false positives in test_airtime * Address CodeRabbit review: the rotate trace, the cap warn, the backoff Four findings from the CodeRabbit pass. Two were introduced by this branch, one is a real inconsistency it inherited, one is a naming slip. The rotate trace was the one that mattered. "Log AirTime outside the lock it serialises" moved the per-packet lines and the two TX-gate warnings out to the shell, but missed LOG_DEBUG("Rotate airtimes, crossed hour %u") because it does not sit in the shell at all: it is inside Windows::syncNow(), the lock-free core, which by construction only ever runs under Held. Nothing at that line looks like a lock, which is why it survived. The exposure is smaller than the review suggests - runOnce() syncs at 1 Hz, so in steady state this is one line an hour, and the PERIODS_TO_LOG - 1 burst needs an hour of light sleep with no intervening sync - but a UART write under a plain binary semaphore with no priority inheritance is exactly what the comment above logAirtime() says this code does not do. syncNow() now accumulates crossings in rotationsPendingLog and runOnce() drains it inside the Held scope, then logs after release. Any caller can cross an hour; only that thread reports it, so a crossing raised elsewhere is traced at most one tick late. The `if (rotations > 0)` guard keeps the drained value read under DEBUG_MUTE, where LOG_DEBUG expands to nothing - the write-only tally that "Count rotations with the loop variable" removed. addFromContact()'s favorite fallback stamped silently when the protected cap refused it. The stamp is new on this branch; the two sibling refusals (ignore, verify) both emit PROTECTED_CAP_WARN_FMT, so the operator lost the only signal that the cap was hit on the one path that has a fallback. lfs_assert() mixed clocks: Throttle read Time::getMillis(), the remainder was computed from a second, bare millis(). The review's stated failure mode - a native test overriding the clock - cannot happen, since the hook is behind PIO_UNIT_TESTING and this file is nRF52-only. The real defect is the second read: a tick landing on the 20-minute boundary between the check and the subtraction underflows the remainder into delay(~50 days), on a device that has just found its flash corrupt. One read, clamped, and preFSBegin() stores from the same clock. The eviction test is renamed to test_eviction_prefersCurrentBootStampOverPost2038Epoch. The finding is right that it was snake_case, but the suggested testEvictionPrefers... does not match this file either, which is test_<area>_<camelCase> throughout. Not taken, both pre-existing and out of scope for a rollover branch: - t5s3_epaper's touchResumeAtMs/suppressFromMs read an active suppression as inactive if the wake lands in the 1 ms where millis() is 0. Consequence is one skipped 150 ms touch-settle window per 49.7-day wrap. - NRF52Bluetooth::onPairingPasskey() busy-waits 30 s in a BLE callback. Worth saying plainly that this branch makes it more visible: the old `millis() < start_time + 30000` overflowed at the wrap and cut the wait short, so the correct Throttle form is what lets it run the full 30 s. Reworking it into an OSThread is its own change. Native suite GREEN, 48/48, 672 cases.

* Skadoosh * Removing deprecated fields * Remove remaining deprecations * Macro * Macro

thebentern added 2 commits June 7, 2023 06:19

Skadoosh

c51b04f

Removing deprecated fields

6e4f51f

thebentern requested review from GUVWAF and caveman99 June 7, 2023 18:22

Merge branch 'master' into remove-mac

3efefff

thebentern added a commit to meshtastic/artifacts that referenced this pull request Jun 7, 2023

Upload pr2552-firmware-2.1.16.3efefff.zip (6e4f5)

dcaf014

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@6e4f51f

thebentern added a commit to meshtastic/artifacts that referenced this pull request Jun 7, 2023

Upload pr2552-firmware-2.1.16.3efefff.zip (3efef)

27c0554

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@3efefff

GUVWAF reviewed Jun 7, 2023

View reviewed changes

Comment thread src/airtime.cpp Outdated

thebentern added 3 commits June 7, 2023 21:05

Remove remaining deprecations

5ee40a5

Macro

e317e7c

Macro

747c674

thebentern added a commit to meshtastic/artifacts that referenced this pull request Jun 8, 2023

Upload pr2552-firmware-2.1.16.747c674.zip (747c6)

d317892

Pull request: meshtastic/firmware#2552 Commit: meshtastic/firmware@747c674

GUVWAF approved these changes Jun 8, 2023

View reviewed changes

thebentern merged commit 81f8054 into master Jun 8, 2023

thebentern changed the title ~~Remove deprecated MyNodeInfo fields~~ Remove references to deprecated MyNodeInfo fields Jun 8, 2023

andrekir added a commit to andrekir/Meshtastic-device that referenced this pull request Jun 8, 2023

Revert "Remove deprecated MyNodeInfo fields (meshtastic#2552)"

ae65eca

This reverts commit 81f8054

jeek pushed a commit to jeek/Meshtastic-Exploiteers-Hacker-Pager that referenced this pull request Jun 30, 2026

Remove deprecated MyNodeInfo fields (meshtastic#2552)

861d3cc

* Skadoosh * Removing deprecated fields * Remove remaining deprecations * Macro * Macro

NomDeTom mentioned this pull request Aug 6, 2026

Serialise AirTime behind a lock, and stop handing out its buckets #11362

Merged

8 tasks

Itzdavid01 pushed a commit to Itzdavid01/firmware that referenced this pull request Sep 5, 2026

Remove deprecated MyNodeInfo fields (meshtastic#2552)

ee6986b

* Skadoosh * Removing deprecated fields * Remove remaining deprecations * Macro * Macro

Navigation Menu

Uh oh!

Uh oh!

Remove references to deprecated MyNodeInfo fields - #2552

Remove references to deprecated MyNodeInfo fields#2552
thebentern merged 6 commits into
masterfrom
remove-mac

thebentern commented Jun 7, 2023

Uh oh!

github-actions Bot commented Jun 7, 2023 •

edited

Loading

Uh oh!

Uh oh!

thebentern commented Jun 8, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

thebentern commented Jun 7, 2023

Uh oh!

github-actions Bot commented Jun 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Pull request artifacts

Uh oh!

Uh oh!

thebentern commented Jun 8, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jun 7, 2023 •

edited

Loading