From 2f806079361e18f9045eb49ddeb5db5b7d20a58d Mon Sep 17 00:00:00 2001 From: nomdetom Date: Thu, 13 Aug 2026 13:17:14 +0100 Subject: [PATCH] docs: move the firmware design docs to the documentation site The five documents under docs/ were written in this repo while their features were developed. Four of them describe shipped, upstream behaviour and belong on meshtastic.org, where users and client authors will look for them: traffic_management_module.md -> configuration/module/traffic-management + development/reference/traffic-management-internals node_info_stores.md -> development/reference/node-info-stores mesh_beacon_module.md -> configuration/module/mesh-beacon + development/reference/mesh-beacon-internals + development/device/mesh-beacon-client-interface lora_region_preset_compatibility_client_spec.md -> development/device/region-preset-compatibility Each is split by audience: settings pages carry the config surface in user terms, reference pages carry firmware mechanism, and the device pages carry the protocol a client app speaks. The region-preset spec always said it should graduate out of this repo once its protobuf landed upstream, which it has (FromRadio.region_presets, field 19). nexthop-routing-reliability.md is not documentation - it is a working document with a mitigation plan, a "files to modify" list and commit sequencing. Its mitigations shipped in #10745, so the plan is history and the analysis is superseded; it is dropped rather than published. Comments that cited the deleted files now point at the published pages, and the NextHop test header cites #10745 instead of the deleted plan. --- ...region_preset_compatibility_client_spec.md | 293 ----------- docs/mesh_beacon_module.md | 454 ----------------- docs/nexthop-routing-reliability.md | 456 ------------------ docs/node_info_stores.md | 321 ------------ docs/traffic_management_module.md | 222 --------- src/modules/TrafficManagementModule.cpp | 9 +- src/modules/TrafficManagementModule.h | 15 +- test/test_nexthop_routing/test_main.cpp | 2 +- 8 files changed, 17 insertions(+), 1755 deletions(-) delete mode 100644 docs/lora_region_preset_compatibility_client_spec.md delete mode 100644 docs/mesh_beacon_module.md delete mode 100644 docs/nexthop-routing-reliability.md delete mode 100644 docs/node_info_stores.md delete mode 100644 docs/traffic_management_module.md diff --git a/docs/lora_region_preset_compatibility_client_spec.md b/docs/lora_region_preset_compatibility_client_spec.md deleted file mode 100644 index bb1749672f3..00000000000 --- a/docs/lora_region_preset_compatibility_client_spec.md +++ /dev/null @@ -1,293 +0,0 @@ -# LoRa Region → Preset Compatibility - Client Implementation Spec - -**Status:** Draft for 2.8 · **Audience:** Meshtastic client app developers (Android first, -Apple second, then web/python) · **Firmware side:** implemented in `firmware` -(`FromRadio.region_presets`, see below). - -> This document lives in the firmware repo while the feature is developed. It is meant to -> graduate to `meshtastic/protobufs` (and/or the docs site) alongside the upstream protobuf -> PR that reserves `FromRadio` field **19**. - ---- - -## 1. Why this exists - -For 2.8 the LoRa regions and modem presets were reworked. **Not every modem preset is legal -in every region** - narrow EU SRD bands, the EU 868 "narrow" band, amateur/ham bands, and -the 2.4 GHz band each accept only a specific subset of presets. The firmware already -enforces this internally (it clamps or rejects illegal combinations), but until now a client -had no way to _know_ the rules, so a user could pick an illegal region+preset pair in the UI -and only discover the problem after the device silently corrected it. - -This feature has the firmware **declare the legal region→preset combinations** to the client -during the `want_config` handshake, so the client UI can constrain the preset picker to the -valid set for the currently selected region (and warn about licensed-only bands). It is -purely advisory metadata - the firmware remains the source of truth and still -validates/clamps on its own. - ---- - -## 2. Protocol additions - -Three new messages in `meshtastic/mesh.proto`, plus one new `FromRadio` oneof variant. - -### 2.1 `FromRadio.region_presets` (field 19) - -```proto -message FromRadio { - uint32 id = 1; - oneof payload_variant { - // ... fields 2..18 unchanged ... - LoRaRegionPresetMap region_presets = 19; - } -} -``` - -### 2.2 Messages - -```proto -// A distinct set of legal modem presets shared by one or more LoRa regions. -message LoRaPresetGroup { - repeated Config.LoRaConfig.ModemPreset presets = 1; // legal presets for this group - Config.LoRaConfig.ModemPreset default_preset = 2; // always one of `presets` - bool licensed_only = 3; // ham/amateur band → warn/gate -} - -// Associates a single LoRa region with its preset group (by index). -message LoRaRegionPresets { - Config.LoRaConfig.RegionCode region = 1; - uint32 group_index = 2; // index into LoRaRegionPresetMap.groups -} - -// The full map, delivered grouped to fit one FromRadio packet. -message LoRaRegionPresetMap { - repeated LoRaPresetGroup groups = 1; // each distinct preset list - repeated LoRaRegionPresets region_groups = 2; // every known region → a group index -} -``` - -### 2.3 Why grouped (and the size envelope clients should respect) - -A `FromRadio` packet is capped at **512 bytes** (`MAX_TO_FROM_RADIO_SIZE`). Most regions -share one identical preset list (the "standard" 10-preset list), so the map is delivered -**grouped**: `groups` holds each _distinct_ preset list once, and `region_groups` maps every -known region to one of those groups by index. This keeps the encoded size additive -(`groups` + `region_groups`) rather than multiplicative, well under the cap. - -nanopb (firmware) array bounds - clients do **not** need to enforce these, but they bound -what you can receive: - -| field | max_count | -| ----------------------------------- | ------------------------------------ | -| `LoRaRegionPresetMap.groups` | 8 | -| `LoRaRegionPresetMap.region_groups` | 38 (= number of `RegionCode` values) | -| `LoRaPresetGroup.presets` | 11 | - ---- - -## 3. When it is delivered - -`region_presets` is sent **once** during the `want_config` handshake, as a single -`FromRadio` message, in this position: - -```text -my_info → (deviceuiConfig) → node_info(self) → metadata → region_presets → channel… → config… → moduleConfig… → node_info(others)… → fileInfo… → config_complete_id → (live packets) -``` - -i.e. **immediately after `metadata` and before the first `channel`**. - -- It is included for a normal full `want_config` and for the **config-only** nonce. -- It is **omitted** for the **nodes-only** nonce (that path skips metadata/config entirely). -- A client must **not** assume it always arrives (see §5). - ---- - -## 4. Decoding into a usable lookup - -Flatten the grouped wire form into `Map`: - -```text -struct RegionPresetInfo { Set presets; ModemPreset default; bool licensedOnly } - -fun decode(map: LoRaRegionPresetMap): Map { - result = {} - for (rg in map.region_groups) { - if (rg.group_index >= map.groups.size) continue // defensive: malformed/forward data - g = map.groups[rg.group_index] - result[rg.region] = RegionPresetInfo( - presets = g.presets.toSet(), - default = g.default_preset, - licensedOnly = g.licensed_only) - } - return result -} -``` - -Persist this map alongside the rest of the downloaded config so the LoRa config screen can -read it synchronously. - ---- - -## 5. Semantics & rules (the load-bearing part) - -These rules are what keep the UX correct across firmware versions. Implement all of them. - -1. **Absent region ⇒ no constraint.** If a `RegionCode` does not appear in `region_groups`, - the client has _no_ compatibility info for it and **must not restrict** its preset - choices (fall back to allowing the full `ModemPreset` list). This happens for a handful - of `RegionCode` enum values that have no firmware band table entry (today: `EU_874`, - `EU_917`, `ITU1_70CM`, `ITU2_70CM`, `ITU3_70CM`). - -2. **Absent message ⇒ no constraint.** Firmware older than 2.8 never sends `region_presets`. - New clients **must** tolerate the message being absent entirely and keep their existing - (unconstrained) behavior. Do not block the config screen waiting for it. - -3. **`default_preset`** is always a member of that group's `presets`. Use it to pre-select a - preset when the user switches to a region whose valid set does not include the currently - selected preset (instead of leaving an illegal selection or guessing). - -4. **`licensed_only`** marks ham/amateur bands. Surface a warning or gate (the firmware also - requires the operator's `is_licensed` flag for these regions; coordinate the two so the - user isn't allowed to pick a licensed band without acknowledging licensing). - -5. **EU region auto-swap caveat.** The firmware treats the EU sibling regions - (`EU_868` / `EU_866` / `EU_N_868`) specially: if the user is in one of them and selects a - preset that belongs to a sibling's list, the firmware **swaps the region** rather than - rejecting the preset. To make this visible in the picker, the firmware advertises the - **same superset** (the union of the trio's presets) for all three sibling regions, so a - client filtering per §6 will offer every EU 86x preset regardless of which sibling is - currently selected. Consequence for clients: **do not assume the region is immutable - across a preset change** - after an admin config write, re-read the resulting - `LoRaConfig` and reflect the (possibly changed) region back into the UI. - -6. **Use it as a UI guard, not a validator of truth.** The firmware still validates/clamps - on its own. The map exists to prevent the user from _selecting_ an illegal combo; it is - not a security or correctness boundary. - ---- - -## 6. UI/UX recommendations - -- In the LoRa config screen, when a region is selected, **filter/enable the modem-preset - picker to that region's `presets`** (when `use_preset`/`use_modem_preset` is on). -- If the current preset is not in the newly selected region's set, switch the selection to - that region's `default_preset`. -- Show a **licensed badge / confirmation** for regions where `licensed_only == true`. -- If a region is absent from the map (rule §5.1) or the whole message is absent (§5.2), - render the full preset list as before - never show an empty picker. - ---- - -## 7. Forward / backward compatibility - -- **Old clients, new firmware:** an unknown `FromRadio` oneof variant (field 19) is ignored - by protobuf/nanopb decoders; the relative ordering of the known messages is unchanged, so - existing apps are unaffected. -- **New clients, old firmware:** message simply never arrives → treat as "no constraints" - (§5.2). -- **Enum growth:** new `RegionCode`/`ModemPreset` values may appear over time. Decoders - should pass through unknown enum values rather than crashing; an unknown region in - `region_groups` is harmless (the client just won't have a localized name for it). - ---- - -## 8. Platform notes - -> Verified against the `main` branch of each repo. Both have been refactored away from -> older layouts; re-pin file paths against a specific commit if you need them durable. - -### 8.1 Android - `meshtastic/Meshtastic-Android` (Kotlin / Compose, KMP) - -- **Protobufs are a published Maven artifact, _not_ a submodule.** Declared in - `gradle/libs.versions.toml` (`org.meshtastic:protobufs`, currently `2.7.25`); generated - package is **`org.meshtastic.proto`**. **A `region_presets`-aware build requires a new - published `org.meshtastic:protobufs` release**, then bumping that one version string. -- **The protobufs are Wire-generated**, so the `FromRadio` oneof is **not** a - `payloadVariantCase` enum - each arm is a **nullable field**. Handle the new variant in - `FromRadioPacketHandlerImpl.handleFromRadio(...)` - (`core/data/.../manager/FromRadioPacketHandlerImpl.kt`) by adding a - `regionPresets != null -> …` arm to the existing `when { … }`, delegating to a handler - (mirror `handleLocalMetadata` / `handleConfigComplete`). -- **State holder:** expose the decoded map from `RadioConfigRepository` / - `RadioConfigRepositoryImpl` as a `Flow` (mirroring `localConfigFlow`/`channelSetFlow`), - consumed by `feature/settings/.../radio/RadioConfigViewModel.kt`. -- **UI:** the region & preset dropdowns are `DropDownPreference`s in - `feature/settings/.../radio/component/LoRaConfigItemList.kt` (public composable - `LoRaConfigScreen`). Gate/filter the `ChannelOption` (preset) dropdown by the selected - `RegionInfo`'s entry in the map. - -### 8.2 Apple - `meshtastic/Meshtastic-Apple` (Swift / SwiftUI) - -- **Protobufs are vendored** into a local Swift package `MeshtasticProtobufs` - (`MeshtasticProtobufs/Sources/meshtastic/*.pb.swift`), generated from the `protobufs` git - submodule via `scripts/gen_protos.sh`. **To get field 19:** advance the `protobufs` - submodule, run `scripts/gen_protos.sh`, commit the regenerated `.pb.swift` + submodule - pointer. (No published-artifact dependency - Apple can regenerate from any commit.) -- **Dispatch:** `AccessoryManager.processFromRadio(_:)` - (`Meshtastic/Accessory/Accessory Manager/AccessoryManager.swift`) is a real - `switch decodedInfo.payloadVariant { … }` - add a `.regionPresets` case, with the handler - in `AccessoryManager+FromRadio.swift` (mirror `handleConfig` / `handleMetadata`). -- **Persistence:** config is **SwiftData** (`@Model` entities), upserted via - `MeshPackets`/`UpdateSwiftData.swift`. Store the decoded map (e.g. on a settings/connection - model) so the LoRa view can read it. -- **UI:** `Meshtastic/Views/Settings/Config/LoRaConfig.swift` (`struct LoRaConfig: View`) - has the `Picker("Region", …)` (`RegionCodes.userSelectable`) and `Picker("Presets", …)` - (`ModemPresets.userSelectable`, gated on `usePreset`). Filter the presets picker by the - selected region's entry. Enums live in `Meshtastic/Enums/LoraConfigEnums.swift`. - -### 8.3 Other clients - -- **python (`meshtastic` / Meshtastic-python)** and **web** consume the published protobufs; - they will see `region_presets` once their protobuf dependency includes field 19, and can - ignore it until then (it decodes as an unknown field). - ---- - -## 9. Reference payload (current firmware table) - -For decoder unit tests. With the 2.8 region table, the firmware emits **6 groups**. Group -indices are assigned in region-table order (first region to use a profile creates its group), -so they are stable as listed here: - -| group_index | default_preset | licensed_only | presets | -| ----------------------- | -------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| 0 (standard) | `LONG_FAST` | false | LONG_FAST, LONG_SLOW, MEDIUM_SLOW, MEDIUM_FAST, SHORT_SLOW, SHORT_FAST, LONG_MODERATE, SHORT_TURBO, LONG_TURBO, MEDIUM_TURBO | -| 1 (EU 868) | `LONG_FAST` | false | _EU 86x superset_ (see below) | -| 2 (EU 866 SRD / "lite") | `LITE_FAST` | false | _EU 86x superset_ (see below) | -| 3 (EU 868 narrow) | `NARROW_SLOW` | false | _EU 86x superset_ (see below) | -| 4 (ham 20 kHz) | `TINY_FAST` | **true** | TINY_FAST, TINY_SLOW | -| 5 (ham 100 kHz) | `NARROW_SLOW` | **true** | NARROW_FAST, NARROW_SLOW | - -The **EU 86x superset** advertised by groups 1, 2 and 3 is the union of the trio's own -band presets, because the firmware auto-swaps region within the trio on preset selection -(§5), so any of these is a legal pick from any of the three regions: - -```text -LONG_FAST, LONG_SLOW, MEDIUM_SLOW, MEDIUM_FAST, SHORT_SLOW, SHORT_FAST, LONG_MODERATE, LITE_FAST, LITE_SLOW, NARROW_FAST, NARROW_SLOW -``` - -The three groups still differ by `default_preset` (`LONG_FAST` / `LITE_FAST` / `NARROW_SLOW`), -which is why they remain distinct groups despite sharing this preset list. - -`region_groups` (region → group_index): - -| group | regions | -| ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 0 | US, EU_433, CN, JP, ANZ, ANZ_433, RU, KR, TW, IN, NZ_865, TH, UA_433, MY_433, MY_919, SG_923, PH_433, PH_868, PH_915, KZ_433, KZ_863, NP_865, BR_902, LORA_24 | -| 1 | EU_868 | -| 2 | EU_866 | -| 3 | EU_N_868 | -| 4 | ITU1_2M, ITU2_2M, ITU3_2M | -| 5 | ITU2_125CM | - -> Note that several groups can carry overlapping preset lists but remain distinct: groups 1, -> 2 and 3 share the EU 86x superset yet differ in `default_preset`, and group **5** (ham -> 100 kHz) shares the `NARROW_*` presets with group 3 but differs in `licensed_only`. -> Decoders must key on the group, not on the preset list, to preserve `default_preset` and -> the licensing flag. -> -> Regions **absent** from the table (no constraint info; see §5.1): `EU_874`, `EU_917`, -> `ITU1_70CM`, `ITU2_70CM`, `ITU3_70CM`. - -This table is generated from the firmware's region table at runtime; treat the firmware as -authoritative and these values as the expected snapshot for the 2.8 table. diff --git a/docs/mesh_beacon_module.md b/docs/mesh_beacon_module.md deleted file mode 100644 index 67a391fe0e1..00000000000 --- a/docs/mesh_beacon_module.md +++ /dev/null @@ -1,454 +0,0 @@ -# Mesh Beacon Module - Function, Settings, and Client Interface Spec - -Status: draft, tracks firmware branch `feat/mesh-beacon`. -Audience: firmware reviewers (Part 1) and client-app developers - Android / Apple / Web / Python (Part 2). - -The Mesh Beacon module lets a node periodically **advertise the existence of a mesh** to -nodes that are not yet on it - broadcasting a short human-readable message plus an optional -"join offer" (a channel, region, and modem preset). It is the mechanism behind invitations -like _"Join us on NarrowSlow"_: a node sitting on one preset/region can shout an invitation -that listeners on other presets/regions can hear and surface to their user. - -The module is deliberately **advisory**. The firmware never auto-joins an advertised -channel or auto-switches preset/region in response to a received beacon - it delivers the -information to the client app and stops there. All "should I act on this?" decisions belong -to the client and, ultimately, the user. - ---- - -## Part 1 - Function and settings choices - -### 1.1 Two roles in one module - -| Role | Class | Active when | What it does | -| --------------- | --------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| **Broadcaster** | `MeshBeaconBroadcastModule` | `FLAG_BROADCAST_ENABLED` set | Periodically transmits `MESH_BEACON_APP` packets on the configured radio settings. | -| **Listener** | `MeshBeaconListenerModule` | `FLAG_LISTEN_ENABLED` set | Receives `MESH_BEACON_APP` packets and caches the offer for the client (the packet itself flows to the client unchanged). | - -The boolean toggles live in a single `flags` bitfield (see [§1.8](#18-settings-reference-moduleconfigmeshbeaconconfig-tag-17)) - broadcasting and -listening can be enabled independently on the same node. The whole module compiles out under the -`MESHTASTIC_EXCLUDE_BEACON` build flag. - -### 1.2 Wire message - -Beacons travel on a dedicated port number: - -```protobuf -MESH_BEACON_APP = 37 // meshtastic/portnums.proto -ENCODING: protobuf (meshtastic.MeshBeacon) -``` - -```protobuf -message MeshBeacon { - string message = 1; // human-readable text, max 100 bytes (buffer 101) - ChannelSettings offer_channel = 2; // optional advertised channel (name + PSK + slot) - Config.LoRaConfig.RegionCode offer_region = 3; // optional advertised region (UNSET = none) - optional Config.LoRaConfig.ModemPreset offer_preset = 4; // optional advertised preset -} -``` - -`.options` size caps (enforced at generation and on send): -`message ≤ 100`, `offer_channel.name ≤ 12`, `offer_channel.psk ≤ 32`. - -The three `offer_*` fields together describe _"there is a reachable mesh on this -region+preset, here is the channel to use."_ Any subset may be present; an empty message with -a populated offer (or vice-versa) is valid. - -### 1.3 Transmission behaviour - -Every outgoing beacon packet is stamped uniformly (`sendBeacon` → `stampPacket`): - -- `to = NODENUM_BROADCAST` -- `from = local node` (see [§1.6](#16-broadcast_send_as_node-currently-disabled) for the disabled spoof path) -- **`hop_limit = 0`** - beacons are **zero-hop**. They are never rebroadcast by the mesh; only - direct RF neighbours hear them. This is the primary spam-control mechanism. (`hop_start` is - normally `0` too, but `FLAG_LEGACY_SPLIT` raises it to `1` for old-firmware compatibility - see - [§1.5](#15-legacy-split-flag_legacy_split).) -- `priority = BACKGROUND`, `want_ack = false`. - -Broadcasting is additionally gated at runtime by: - -- airtime utilisation (`isTxAllowedAirUtil()`), and -- device role - **`CLIENT_HIDDEN` never broadcasts**. - -#### Interval - -`broadcast_interval_secs` controls cadence. The floor is **3600 s (1 hour)** -(`default_mesh_beacon_min_broadcast_interval_secs`); `0` means "use default". Values below the -floor are silently raised, both at config-set time (AdminModule) and at runtime. - -The cadence is **reboot-safe**. Each broadcast's time is persisted to flash via `TransmitHistory` -(keyed by `MESH_BEACON_APP`), and the broadcaster reads it back on boot - so a node that reboots -(or crash-loops) won't re-broadcast until a full interval has elapsed since its last real send, -rather than firing ~30 s after every boot. The timestamp is written **before** the transmit, so a -brown-out during the high-current LoRa TX still counts as "sent." This mirrors `NodeInfoModule` / -`PositionModule`. - -#### Radio switching for TX - -A beacon's whole point is often to reach a mesh on a _different_ preset/region/channel than the -broadcaster currently runs. Before transmitting a beacon tagged with target radio settings, the -module temporarily reconfigures the radio (`reconfigureForBeaconTX`), sends, then restores the -prior config. Per-packet target settings are held in an 8-entry **sidecar table** keyed by packet -ID - chosen so the `MeshPacket` proto carries no extra per-packet radio fields, and normal -(non-beacon) traffic is never touched. - -Two safety guards run before any radio switch (`beaconTxConfigInvalid`): - -1. **An unlicensed node never keys up on a licensed-only (ham) region.** (The reverse - a licensed - node operating in a non-ham region - is allowed. The switch only touches preset/region/channel, - never `owner.is_licensed`.) -2. **The preset must be valid for the target region** (`validateConfigLora`). - - If either fails, the radio is **not** switched and the radio driver **drops** the packet rather - than letting it fall through onto the current config. - -#### Channel encryption on an override channel - -Encryption keys off the **primary** channel slot, and the radio-thread channel switch happens -_after_ encryption. So when a beacon goes out on an override channel (different name/PSK), the -module installs the beacon channel into the primary slot for the synchronous duration of -`send()`, then restores it (`sendBeaconPacket`). This guarantees the packet is encrypted with the -beacon channel's key and stamped with its hash - not the primary's. Meshtastic threading is -cooperative, so there is no preemption between swap and restore. - -### 1.4 Where beacons are sent: single-target and multi-target - -The broadcaster can send to one set of radio settings or to several. **Single- and multi-target -are equal options - neither is preferred and neither is legacy.** Pick whichever matches the -deployment. - -- **Single-target:** the scalar `broadcast_on_preset` / `broadcast_on_region` / - `broadcast_on_channel` fields describe one destination. Used when `broadcast_targets` is empty. -- **Multi-target:** `broadcast_targets` (repeated `BroadcastTarget`) describes several. When - non-empty it takes over from the scalar `broadcast_on_*` fields, and the broadcaster sends **one - beacon copy per entry**. Each `BroadcastTarget` is `{ optional preset, region, optional channel_index }`, - where `channel_index` references a slot in the node's own channel table (the channel must already be - configured locally - its key is needed to encrypt the beacon). Within one cycle, targets that - resolve to the **same** effective preset/region/channel are de-duplicated - only the first is - transmitted - so an accidentally repeated entry costs no extra airtime. - -#### Same-settings vs. other-settings - -Independent of single/multi, each destination can either reuse the node's **own current radio -settings** or specify **different** ones: - -- **Same-settings ("message of the day"):** leave the preset / region / channel unset. They fall - back to the running config, so the beacon goes out on the node's current mesh with **no radio - switch** - a plain periodic broadcast to whoever is already on this preset/region. -- **Other-settings (cross-mesh invite):** set a preset / region / channel that differs from the - running config. The radio is temporarily switched for that copy's TX, then restored (see - [§1.3](#radio-switching-for-tx)). - -Both modes support both styles: a single-target beacon with no `broadcast_on_*` overrides is a -message-of-the-day on the current mesh; a multi-target list can mix one entry on the current -settings with others on different presets/regions. - -### 1.5 Legacy split (`FLAG_LEGACY_SPLIT`) - -This one flag controls **two** independent legacy-compatibility behaviours. Both are about making -beacons usable by firmware that predates this module. - -**(a) Text/offer packet split.** A combined `MESH_BEACON_APP` packet carries both the text and the -offer, but old firmware only decodes `TEXT_MESSAGE_APP` and would never show the text. When -`FLAG_LEGACY_SPLIT` is set **and both text and offer content are present**, the broadcaster -emits **two** packets on the same beacon radio settings instead of one: - -- **Packet A** - `MESH_BEACON_APP` carrying the **offer only** (no text). -- **Packet B** - `TEXT_MESSAGE_APP` carrying the **text only**. - -This is an independent two-packet decision, not an either/or: offer-only and text-only payloads -still go out as a single packet in their respective cases; only the both-present case splits. - -**(b) `hop_start = 1` override.** When `FLAG_LEGACY_SPLIT` is set, **every** beacon packet it sends -(combined, split-A, or split-B; even same-settings ones) is stamped with `hop_start = 1` while -`hop_limit` stays `0`. Pre-2.7.20 firmware drops `hop_start == 0` packets in a pre-decryption check -before it can read the bitfield, so `hop_start = 1` lets those nodes accept the beacon - and it -remains genuinely zero-hop (`hop_limit = 0` still prevents any rebroadcast). - -> **Side effect for clients:** with `hop_start = 1, hop_limit = 0`, receivers compute -> `hops_away = hop_start − hop_limit = 1`, so a legacy-split beacon reads as **1 hop away** even -> though it arrived over direct RF. Without legacy-split it reads as direct (0). Don't treat a -> beacon's `hops_away` as a reliable distance signal. - -### 1.6 `broadcast_send_as_node` (currently disabled) - -The schema reserves `broadcast_send_as_node` (field 3) to send beacons _as_ another node ID. **The -firmware application of this field is currently commented out pending review**, so beacons always -go out as the local node today. The access-control rule is, however, already enforced in -AdminModule and should be treated as canonical: - -> A remote admin may only set `broadcast_send_as_node` to **their own** node ID -> (`mp.from`). Any other value is rejected and reset to the stored value. - -Design note for when it is re-enabled: it is a _node-ID_ spoof only - it rewrites `from` but forges -no signature. Once `from` is not us, the packet is no longer `isFromUs()`, so the router skips -XEdDSA signing and receivers get an unsigned packet attributed to another node. - -### 1.7 Reception behaviour (listener) - -When `FLAG_LISTEN_ENABLED` is **off**, the router drops incoming `MESH_BEACON_APP` packets up front -(`Router::handleReceived`, same pattern as a disabled NeighborInfo module) - so they reach neither -the modules nor the phone. When it is **on**, the packet flows normally and the listener's -`wantPacket` accepts it (`has_mesh_beacon` + `FLAG_LISTEN_ENABLED` + `portnum == MESH_BEACON_APP`). -On a valid beacon (`handleReceivedProtobuf`): - -1. **Offer → cache.** Any offer (`offer_channel` / `offer_region` / `offer_preset`) is stored in - the static `lastReceivedOffer` (sender, channel, region, preset, `received_at`). `received_at` - is `0` if the node has no RTC fix yet - **consumers must not treat `0` as a valid timestamp.** -2. **Never auto-applied.** The firmware does not switch channel/preset/region from a received - offer. Acting on it is the client app's job. -3. The handler returns `CONTINUE` (not `STOP`), so the original `MESH_BEACON_APP` packet **flows to - the client unchanged** through the normal FromRadio path (see Part 2). The client reads the - `message` field directly from that packet - there is no separate copy. - -The firmware deliberately does **not** unwrap a combined beacon's text into a synthesized -`TEXT_MESSAGE_APP`, and does **not** fire `EVENT_RECEIVED_MSG`: a beacon is an advisory broadcast, -not a personal message, so it must not duplicate the text or wake the device from sleep. If a -broadcaster needs non-beacon-aware clients to see the text, it uses `FLAG_LEGACY_SPLIT`, which sends -a real `TEXT_MESSAGE_APP` over RF (see [§1.5](#15-legacy-split-flag_legacy_split)). - -### 1.8 Settings reference (`ModuleConfig.MeshBeaconConfig`, tag 17) - -| # | Field | Type | Meaning / constraints | -| --- | ------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------ | -| 1 | `flags` | uint32 (bitfield) | Bitwise-OR of `Flags` values (listen / broadcast / legacy-split toggles). See enum below. | -| 3 | `broadcast_send_as_node` | uint32 | Send-as node ID. **Application disabled in firmware.** Remote admin may only set to own node ID. | -| 4 | `broadcast_message` | string | Text in each broadcast. **Hard-capped at 100 bytes.** | -| 5 | `broadcast_offer_channel` | ChannelSettings | Channel advertised in `offer_channel`. | -| 6 | `broadcast_offer_region` | RegionCode | Region advertised in `offer_region`. Must be a known region or it is cleared. | -| 7 | `broadcast_offer_preset` | optional ModemPreset | Preset advertised in `offer_preset`. Validated against offer region (else cleared). | -| 8 | `broadcast_on_channel` | ChannelSettings | Channel to transmit on (single-target). Empty name → preset display name. | -| 9 | `broadcast_on_region` | RegionCode | Region to transmit on (single-target). | -| 10 | `broadcast_on_preset` | optional ModemPreset | Preset to transmit on (single-target). Validated against on-region (else this + `on_channel` cleared). | -| 11 | `broadcast_interval_secs` | uint32 | Cadence. **Min 3600**, default 3600; `0` = default. | -| 13 | `broadcast_targets` | repeated BroadcastTarget | Multi-target list; when non-empty overrides the single-target `broadcast_on_*` fields. | - -> The three boolean toggles were folded into the `flags` bitfield; field tags 2 and 12 are now -> unused (the branch is unreleased, so the old tags are left as gaps rather than reserved). - -**`Flags` enum** (nested in `MeshBeaconConfig`; OR the values into `flags`): - -| Bit value | Name | Meaning | -| --------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 0 | `FLAG_NONE` | No options enabled. | -| 1 | `FLAG_LISTEN_ENABLED` | Receive beacons; cache the offer. The packet flows to the client, which reads `message` directly. | -| 2 | `FLAG_BROADCAST_ENABLED` | Periodically broadcast beacons from this node. | -| 4 | `FLAG_LEGACY_SPLIT` | Legacy compatibility: (a) split text+offer into separate `TEXT_MESSAGE_APP` + `MESH_BEACON_APP` packets, and (b) stamp `hop_start = 1` on every beacon so pre-2.7.20 firmware accepts it (see [§1.5](#15-legacy-split-flag_legacy_split)). | - -`BroadcastTarget`: `1 preset` (optional, falls back to running config), `2 region` (`UNSET` = running config), `4 channel_index` (optional `uint32`, index into the node's channel table; if unset, the default channel for the preset is used). Tag `3` is an unused gap - it previously held an embedded `ChannelSettings`, dropped to keep `ModuleConfig` within the BLE `FromRadio` size budget. - ---- - -## Part 2 - Client interface specification - -This section is what a client app needs to integrate with the beacon module. Everything goes -through the **standard admin / ToRadio / FromRadio protocol** - there is no bespoke transport. - -### 2.1 Capability detection - -The module is build-flag optional. Treat it as present when the node's `LocalModuleConfig` -contains a `mesh_beacon` sub-message (`LocalModuleConfig.mesh_beacon`, tag 18). If absent, the -firmware was built with `MESHTASTIC_EXCLUDE_BEACON` - hide the beacon UI. - -### 2.2 Reading and writing configuration - -Standard module-config flow - no new admin messages: - -- **Read:** `AdminMessage.get_module_config_request = ModuleConfig.MeshBeaconConfig` (variant 17). - Reply is `get_module_config_response` with the `mesh_beacon` payload. -- **Write:** `AdminMessage.set_module_config { mesh_beacon = … }`. - -The on/off toggles (listen, broadcast, legacy-split) are bits in the `flags` field, not separate -booleans - read/write them with the `MeshBeaconConfig.Flags` values -(`FLAG_LISTEN_ENABLED = 1`, `FLAG_BROADCAST_ENABLED = 2`, `FLAG_LEGACY_SPLIT = 4`). To toggle one -bit, read the current `flags`, set/clear the bit, and write the whole config back. - -The firmware **sanitises on write** - your value may be silently adjusted. Mirror these rules -client-side so the UI doesn't disagree with the device: - -| Rule | Firmware behaviour | -| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -| `broadcast_message` length | Truncated to 100 bytes. | -| `broadcast_interval_secs` | If non-zero and `< 3600`, raised to 3600. | -| `broadcast_on_preset` invalid for `broadcast_on_region` (or current region) | Cleared, **and `broadcast_on_channel` cleared too.** | -| `broadcast_offer_preset` invalid for offer/current region | Cleared. | -| `broadcast_offer_region` not a known region | Cleared to `UNSET`. | -| `broadcast_targets[i].region` not a known region | That entry's region cleared to `UNSET` (TX falls back to running config). | -| `broadcast_targets[i].preset` invalid for that entry's region | That entry's `preset` and `channel_index` cleared. | -| `broadcast_targets[i].channel_index` ≥ `MAX_NUM_CHANNELS` (8) | That entry's `channel_index` cleared (existence is **not** checked - see §2.5). | -| `broadcast_send_as_node` ≠ sender's node ID (remote admin) | Rejected, reset to stored value. | - -Setting beacon config does **not** trigger a reboot (`shouldReboot = false`); changes take effect -on the next broadcast cycle. After a successful write, **re-read** the config to display the -effective (sanitised) values. - -### 2.3 Receiving beacons - -A received beacon reaches the client as a normal `FromRadio.packet` (`MeshPacket`) - the listener -returns `CONTINUE`, so the packet is **not** consumed on-device. The client must: - -1. Subscribe to the FromRadio packet stream as usual. -2. For packets with `decoded.portnum == MESH_BEACON_APP (37)`, decode `decoded.payload` as a - `meshtastic.MeshBeacon`. -3. Read `message`, `offer_channel`, `offer_region`, `offer_preset` (presence-checked). -4. `packet.from` is the **originating beaconer** (the firmware preserves it). - -> **Requires `FLAG_LISTEN_ENABLED` set in `flags`.** With listening disabled the firmware drops -> received `MESH_BEACON_APP` packets in the router - before they reach the phone or any on-device -> handler - the same way it drops a disabled module's packets (e.g. NeighborInfo). The node still -> physically receives the RF, but the client will not see beacons over the FromRadio stream until -> listening is enabled. - -#### Reading the text - no duplication - -For a beacon-aware client the text is **simply the `message` field of the `MESH_BEACON_APP` -packet** you already decode for the offer (step 3 above). One packet, one field - the firmware does -**not** inject a separate `TEXT_MESSAGE_APP` copy, so there is nothing to deduplicate. - -The only time a beacon's text arrives as a separate `TEXT_MESSAGE_APP` is when the broadcaster set -`FLAG_LEGACY_SPLIT`: in that mode the `MESH_BEACON_APP` carries the **offer only** (empty `message`) -and the text is sent as a normal `TEXT_MESSAGE_APP` over RF, so legacy/non-beacon-aware clients can -display it. These two cases are mutually exclusive - a given beacon's text appears exactly once, -either in `MESH_BEACON_APP.message` (combined) or as a `TEXT_MESSAGE_APP` (legacy-split) - so a -client never needs to dedup. Render whichever it receives. - -### 2.4 Acting on an offer (the core client responsibility) - -When a `MESH_BEACON_APP` carries offer content, present it to the user as an **invitation** - -e.g. _"Node ⟨from⟩ invites you to join '⟨offer_channel.name⟩' on ⟨preset⟩/⟨region⟩."_ Then, only on -explicit user confirmation, apply it by writing normal config: - -- `offer_channel` → add/replace a `Channel` (`set_channel`), typically as a secondary channel. -- `offer_region` / `offer_preset` → `set_config { lora = … }` (`use_preset = true`, set - `modem_preset` and `region`). **Note this changes the node's own radio and will drop it off its - current mesh** - make that consequence explicit in the UI. - -**The firmware will never do any of this for the user. No silent auto-apply.** The on-device -`lastReceivedOffer` cache is a firmware-internal convenience and is **not** currently exposed via -an admin message - clients should source offers from the live `MESH_BEACON_APP` packet stream -(§2.3), not expect a "get last offer" RPC. - -#### Offer trust model - read before applying - -- **The advertised PSK is not a secret.** `offer_channel.psk` is a public join token sent in the - clear inside a broadcast; it is a convenience, not a security boundary. An operator who wants a - genuinely private channel must distribute the PSK out-of-band and leave `offer_channel` unset. - Surface offered channels as **public/open** to the user. -- **Validate before applying.** Reject or warn if `offer_preset` is not valid for `offer_region`, - and **never** apply a licensed-only (ham) region for a user who is not a licensed operator - - mirror the firmware's own guard. -- Beacons are **unsigned** when sent as another node (the disabled send-as path), and even normal - beacons assert nothing about the sender's authority. Treat `from` as informational. - -### 2.5 Configuring this node as a broadcaster - -To make a node advertise a mesh, write `MeshBeaconConfig` with `FLAG_BROADCAST_ENABLED` set in -`flags` and at least one of: a non-empty `broadcast_message`, or offer content -(`broadcast_offer_*`). With neither, the broadcaster has nothing to send and stays silent. - -Typical multi-region invite beacon: - -```text -flags = FLAG_BROADCAST_ENABLED | FLAG_LEGACY_SPLIT // broadcast on; split so legacy nodes still see the text -broadcast_message = "Join us on NarrowSlow!" -broadcast_offer_preset = NARROW_SLOW -broadcast_offer_region = EU_N_868 -broadcast_offer_channel = { name: "MyChannel", psk: <32-byte key> } -broadcast_interval_secs = 3600 -// channel_index points at slots in THIS node's channel table - configure those channels first. -broadcast_targets = [ - { preset: LONG_FAST, region: EU_868, channel_index: 0 }, - { preset: NARROW_SLOW, region: EU_N_868, channel_index: 1 }, -] -``` - -The same fields can be baked in at build time via `userPrefs.jsonc` -(`USERPREFS_MESH_BEACON_*`) - see that file for the full list, including -`USERPREFS_MESH_BEACON_TARGET__*` for multi-target entries. - -#### Single-target vs. multi-target - equal options, different channel representation - -Single-target and multi-target are **equal, first-class options**. Neither is preferred, -deprecated, or a "legacy" fallback - pick whichever matches the deployment (a single-target -beacon with no overrides is a plain message-of-the-day; a multi-target list reaches several -preset/region/channel combinations). The broadcaster uses `broadcast_targets` when it is -non-empty and the scalar `broadcast_on_*` fields when it is empty. - -The one **subtle implementation difference** is how each names its TX channel: - -| Path | TX channel is specified by | Channel name/PSK live… | -| ------------- | ------------------------------------------------------- | ----------------------------------------- | -| Single-target | `broadcast_on_channel` - an embedded `ChannelSettings` | …inline in the beacon config | -| Multi-target | `broadcast_targets[i].channel_index` - a `uint32` index | …in the node's channel table (referenced) | - -This asymmetry is deliberate: embedding a full `ChannelSettings` in every one of the (up to -four) targets would push `ModuleConfig` past the BLE `FromRadio` size limit, so a target -references an already-configured channel-table slot instead. `broadcast_offer_channel` (the -advertised join token) is **always** inline regardless of path - it is the advertisement payload -and must carry the actual name/PSK. - -#### Configuring a multi-target broadcaster (two-step) - -Because a target's channel is a reference, configuring a multi-target broadcaster takes **two -admin writes**, in order: - -1. **Create/define each channel in the node's channel table** with the normal channel admin flow - (the same `set_channel` your app already uses for adding channels): - - ```text - AdminMessage.set_channel { index: 1, role: SECONDARY, - settings: { name: "NarrowSlow", psk: , channel_num: 0 } } - ``` - -2. **Write the beacon config**, pointing each target at the slot index from step 1: - - ```text - AdminMessage.set_module_config { mesh_beacon: { - flags = FLAG_BROADCAST_ENABLED - broadcast_targets = [ { preset: NARROW_SLOW, region: EU_N_868, channel_index: 1 } ] - } } - ``` - -Notes: - -- A target may **only** reference a channel that already exists locally - the node needs that - channel's key to encrypt the beacon. A `channel_index` that is out of range, or points at a - blank/unconfigured slot, is not an error: the beacon falls back to the node's **current/primary - channel** (its name, PSK, and slot) on the target preset/region. The channel name only defaults - to the preset's display name (e.g. `LongFast`) when the primary channel itself is unnamed - so - the fallback is "broadcast on my home channel," **not** a freshly-synthesised default-PSK channel - for that preset. -- `channel_index` must be `< MAX_NUM_CHANNELS` (8); the firmware clears it on write otherwise (see - §2.2 sanitise rules). This is the **only** check on write - the firmware does **not** verify that - the referenced slot is actually populated, because you may legitimately write the beacon config - before creating the channel. **Validating that a referenced channel exists is the client app's - responsibility.** A dangling reference doesn't error; it silently falls back to the preset's - default channel - so without a client-side check, the user can believe they're advertising - channel _X_ while the node is really transmitting on the preset default. Before writing, confirm - each `channel_index` maps to a configured `Channel`, and warn the user otherwise. -- **No automatic deduplication of channels.** Neither the beacon config nor the channel table - dedups by content: two `broadcast_targets` may carry the same `channel_index`, or different - indices whose slots hold identical settings, and `set_channel` will happily store two slots with - the same name/PSK. The broadcaster _does_ skip transmitting a target whose effective - preset/region/channel duplicates an earlier one in the same cycle (so a duplicated entry wastes - no airtime), but it does not rewrite or reject your config - keeping the target list free of - redundant entries is up to the client. -- The single-target path needs no separate `set_channel` step - its `broadcast_on_channel` is - written inline in the same beacon-config message. - -### 2.6 Quick reference - -| Concern | Value | -| ---------------------- | ---------------------------------------------------------------------------------------- | -| Port number | `MESH_BEACON_APP = 37` | -| Wire message | `meshtastic.MeshBeacon` | -| Config message | `ModuleConfig.MeshBeaconConfig` (variant tag 17) | -| On/off toggles | `flags` bitfield (`MeshBeaconConfig.Flags`) | -| Local config presence | `LocalModuleConfig.mesh_beacon` (tag 18) | -| Min broadcast interval | 3600 s (1 h) | -| Message max length | 100 bytes | -| Hop behaviour | Zero-hop (`hop_limit = 0`), never rebroadcast; `hop_start = 1` under `FLAG_LEGACY_SPLIT` | -| Auto-apply offers? | **Never** - client + user decide | -| Offer PSK | Public join token, not a secret | -| Disabled today | `broadcast_send_as_node` application | diff --git a/docs/nexthop-routing-reliability.md b/docs/nexthop-routing-reliability.md deleted file mode 100644 index 42a08d0776f..00000000000 --- a/docs/nexthop-routing-reliability.md +++ /dev/null @@ -1,456 +0,0 @@ -# NextHop direct-message reliability on dense meshes - findings & plan - -**Status:** Implemented - mitigations and tests in `PR3-tmm-nexthop` -**Date:** 2026-06-13 -**Area:** `src/mesh` router stack (`NextHopRouter`, `ReliableRouter`, `FloodingRouter`, `Router`, `NodeDB`, `PacketHistory`) -**Constraint:** No over-the-air / wire-format changes - `next_hop` and `relay_node` stay 1 byte, no `PacketHeader` changes, no breaking protobuf changes. All new state is RAM-only. - -This document captures the analysis and the proposed mitigations so the work can be -continued on this branch by anyone. It is intentionally code-grounded (file:line -references throughout) and standalone - you should not need the original investigation -context to pick it up. - ---- - -## TL;DR - -NextHop routing for direct messages (DMs) is unreliable on dense meshes. The headline -cause is the **birthday problem**: `next_hop` and `relay_node` are each a single byte -(the last byte of a 32-bit node number), so on a mesh of N nodes the probability that -two share the same byte hits ~50% at **~19 nodes** and is near-certain by 50-100. But -there are **other, equally important issues**: that single byte is trusted blindly at -five different code sites, learned routes **never decay**, routes are learned from the -**reverse (ACK) path** (asymmetric-link hazard), and collision-driven spurious -rebroadcasts **amplify congestion** exactly when the mesh is busy. - -Because we can't widen the on-wire field, the fix is **interpretation-side** ("don't -trust a byte that doesn't map to a unique reachable neighbor - flood instead") plus -**recovery-side** ("decay stale/failing routes so they get re-discovered"). Four -mitigations, M1-M4, all RAM-only. The net behavioral change: on dense/mobile meshes a -DM that today silently misroutes or black-holes instead falls back to managed flooding -(which still delivers) and re-learns a fresh route quickly. Sparse-mesh happy paths are -unchanged. - ---- - -## How NextHop routing works today (mechanics) - -Inheritance chain: `Router` → `FloodingRouter` → `NextHopRouter` → `ReliableRouter`. - -**The single-byte identifiers.** Both routing bytes come from one helper: - -```cpp -// src/mesh/NodeDB.h:255 -uint8_t getLastByteOfNodeNum(NodeNum num) { return (uint8_t)((num & 0xFF) ? (num & 0xFF) : 0xFF); } -``` - -It projects a 32-bit node number onto 255 values (`0x00` is remapped to `0xFF` so it -never collides with the `0`-valued sentinels `NO_NEXT_HOP_PREFERENCE` / `NO_RELAY_NODE`, -`src/mesh/MeshTypes.h:44-46`). `next_hop` and `relay_node` in the packet header are -`uint8_t` (`src/mesh/mesh.pb.h`, comments "Last byte of the node number…"). The learned -route stored per destination, `meshtastic_NodeInfoLite::next_hop`, is also a single byte -(`src/mesh/generated/meshtastic/deviceonly.pb.h:83`). - -**Sending a DM** - `NextHopRouter::send` (`src/mesh/NextHopRouter.cpp:23`): - -1. `p->relay_node = getLastByteOfNodeNum(getNodeNum())` (mark ourselves as relayer). -2. `p->next_hop = getNextHop(p->to, p->relay_node)` (`src/mesh/NextHopRouter.cpp:192`): - look up `nodeDB->getMeshNode(to)->next_hop`; return it unless it equals the relayer - byte; otherwise `NO_NEXT_HOP_PREFERENCE` (→ flood). - -**Relaying** - `NextHopRouter::perhapsRebroadcast` (`src/mesh/NextHopRouter.cpp:133`): -rebroadcast iff `next_hop == NO_NEXT_HOP_PREFERENCE` (flood) **or** -`next_hop == getLastByteOfNodeNum(getNodeNum())` (we are the addressed next hop) -(`:147`). Each node only ever compares against **its own** byte. - -**Learning** - `NextHopRouter::sniffReceived` (`src/mesh/NextHopRouter.cpp:89`): on an -ACK/reply (`request_id`/`reply_id` set), if the relayer of the ACK was also a relayer of -the original packet (validated via `PacketHistory::checkRelayers`), set -`origTx->next_hop = p->relay_node` (`:114`). I.e. the **forward** next-hop is learned -from the **reverse** path's relayer. - -**Retransmission / fallback** - `NextHopRouter::doRetransmissions` -(`src/mesh/NextHopRouter.cpp:284`). Budgets: `NUM_RELIABLE_RETX=3` (originator: initial - -- 2 retries), `NUM_INTERMEDIATE_RETX=2` (relayer: 1 retry). On the **last** retry - (`numRetransmissions==1`) it resets `next_hop` to `NO_NEXT_HOP_PREFERENCE` on the packet - **and** clears `sentTo->next_hop` in NodeDB, then floods (`:313-321`). Retransmit timing - comes from `iface->getRetransmissionMsec`, whose contention window **grows with channel - utilization** (`src/mesh/RadioInterface.cpp` `getTxDelayMsec`/`getTxDelayMsecWeighted`). - -**Dedup / relayer history** - `PacketHistory` (`src/mesh/PacketHistory.cpp`): a bounded -ring (`PACKETHISTORY_MAX = max(MAX_NUM_NODES*2, 100)`, 20 B/record) keyed by -`(sender,id)`, tracking up to `NUM_RELAYERS=6` relayer **bytes** per packet in -`relayed_by[]`. `wasRelayer` (`:490`) and `checkRelayers` (`:517`) match bytes against -that array. - ---- - -## Root-cause analysis - -### 1. The single byte is trusted blindly at five sites (the birthday problem) - -| # | Site | File:line | Failure on collision | -| --- | -------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| 1 | Rebroadcast self-check | `NextHopRouter.cpp:147` | A remote "impostor" node sharing the intended next-hop's byte also rebroadcasts → wasted airtime / congestion. | -| 2 | Route learning | `NextHopRouter.cpp:111-114` | Stores an ambiguous byte as the route; later resolves to the wrong physical node. | -| 3 | Relayer validation | `PacketHistory.cpp:490-538` | `wasRelayer(byte)` returns true for the wrong node → mis-validated ACK / mis-learn. | -| 4 | Favorite-router hop preservation | `Router.cpp:120-145` | **First** NodeDB node whose last byte matches wins - non-deterministic; can preserve hops for the wrong relay (hop leak). | -| 5 | Send-path lookup | `NextHopRouter.cpp:192-207` | Emits a byte that may address the wrong node; no check it still maps to a reachable neighbor. | - -Collision math (uniform last byte over 255 buckets): P(collision) ≈ 50% at ~19 nodes, - -> 99% by ~75 nodes. Dense meshes are squarely in the "always colliding" regime. - -### 2. Stale routes never decay - -The learned `next_hop` byte is cleared only on the **current DM's** last retry -(`NextHopRouter.cpp:313-321`). A route learned hours ago that has since gone dead is -still trusted on the **next** DM's first attempt - which on a congested mesh is also the -slowest attempt. Result: silent black-hole at a dead hop until the retransmission budget -drains, then a late flood. Intermediate nodes hold stale routes indefinitely. - -### 3. Reverse-path (asymmetric-link) learning - -`origTx->next_hop` is learned from the ACK's relayer (`NextHopRouter.cpp:110-114`) - the -**reverse** direction. RF links are frequently asymmetric, so the best reverse relay can -be a poor forward relay. Worse, the next reverse ACK immediately re-learns the same bad -hop, so the route **flaps** back to the bad value even after a failure reset. - -### 4. Congestion amplification - -Collision-driven impostor rebroadcasts (issue 1) add airtime; the contention window -grows with channel utilization, so retransmit intervals **lengthen** exactly when the -mesh is busy. The 3-try reliable budget can then expire before delivery. On dense -meshes, efficiency _is_ reliability. - -### Note: pubkey-derived node numbers (develop / 2.8) - does not change the plan - -develop derives the node number from the public key: -`my_node_num = crc32Buffer(public_key)` (`src/mesh/NodeDB.cpp:481`), re-derived on key -change in `createNewIdentity()` (`src/mesh/NodeDB.cpp:3113`). This **reinforces** the -plan rather than changing it: - -- **Birthday problem unchanged and now textbook-exact.** CRC32 mixes well → the last - byte is uniformly distributed over 256 values. Derivation adds no wire bits. -- **Node numbers are now immutable / identity-bound.** Pre-2.8 `pickNewNodeNum()` could - renumber a node to dodge a conflict; now the number is fixed by the key, so a last-byte - collision **cannot be resolved operationally by renumbering** → M1/M2/M3 become _more_ - necessary. -- **Resolver gets cleaner inputs.** Stable node numbers keep a learned byte bound to one - identity (good for M3 freshness). `createNewIdentity()` retires the old entry by marking - it **ignored** and clearing its pubkey (`src/mesh/NodeDB.cpp:3123-3125`), which M1's - candidate gate already skips - so key rotation can't pollute resolution. -- **No wire-free disambiguation unlocked.** A receiver still gets only 1 byte and cannot - recover which full node number a colliding value meant - so "detect ambiguity → flood" - remains the correct strategy. - ---- - -## Proposed mitigations - -Key insight for all of M1/M2: **a 1-byte ID only needs to be unique among a node's -direct neighbors / plausible relays, not the whole mesh.** That candidate set is small -(typically 5-15), so a byte usually resolves unambiguously there; when it doesn't, fall -back to the _safe_ behavior (flood / decrement / don't-learn). - -### M1 - Ambiguity-aware last-byte resolution (new NodeDB primitive) - -New types + methods in `src/mesh/NodeDB.h` (near line 255) / `src/mesh/NodeDB.cpp` -(near `getMeshNode`, ~2936): - -```cpp -enum class LastByteResolution : uint8_t { None, Unique, Ambiguous }; -struct ResolvedNode { LastByteResolution status = LastByteResolution::None; NodeNum num = 0; }; - -// Resolve a single on-wire last-byte to a unique full NodeNum among relevant candidates. -ResolvedNode resolveLastByte(uint8_t lastByte, bool requireDirectNeighbor); -// Convenience: true iff exactly one relevant candidate (Ambiguous and None both -> false = SAFE). -bool resolveUniqueLastByte(uint8_t lastByte, bool requireDirectNeighbor, NodeNum *outNum = nullptr); -``` - -- **One linear pass** over `meshNodes`, reusing `getNumMeshNodes()`/`getMeshNodeByIndex()`, - the bitfield helpers (`nodeInfoLiteIsFavorite/HasUser/IsIgnored`), `sinceLastSeen()`, - and `getLastByteOfNodeNum()`. **Early-exit** on the 2nd match (return `Ambiguous`). -- **Guard:** `if (lastByte == 0) return {None, 0};` (covers `NO_RELAY_NODE` / MQTT-invalid). -- **Candidate gate** (skip): `num == getNodeNum()` (never resolve to ourselves), `num == 0`, - `num == NODENUM_BROADCAST`, `nodeInfoLiteIsIgnored`. Then match - `getLastByteOfNodeNum(node->num) == lastByte` (cheapest test last, mirroring `Router.cpp:119`). -- **Relevance gate:** - - `requireDirectNeighbor == true` (strict, for SEND): `has_hops_away && hops_away == 0` - **and** `sinceLastSeen(node) < NEXTHOP_NEIGHBOR_FRESH_SECS`. - - `requireDirectNeighbor == false` (lenient, for learn / hop-preserve): accept if direct - neighbor **or** `nodeInfoLiteIsFavorite` **or** role ∈ {ROUTER, ROUTER_LATE, CLIENT_BASE}. -- **No tie-break.** A collision must return `Ambiguous` - picking "best SNR" would - resurrect the silent-misroute bug. (Deliberate non-goal; document in code.) - -New constant in `src/mesh/MeshTypes.h` (near line 44): -`#define NEXTHOP_NEIGHBOR_FRESH_SECS (60 * 60 * 2)` (mirrors `NUM_ONLINE_SECS`). - -### M2 - Only route on bytes that resolve to a unique, reachable neighbor - -In `getNextHop` (`src/mesh/NextHopRouter.cpp:192-207`), after the existing split-horizon -check (`node->next_hop != relay_node`), require the stored byte to resolve to a **unique, -currently-fresh direct neighbor**; else flood: - -```cpp -if (node->next_hop != relay_node) { - ResolvedNode r = nodeDB->resolveLastByte(node->next_hop, /*requireDirectNeighbor=*/true); - if (r.status == LastByteResolution::Unique) return node->next_hop; - LOG_WARN("Next hop 0x%x for 0x%x %s -> flood", node->next_hop, to, - r.status == LastByteResolution::Ambiguous ? "ambiguous among neighbors" : "no longer a neighbor"); - return std::nullopt; -} -``` - -This self-heals when a neighbor goes away (unicast-into-a-void becomes a flood). It -applies to originating, relaying, and retrying, since all route through `getNextHop`. - -Apply M1's safe fallback at the other sites: - -- **Learning** (`NextHopRouter.cpp:111-114`): gate `origTx->next_hop = p->relay_node` on - `resolveUniqueLastByte(p->relay_node, /*direct=*/false)`. Ambiguous/unknown → don't - learn (leave route unset → flood). -- **Favorite-router preservation** (`Router.cpp:120-145`): replace the "first match wins" - loop with `resolveUniqueLastByte(p->relay_node, /*direct=*/false)` + a re-check that the - resolved node is favorite/has_user/router. Ambiguous/none/not-favorite → **decrement** - (safe). Net: removes one full DB scan, adds one resolver scan (wash). - -**Left unchanged, by design (document why in code):** - -- **Site 1** rebroadcast self-check (`NextHopRouter.cpp:147`) and self-identity checks - (`ReliableRouter.cpp:127`): a node matches its **own** byte - no DB resolution helps. A - remote impostor sharing the intended next-hop's byte will still rebroadcast. M1/M2 - shrink the blast radius by reducing how often an ambiguous byte is ever stored or - originated; a true fix needs a wider field (out of scope). **This is the one residual - the plan cannot fully close.** -- **Site 3** `wasRelayer`/`checkRelayers` (`PacketHistory.cpp:490-538`): intentionally - byte-domain (both sides are on-wire bytes); the consumer (learning) is now hardened. - Add a one-line comment; do not change. - -### M3 - Route freshness / failure memory (RAM table on NextHopRouter) - -A bounded, LRU-evicted table keyed by destination, mirroring `PacketHistory`'s -reuse-oldest discipline (not an unbounded map) to cap RAM. - -`src/mesh/NextHopRouter.h` (near `pending`, line 99): - -```cpp -struct RouteHealth { - NodeNum dest = 0; // 0 == empty slot - uint32_t learnedAtMsec = 0; // millis() at last (re)learn; rollover-aware - uint8_t consecutiveFailures = 0; - uint8_t lastNextHop = NO_NEXT_HOP_PREFERENCE; // byte this health refers to -}; -static constexpr uint8_t ROUTE_HEALTH_MAX = 32; // ~384B; drop to 16 if RAM-tight -RouteHealth routeHealth[ROUTE_HEALTH_MAX] = {}; -// Helpers take `now` (pure/testable): findRouteHealth, getOrAllocRouteHealth, -// noteRouteLearned, noteRouteSuccess, noteRouteFailure, isRouteStale, clearRouteHealth -``` - -Policy: - -| Constant | Value | Rationale | -| ------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `ROUTE_TTL_MSEC` | 30 min | Survives a normal conversation; re-discovers a moved node within a telemetry interval. | -| `ROUTE_FAILURE_THRESHOLD` | 3 | 1-2 consecutive failures are transient LoRa collisions; 3 to the same hop = dead. Accumulates **across** DMs (independent of the per-DM 3-try budget). | - -`isRouteStale(h, now)` = `(now - h.learnedAtMsec) >= ROUTE_TTL_MSEC || h.consecutiveFailures >= ROUTE_FAILURE_THRESHOLD`. -All age math uses **unsigned subtraction** (rollover-safe, matching -`PacketHistory.cpp:364`); treat `learnedAtMsec == 0` as "set now". - -Wiring (as built - `src/mesh/NextHopRouter.cpp`, `src/mesh/ReliableRouter.cpp`): - -- `getNextHop`: if a health record matches the stored byte and `isRouteStale`, clear - `node->next_hop` (NodeDB) **and** `clearRouteHealth`, return `nullopt` (flood). No - record yet (cold path, first DM after boot) → trust NodeDB, but the M2 strict-neighbor - gate still applies. -- `sniffReceived` learn: gate the write through `resolveUniqueLastByte` (M2), then - `noteRouteLearned(p->from, p->relay_node, millis())` - resets `consecutiveFailures` - **only if the hop changed** (anti-flap for asymmetric re-learn); otherwise just refreshes - `learnedAtMsec`. (No success signal is taken on the intermediate reverse-pass: an ACK - merely passing through us is not proof that _we_ delivered, and resetting failures there - would reintroduce the asymmetric flap.) -- `doRetransmissions`: on the last-retransmission branch (`numRetransmissions == 1`, the - point a directed delivery has gone un-ACKed for both originator and intermediate) → - `noteRouteFailure(to)`, then the existing NodeDB `next_hop` reset + flood. We deliberately - do **not** `clearRouteHealth` here: keeping the record is what lets the failure count - accumulate across DMs so a flapping reverse-path-relearned dead hop eventually ages out. -- `ReliableRouter::sniffReceived` ACK path → `noteRouteSuccess(getFrom(p), millis())` - (an end-to-end ACK addressed to us is genuine forward-delivery proof; clears failures and - refreshes freshness). `noteRouteSuccess`/`noteRouteFailure` are no-ops when no record - exists, so flood-only destinations never pollute the table. - -**Reconciliation (no double-handling):** `doRetransmissions` owns _in-flight_ failure of -the current DM (reset NodeDB `next_hop` + flood, and bump the cross-DM failure counter); -`getNextHop` owns _between-DM_ staleness (TTL or failure-threshold → flood + clear). The -only place that erases a health record is the `getNextHop` decay path; the retransmission -path leaves it intact so the counter survives a reverse-path re-learn. - -### M4 - Earlier flood for unverified routes (gated, off by default) - -Compile-gated so healthy sparse meshes are untouched. **Default is off** - the define -lives in `NextHopRouter.h` and must be flipped to measure: -`#define NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED 1`. - -In `doRetransmissions`, the directed-retry `else` branch: if the route is **not verified** -(`!findRouteHealth(to) || consecutiveFailures > 0 || isRouteStale`), reset `next_hop` and -flood on this attempt instead of spending another directed try. A **verified** route -(record present, `consecutiveFailures == 0`, within TTL - i.e. recently ACKed) takes the -unchanged directed-retry path, so the sparse-mesh happy path is untouched. Trade-off: -airtime ↔ latency; the gate ensures we never pay the flood cost on a proven route, only on -one we already distrust. Off by default precisely so it can be A/B-measured on the -simulator before broad enable. - ---- - -## Files to modify - -| File | Change | -| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -| `src/mesh/MeshTypes.h` | `NEXTHOP_NEIGHBOR_FRESH_SECS`, `ROUTE_TTL_MSEC`, `ROUTE_FAILURE_THRESHOLD`, `NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED` | -| `src/mesh/NodeDB.h` / `src/mesh/NodeDB.cpp` | `LastByteResolution`, `ResolvedNode`, `resolveLastByte`, `resolveUniqueLastByte` | -| `src/mesh/NextHopRouter.h` | `RouteHealth` + array + helpers; `#ifdef PIO_UNIT_TESTING public:` for helpers and `getNextHop` | -| `src/mesh/NextHopRouter.cpp` | `getNextHop` (M2 gate + M3 decay); `sniffReceived` (learn gate + health seed + success); `doRetransmissions` (failure counting + M4); comment site 1 | -| `src/mesh/Router.cpp` | `shouldDecrementHopLimit` → resolver + favorite/router re-check | -| `src/mesh/ReliableRouter.cpp` | ACK path → `noteRouteSuccess` | -| `test/test_nexthop_routing/test_main.cpp` | **new** unit suite (auto-built under `[env:native]`) | - -**Reuse, don't reinvent:** `getLastByteOfNodeNum`, `sinceLastSeen`, the bitfield helpers, -`getMeshNodeByIndex`/`getNumMeshNodes`, PacketHistory's reuse-oldest eviction shape, and -`MockNodeDB::addTestNode` (from `test/test_hop_scaling/test_main.cpp`). - ---- - -## Edge cases - -- **`0x00`↔`0xFF` projection:** the resolver compares via `getLastByteOfNodeNum` on both - sides, so a `…00` node and a `…FF` node correctly collide on `0xFF` → `Ambiguous`. Test - explicitly. -- **MQTT packets:** `relay_node`/`next_hop` are forced invalid when `hop_start == 0` - (`src/mesh/RadioLibInterface.cpp:603-605`) → byte 0 → resolver `None` → don't learn - (correct). -- **`has_hops_away == false`** nodes are excluded from the strict gate (never fabricate a - Unique neighbor for M2); admitted to the lenient gate only via favorite/router role. - Safe; self-corrects once `hops_away` is learned. -- **Self / broadcast:** the resolver skips `getNodeNum()` and `NODENUM_BROADCAST`; - `getNextHop` already early-returns for broadcast. -- **Perf:** M2 adds one O(N) resolver scan per directed send/relay (early-exit on the 2nd - match), cheaper than the crypto already on that path; site-4 is a wash. If ever hot, a - future 256-entry last-byte index is the optimization (not now - RAM). - ---- - -## Verification (all tiers) - -### 1. Native unit tests - new `test/test_nexthop_routing/test_main.cpp` - -`pio test -e native -f test_nexthop_routing`; on macOS `./bin/test-native-docker.sh -f test_nexthop_routing`. -Design the RouteHealth helpers to take `now` as a parameter so the 30-min TTL logic is -testable without a clock mock. - -- **Resolver:** None / Unique / **Ambiguous (birthday collision)** / strict-excludes-stale / - strict-excludes-far / lenient-includes-favorite-router / lenient-collision / skips-self / - skips-ignored / **`0x00`↔`0xFF` collision** / early-exit. -- **`getNextHop`:** unique→byte, **ambiguous→nullopt**, stale-neighbor→nullopt, - split-horizon (relay==next_hop)→nullopt, broadcast→nullopt. -- **RouteHealth:** TTL boundary, **rollover** (learn near `0xFFFFFFFF`, check after wrap), - failure threshold, success-resets, **re-learn-same-hop keeps fails (anti-flap)**, - re-learn-new-hop resets, LRU eviction bound, clear. -- **Site-4:** preserve on unique favorite router; **decrement on two colliding favorites**; - decrement when the resolved node is not a favorite. -- **Sparse-mesh regression:** all-distinct last bytes → every resolve Unique, `getNextHop` - returns the stored byte unchanged (proves no happy-path change). -- Re-run `test_packet_history` and `test_hop_scaling` for no regression. - -### 2. portduino SimRadio simulator - -`pio run -e native && ./bin/test-simulator.sh`. Best vehicle for the **intermediate-node** -path the 2-device bench can't reach. Line topology A - B - C: establish A→C (B learns a -directed route), stop B relaying that dest, confirm A re-discovers via flood within -`ROUTE_FAILURE_THRESHOLD` and that B's `noteRouteFailure`/`clearRouteHealth` fires (visible -via the `LOG_INFO "Route to … stale"` / "Resetting next hop" lines). Use this to A/B M4 -(attempts-to-delivery, total airtime). - -### 3. Hardware via meshtastic MCP (auto-detect; 3+ devices for a real hop) - -- `meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py` - **the multi-hop validator - for this work** (added on this branch). Self-discovers an A - relay - C line, asserts a - directed DM is delivered across the relay (next_hop + M1/M2/M3 engaged), and asserts - delivery recovers after the relay is power-cycled (M3). Skips unless the bench is a true - multi-hop line (≥3 roles via `--hub-profile`, endpoints out of direct RF range). -- `meshtastic-mcp/tests/mesh/test_direct_with_ack.py` - happy-path regression: a fresh/unique - route still delivers a want_ack DM on the first/second try (M4's gate must keep this - green). -- `meshtastic-mcp/tests/mesh/test_peer_offline_recovery.py` - 2-device recovery validator: peer - off mid-conversation then back. Must stay green and ideally recover in fewer attempts. - -### 4. Build / format sanity - -native-macos **and** Docker both ways; trunk clang-format@16.0.3; a release `pio run` to -confirm the `#ifdef PIO_UNIT_TESTING` visibility widening does **not** leak into -production; sanity-check RAM headroom on the smallest nRF52 build for the ~384 B table. - ---- - -## Verification status (as built on `nexthop-redux`) - -| Tier | What ran | Result | -| -------------------------------- | ----------------------------------------------------------------------------------- | ------------------- | -| Unit (native-macos) | `test_nexthop_routing` (31 cases) | ✅ 31/31 | -| Unit (Docker / Linux, CI parity) | `test_nexthop_routing` | ✅ 31/31 | -| Regression | `test_packet_history`, `test_hop_scaling`, `test_mqtt`, `test_traffic_management` | ✅ 105/105 | -| Build | `pio run -e native-macos` (M4 off) and with `-DNEXTHOP_EARLY_FLOOD_ON_UNVERIFIED=1` | ✅ both link | -| Format | trunk `clang-format@16.0.3` | ✅ no issues | -| Simulator (CI `simulator-tests`) | `meshtasticd -s` + `meshtastic.test.testSimulator()` on native-macos | ✅ exit 0, no crash | - -**Pending (environment-blocked, not yet run):** - -- **Multi-hop A-B-C recovery sim** - the `simulator/` broker hub is **not git-tracked** - (only stale local `.pyc`), and two `meshtasticd -s` instances can't hear each other - without it. The intermediate-node failure-count path and the M4 A/B therefore have unit - coverage of their logic but no end-to-end multi-node run yet. -- **Hardware / multi-hop tier** - a committable bench test now exists: - `meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py`. It self-discovers a real - multi-hop pair (A - relay - C), asserts a directed DM is delivered across the relay, and - asserts delivery recovers after the relay is power-cycled (the M3 path). It - `pytest.skip`s cleanly unless the bench is a true line with endpoints out of direct RF - range (≥3 roles via `--hub-profile`), so it's safe to commit and only asserts when the - NextHop path is genuinely exercised. Collected + verified to skip without hardware; - not yet run on a bench. `test_direct_with_ack.py` / `test_peer_offline_recovery.py` - remain the 2-device happy-path/recovery regressions. - ---- - -## Risks & limitations - -- **Site-1 impostor rebroadcast** is unfixable without a wider field - documented; M1/M2 - only shrink its frequency. -- **Dense meshes flood DMs more often** - intended (a flooded DM arrives; a mis-unicast one - black-holes). Call out in the PR so reviewers expect a slightly higher DM flood rate on - very dense meshes. -- **M4 airtime** if the gate is too loose → default conservative + compile-gated + - simulator A/B before broad enable. -- **RAM** ~384 B (32 slots); 16 slots (~192 B) with graceful LRU degradation if tight. -- **Asymmetric flap** not fully closed (a _new_ bad hop resets the counter); the TTL - backstop bounds it. Per-hop failure history is future work (more RAM). - ---- - -## How to continue this work (commit sequencing) - -Each step is independently testable; land them as separate commits. - -1. **M1 resolver + unit tests** - `NodeDB` only; no behavior change until wired. Lands the - `resolveLastByte`/`resolveUniqueLastByte` primitive and its full unit-test matrix. -2. **M2 + wiring + tests** - `getNextHop` strict gate, learning gate, favorite-router - preservation rewrite. Adds the `getNextHop` and site-4 tests. -3. **M3 health table + decay + tests** - RAM `RouteHealth` table, decay-on-read, failure/ - success accounting, reconciliation with the existing last-retry reset. Adds the - route-health unit tests and the simulator recovery check. -4. **M4 gated tuning** - early-flood-on-unverified behind the compile flag; simulator A/B - and hardware regression. - -Reference plan (with the same content) was developed at -`~/.claude/plans/nexthop-routing-for-direct-lexical-shell.md` on the author's machine; this -in-repo doc is the canonical handoff copy. diff --git a/docs/node_info_stores.md b/docs/node_info_stores.md deleted file mode 100644 index 7908f9f9053..00000000000 --- a/docs/node_info_stores.md +++ /dev/null @@ -1,321 +0,0 @@ -# NodeInfo stores: the base and extended databases - -This document is an overview of the node-identity and traffic-state databases that the -TrafficManagementModule (TMM) either owns or leans on. There are four stores in play, but -only three form the identity lookup chain: - -1. **NodeDB hot store** - the authoritative `NodeInfoLite` array (identity tier 1). -2. **Warm tier** (`WarmNodeStore`) - minimal persisted records for hot-store evictees - (identity tier 2). -3. **TMM NodeInfo payload cache** (extended) - the ephemeral **third identity tier**: full - `User` payloads plus direct-response metadata; PSRAM-backed on hardware, plain heap in - native tests. - -The fourth store, the **TMM unified cache** (base - flat 10-byte-per-node traffic-shaping -state), is not part of that chain: it sits beside it, keyed by the same NodeNum, and only -its 4-bit cached role acts as a final fallback when all three identity tiers miss. - -Sources of truth: `src/mesh/NodeDB.{h,cpp}`, `src/mesh/WarmNodeStore.h`, -`src/modules/TrafficManagementModule.{h,cpp}`, sizing in `src/mesh/mesh-pb-constants.h`. - -**Memory classes.** The warm tier (§2) and unified cache (§3) size themselves from -`MESHTASTIC_MEM_CLASS` (`src/memory/MemClass.h`), which ranks a build by _usable app heap after -platform overheads_ (SoftDevice, WiFi+BLE stacks) rather than by raw RAM or chip family. The hot -store (§1) is flash-shaped and the NodeInfo cache (§4) is present-or-absent, so neither is classed: - -| Class | Heap | Parts | -| ------ | --------------------- | -------------------------------------------- | -| LARGE | PSRAM or host | ESP32-S3 with PSRAM, portduino/native | -| MEDIUM | ~250-500 KB, no PSRAM | ESP32-S3/C6/P4 without PSRAM | -| SMALL | ~100-250 KB | classic ESP32/S2/C3, nRF52840, RP2040/RP2350 | -| TINY | <32 KB | STM32WL | - -An unclassified chip lands in SMALL on purpose: small caches are a recoverable default, an -exhausted heap is not. Where a capacity table names a specific part beside these classes, that -part is deliberately class-deviant and the reason is given under the table. - ---- - -## 1. NodeDB hot store (authoritative) - -- **What:** the classic `meshNodes` array of `meshtastic_NodeInfoLite` - full identity as - flattened fields (names, role, public key, bitfield flags such as `HAS_XEDDSA_SIGNED`; - position/telemetry live in satellite stores reached via copy-out accessors, not nested - members). Everything else in this document is a cache or a fallback for it. -- **Eviction:** oldest non-protected node when full (`getOrCreateMeshNode`). On eviction - the node's essentials are **absorbed into the warm tier** (see §2); on re-admission the - warm record is rehydrated back (`take()`), including the XEdDSA-signed bit. -- **Persistence:** the node database file in LittleFS, saved on the usual NodeDB cadence. -- **Authority:** key pinning (`updateUser`'s "Public Key mismatch" drop), signer - provenance, and identity content all originate here. The lookup helpers that other - stores mirror: - - `copyPublicKeyAuthoritative(n, out)` - hot store, then warm tier. The pin reference - for caches; never consults opportunistic caches. - - `copyPublicKey(n, out)` - the above, then **TMM's NodeInfo cache as last resort** - (extends the encrypt-to pool for nodes both tiers have forgotten). - - `isVerifiedSignerForKey(n, key32)` - key-matched signer verdict across hot + warm. - - `isKnownXeddsaSigner(n)` - key-agnostic "should this node's signable traffic arrive - signed", across hot + warm. Gates that check only the hot store would let a - warm-evicted signer be impersonated with unsigned frames. - - `getNodeRole(n)` - hot store, then the role cached in the warm tier, else `CLIENT`. - -**Capacity** - `MAX_NUM_NODES`: - -| ESP32-S3 | Native (portduino) | nRF52840, generic ESP32 | STM32WL | -| --------------- | ------------------ | ----------------------- | ------- | -| 250 / 200 / 100 | 200, configurable | 120 | 10 | - -This one is flash-shaped rather than heap-shaped, so it is unclassed: `nodes.proto` has to fit the -filesystem. The fixed-cap platforms get their value from `mesh-pb-constants.h`; the 120 covers -nRF52840 plus generic ESP32 including C3, and is what keeps `nodes.proto` inside the stock 28 KB -LittleFS. - -**Two platforms do not take their cap from that header, and neither is a compile-time constant:** - -- **ESP32-S3** picks a tier at boot from the flash chip size (>=15 MB / >=7 MB / smaller). -- **Native/portduino** resolves it from _runtime_ config: - `variants/native/portduino{,-buildroot}/variant.h` define `MAX_NUM_NODES portduino_config.MaxNodes`, - default **200** (`PortduinoGlue.h`), overridable per-host with `General: MaxNodes` in the YAML. - Because `variant.h` is reached first, the `ARCH_PORTDUINO` branch of `mesh-pb-constants.h` never - fires - it is `#error`-guarded so it can no longer be misread as the native cap. - -Do not grep `mesh-pb-constants.h` for the native number: the protected-node cap derives from -`MAX_NUM_NODES` (`numProtectedNodes() < MAX_NUM_NODES - 2`), so a wrong reading gives a wrong cap -(248 instead of 198) and makes a genuinely saturated database look impossible. - -The separate `250` in `NodeDB::getMaxNodesAllocatedSize()` is `NODEDB_MIGRATION_LOAD_CEILING`, a -decode allowance for files written by larger-cap firmware. It is not a cap on this build. - -## 2. Warm tier - `WarmNodeStore` (NodeDB-owned) - -- **What:** the "long-tail" second tier. When a node ages out of the hot store, a minimal - record survives so DMs keep encrypting: the key is expensive to re-learn; everything - else rebuilds from traffic in seconds. -- **Entry:** exactly 40 bytes - `num(4) | last_heard(4) | public_key(32)`. The low 7 bits - of `last_heard` are omitted, and replaced with metadata (role: 4 bits, protected - category: 2, XEdDSA-signed bit: 1), leaving ~128 s recency resolution - plenty for LRU ranking. -- **Capacity:** `WARM_NODE_COUNT` (100 on constrained parts; platform-tiered). -- **Eviction:** LRU by `last_heard`, with keyed entries outranking keyless; keyless - candidates never displace keyed entries. -- **Persistence:** nRF52840 uses a 12 KB raw-flash record-ring below LittleFS - (append/replay/compact); everywhere else `/prefs/warm.dat` (LittleFS). -- **Membership invariant:** a node lives in the hot **XOR** warm tier. `take()` removes - the warm record when the node is re-admitted hot, restoring role/protected/XEdDSA-signed bits. - -**Capacity** - `WARM_NODE_COUNT` (`mesh-pb-constants.h`): - -| LARGE | MEDIUM | RP2040 / RP2350 | nRF52840 | SMALL | TINY | -| ----- | ------ | --------------- | -------- | ----- | ---- | -| 2000 | 150 | 150 | 100 | 100 | 0 | - -TINY's 0 disables the tier outright. At 40 B/entry, LARGE costs ~80 KB and lives in PSRAM, MEDIUM -~6 KB of heap. Both named parts are class-deviant on purpose: RP2040/RP2350 is bounded so the -`warm.dat` write fits the 8 s watchdog (#10746) rather than by RAM, and nRF52840 dropped from 200 to -100 because its RAM cache is calloc'd from the ~115 KB heap arena shared with SoftDevice, which -2.8.0 field reports showed at 99% use. - -## 3. TMM unified cache (base, traffic state) - -- **What:** TMM's own flat array of packed 10-byte `UnifiedCacheEntry` records - the - per-node state behind position dedup, rate limiting, unknown-packet filtering, plus two - piggybacked caches: - - `next_hop` - last-byte relay hint, written only from ACK-confirmed NextHopRouter - decisions (no TTL; keeps the slot alive across sweeps). - - a **4-bit device role** (split across the top bits of two count bytes) - the _third_ - fallback for role-aware policy after the hot store and warm tier, surviving even total - NodeDB eviction. Read through `resolveSenderRole()`, refreshed by - `updateCachedRoleFromNodeInfo()` on observed NodeInfo. -- **Entry layout:** - `node(4) | pos_fingerprint(1) | rate_count(1) | unknown_count(1) | pos_time(1) | rate_unknown_time(1) | next_hop(1)` - = 10 bytes, all platforms. Timestamps are free-running modular ticks (uint8 / nibbles) - with presence carried by non-zero sentinels - no epochs, no absolute time. -- **Eviction:** linear scan; insertion on a full cache evicts the stalest entry, - preferring to keep entries with a `next_hop` hint **or** a cached special (non-`CLIENT`) - role - the long-tail state this cache exists to retain (`findOrCreateEntry`'s `preferred` - test covers both, not just `next_hop`). -- **Persistence:** none - PSRAM (or heap) only, rebuilt from traffic. - -**Capacity** - `TRAFFIC_MANAGEMENT_CACHE_SIZE` (`mesh-pb-constants.h`), variant-overridable: - -| LARGE | MEDIUM | SMALL | nRF52840 | `HAS_TRAFFIC_MANAGEMENT=0` | -| ----- | ------ | ----- | -------- | -------------------------- | -| 2048 | 500 | 400 | 250 | 0 | - -At 10 B/entry that is ~5 KB on MEDIUM and ~2.5 KB on nRF52840, which is class-deviant for the same -heap reason as the warm tier (its class would give 400); 250 entries still tracks over 2x the -120-node hot store, and LRU victim recycling absorbs busier meshes. - -## 4. TMM NodeInfo payload cache (extended, the ephemeral third tier) - -- **What:** a flat array of `NodeInfoPayloadEntry` (PSRAM-backed on hardware; see - Availability) - the full cached `User` payload (names, role, key) plus the metadata that - backs TMM's **spoofed direct NodeInfo replies** on a target's behalf, independent of - NodeDB (the serve/throttle behaviour is documented in - [traffic_management_module.md](traffic_management_module.md)). Also the last-resort key - source for `NodeDB::copyPublicKey()`. -- **Availability:** `TMM_HAS_NODEINFO_CACHE` - ESP32 with PSRAM (production home; 2000 - entries is too large for MCU internal RAM), plus native unit-test builds on the plain - heap so the trust/retention paths run in CI. -- **Entry:** `node`, `user` (full nanopb `User`), the `obsTick` recency stamp (3 min/tick), - `sourceChannel`, `decodedBitfield`, and packed 1-bit flags: `hasDecodedBitfield`, - `keyXeddsaSigned`, `keyManuallyVerified`, `hasObserved`, `hasFullUser`, `isMember`. (The direct-response throttle - no longer keeps per-entry state here - it is a pair of separate RAM tables; see the module - doc.) -- **Persistence:** none - this tier is deliberately ephemeral; it reconstructs from NodeDB - seeding plus observed traffic after every boot. - -**Capacity** - `kNodeInfoCacheEntries` (`TrafficManagementModule.h`), gated by -`TMM_HAS_NODEINFO_CACHE`: - -| ESP32 + PSRAM | Native unit-test builds | Everything else | -| ------------- | ----------------------- | --------------- | -| 2000 | 2000 | not compiled | - -Not class-tiered: the array is either compiled or it isn't. ESP32+PSRAM is the production home (in -PSRAM); native test builds put the same 2000 entries on the plain heap so the trust and retention -paths run in CI. Linear scan in every build - NodeInfo traffic is low-rate. - -### Trust & provenance model - -- **Key pin, three layers deep:** an incoming NodeInfo key is checked against - `copyPublicKeyAuthoritative()` (hot then warm - the same coverage as `updateUser`'s own - pin), and, failing NodeDB knowledge, against the cache's **own previously cached key** - (TOFU pin). Mismatches are dropped, never overwritten. A frame advertising _our own_ key - is dropped outright (impersonation). -- **Key provenance (`keyXeddsaSigned` + `keyManuallyVerified`, combined via `keyProven()`):** - `keyXeddsaSigned` is set when a frame's XEdDSA signature was router-verified - (`mp.xeddsa_signed`) or when NodeDB already knew the node as a signer **for the same key** - (`isVerifiedSignerForKey`). `keyManuallyVerified` is set when the user confirmed possession - out-of-band (QR / fingerprint), routed via `onNodeKeyCommitted(proven)` and re-seeded from the - hot store's `is_key_manually_verified` bit at reconcile. Either bit makes `keyProven()` true - - the predicate the replay gate, eviction tiering, and pubkey-pool callers use. Both are monotonic - per slot; a changed key resets both. -- **Unsigned-identity gate:** a NodeInfo arriving _unsigned_ from a node we have ever - verified as a signer - per `NodeDB::isKnownXeddsaSigner()`, which covers hot **and - warm** tiers - drives no cache, role, or `updateUser()` write. (Warm coverage matters: a - signer evicted to the warm tier would otherwise be forgeable with its own public key - until re-heard. The same rule guards `Router::checkXeddsaReceivePolicy`'s - unsigned-broadcast drop.) -- **Serve gate honesty:** only a genuinely _heard_ NODEINFO frame stamps - `obsTick`/`hasObserved` - seeding and write-through don't, so a silent node never looks alive - to the replay path. The sweep clears `hasObserved` to enforce the 6 h serve window. The - spoofed-reply throttle this gate feeds lives in the module (see - [traffic_management_module.md](traffic_management_module.md)). - -### Consistency with NodeDB (anti-entropy) - -Four mechanisms keep this tier a superset of NodeDB's identities. All **merge rather than -overwrite**, so a keyless commit never costs the cache a learned TOFU key. - -| Mechanism | When | Role | -| --------------------------------------------------------------------- | --------------------------- | -------------------------------- | -| Write-through hooks (`onNodeIdentityCommitted`, `onNodeKeyCommitted`) | every identity/key commit | immediate upsert | -| Reconcile sweep (`reconcileNodeInfoFromNodeDBLocked`) | boot seed, then hourly | re-seed from hot + warm tiers | -| Membership refresh | inside the hourly reconcile | re-mark which nodes NodeDB holds | -| Purge hooks (`purgeNode`, `purgeAll`) | node removal / reset | drop the node from both caches | - -Two details that bite: the reconcile sweep transfers signer verdicts only when **key-matched**; -and membership refresh clears-then-re-marks from both tiers rather than a per-entry NodeDB lookup -each sweep (which would be O(entries x members) under the lock). A keyless warm-tier record still -marks membership (`isMember`) even though it has no `User` to seed - `isMember` is a keep-alive, -independent of `hasFullUser`. Because the re-mark is only hourly, hook-driven additions and -`purgeNode()` removals are immediate, but a **passive** NodeDB eviction may lag membership by up to -an hour. - -**Retention:** no timed eviction. Slots die only by LRU displacement on insert, ranked by -trust tiers - members and key-proven keys are stickiest; the seeding pass additionally -refuses to churn one member out for another (`spareMembers`). - -**Key-commit funnel:** every path that writes a remote key into the hot store must route -the write-through. Full-identity commits funnel through `NodeDB::updateUser()`; bare-key -commits (admin-channel learn in `Router::perhapsDecode`, manual verification in -`KeyVerificationModule`) funnel through `NodeDB::commitRemoteKey()`, which carries an -explicit `KeyCommitTrust` provenance (`ManuallyVerified` sets the `keyManuallyVerified` bit in this -cache). Never assign `info->public_key` directly when **learning or rotating a remote -key** - the cache would silently diverge until the next reconcile. (The lone direct write -in `getOrCreateMeshNode()`'s warm-tier re-admission is exempt: it restores a key the warm -tier already holds, which this cache already tracks as a member, so nothing new is learned -and the hourly reconcile re-seeds it even if the packet path had LRU-evicted that slot.) - -**Enable gate:** the write-through hooks, the sweep, the packet path, **and the -`copyPublicKey()`/`copyUser()` accessors** all no-op while `moduleConfig.has_traffic_management` -is off, so cache content, maintenance, and reads are keyed to the same condition. This enforces -(not just documents) the corollary that the pubkey-pool superset property holds only while the -module is enabled: a disabled module's frozen cache never feeds PKI resolution or name -rehydration. - -### Tick clocks and wrap safety - -This cache's `obsTick` recency stamp, like the unified cache's pos/rate/unknown stamps, is a -free-running modular tick rather than an absolute time, and depends on the maintenance sweep to -clear expired state before it aliases. The per-clock periods, windows, and what keeps each honest -are documented with the module in -[traffic_management_module.md](traffic_management_module.md#tick-clocks-and-wrap-safety). The sharp -case for this tier is `obsTick`: the sweep clearing `hasObserved` is the _sole_ guarantee the 6 h -serve gate never reads an aliased stamp, which is why it is a compile-time invariant guarded by -`TMM_HAS_NODEINFO_CACHE` alone. - -The warm tier is different by design: `WarmNodeStore.last_heard` is an **absolute** unix-seconds -timestamp (128 s quantised), so it cannot wrap until 2106 and needs no sweep - the TMM caches -chose 1-byte ticks instead to stay at 10 B/entry across up to 2048 entries. - -### Direct-response behavior - -How this cache's identities are served as spoofed direct NodeInfo replies - the serve gates, -the per-requester/per-target/global throttle, and the "throttled forwards, not dropped" -behaviour - is documented with the module in -[traffic_management_module.md](traffic_management_module.md). - ---- - -## Property matrix - -Side-by-side view of what each store actually holds ("-" = not held). Details and -rationale live in the per-store sections above. - -| Property | 1. Hot store | 2. Warm tier | 3. NodeInfo cache | 4. Unified cache | -| -------------------------- | ---------------------------------- | ------------------------------ | ---------------------------------- | ------------------------------- | -| Struct | `NodeInfoLite` | `WarmNodeEntry` | `NodeInfoPayloadEntry` | `UnifiedCacheEntry` | -| Node number | yes | yes | yes (0 = free) | yes (0 = free) | -| Names + user id | yes (flattened) | - | yes (full `User`) | - | -| Public key (32 B) | yes (authoritative) | yes (keyed entries) | yes (TOFU/proven; pinned) | - | -| Key source - XEdDSA signed | `HAS_XEDDSA_SIGNED` bit | 1 bit (in `last_heard`) | `keyXeddsaSigned` | - | -| Key source - manual scan | `IS_KEY_MANUALLY_VERIFIED` bit | - (not carried) | `keyManuallyVerified` | - | -| Device role | `role` field | 4-bit role (metadata steal) | in cached `User` | 4-bit role (final fallback) | -| Recency | `last_heard` (unix s) | `last_heard` (128 s quant.) | `obsTick` (3 min) + `hasObserved` | modular ticks | -| Position / telemetry | satellite accessors | - | - | 8-bit pos fingerprint (dedup) | -| Protected / favorite | bitfield flags | 2-bit protected category | - (`isMember` instead) | - | -| Routing hint (`next_hop`) | yes (persisted) | - | - | ACK-confirmed relay byte | -| Direct-reply metadata | - | - | `sourceChannel`, `decodedBitfield` | - | -| Traffic-shaping counters | - | - | - | rate + unknown counts, pos fp | -| Entry size | largest (full struct) | 40 B exact | ~`sizeof(User)`+8 (padded) | 10 B exact | -| Capacity (symbol) | `MAX_NUM_NODES` | `WARM_NODE_COUNT` | `kNodeInfoCacheEntries` | `TRAFFIC_MANAGEMENT_CACHE_SIZE` | -| Capacity (entries) | 250/200/120/100/10 (native: 200\*) | ~100 | 2000 | 2048/500/400/250/0 | -| Persistence (durable) | LittleFS (node DB) | flash ring (nRF52840)/LittleFS | none (rebuilt) | none | -| Storage (runtime) | heap | heap / PSRAM (ESP32) | PSRAM (hw) / heap (test) | PSRAM / heap | - -\* Native/portduino is not a compile-time value: it is `portduino_config.MaxNodes`; the host default -is 200, settable per-host via `General: MaxNodes`, and the WASM build overrides it to 80 -(`wasm_config_apply()`). See the hot-store capacity section above. - -## How a lookup falls through the tiers - -```text -identity/role/key consumer - │ - ▼ - 1. hot store (NodeInfoLite) full identity, authoritative - │ miss - ▼ - 2. warm tier (WarmNodeStore) key + role/protected/XEdDSA-signed bits, persisted - │ miss - ▼ - 3. TMM NodeInfo cache (extended) full User payloads + TOFU/proven keys, ephemeral - │ miss (role-only: 4-bit role in the unified cache) - ▼ - defaults (no key; role = CLIENT) -``` - -The unified cache (§3) sits beside this chain rather than in it: it is traffic-shaping -state keyed by the same NodeNum, whose role bits act as the final role fallback when all -three identity tiers miss. diff --git a/docs/traffic_management_module.md b/docs/traffic_management_module.md deleted file mode 100644 index cf4f9538e04..00000000000 --- a/docs/traffic_management_module.md +++ /dev/null @@ -1,222 +0,0 @@ -# The Traffic Management Module (TMM) - -TMM is an optional module that shapes **transit** traffic on busy meshes. Large networks get -noisy fast - repeated position packets, bursty senders, and unknown/undecryptable frames all -burn limited airtime and power - and TMM filters or answers that traffic before it is -rebroadcast. On supported targets it **ships enabled** (`has_traffic_management` defaults to -true) with position dedup running at its 11 h default; the other features each default off, so -the module is on out of the box but opt-in per feature. It was introduced in -[meshtastic/firmware#9358](https://github.com/meshtastic/firmware/pull/9358). - -This document covers the module's behaviour, with a deep dive on the two TMM-specific -NodeInfo features - **direct-serve** (answering NodeInfo requests on another node's behalf) -and the **throttling** that bounds it. The identity/traffic-state stores those features read -from are documented separately in [node_info_stores.md](node_info_stores.md); this file owns -the direct-serve and throttle behaviour, that file owns the stores. - -Sources of truth: `src/modules/TrafficManagementModule.{h,cpp}`, defaults in -`src/mesh/Default.h`. - ---- - -## How it runs - -- **Enablement is three-gated.** Compile-time `HAS_TRAFFIC_MANAGEMENT` (with the - `MESHTASTIC_EXCLUDE_TRAFFIC_MANAGEMENT` build exclusion), then the runtime - `moduleConfig.has_traffic_management` presence flag. While the runtime gate is off, the - packet path, the maintenance sweep, the NodeDB write-through hooks, and the cache accessors - all no-op - content, maintenance, and reads are keyed to the same condition. -- **It runs before `RoutingModule`** in `callModules()`. Returning `STOP` from - `handleReceived()` fully consumes a packet, so it is never rebroadcast; `CONTINUE` lets it - proceed through normal relay handling. -- **State is cheap.** Per-node traffic-shaping counters live in a flat 10-byte - `UnifiedCacheEntry` array (position fingerprint, rate/unknown counters, modular tick - stamps, a next-hop hint, and a 4-bit role fallback) - see - [node_info_stores.md §3](node_info_stores.md). Direct-serve additionally reads the PSRAM - NodeInfo payload cache (or the NodeDB fallback when that cache is absent). - -## What it does - -| Feature | Default | In one line | -| ------------------------ | -------------- | -------------------------------------------------------------- | -| Position dedup | on, 11 h | Suppresses a stationary sender's repeated position broadcasts. | -| Per-sender rate limit | off | Caps how many transit packets one sender may spend per window. | -| Unknown-packet filter | off | Drops a sender's undecryptable traffic past a threshold. | -| NodeInfo direct response | off | Answers a NodeInfo request on the target's behalf (see below). | -| Position precision clamp | channel-driven | Truncates relayed position to the channel's precision. | - -Config lives under `moduleConfig.traffic_management`; the per-feature sections below give the -exact fields, defaults, and behaviour. NodeInfo direct response has its own deep-dive sections -after these. - -### Position dedup - -`position_min_interval_secs` (default 11 h; `0` disables). Drops a duplicate position from the -same sender inside the interval, where "duplicate" means the same fingerprint on the channel's -`position_precision` grid (firmware default 19-bit, ~90 m cells). Role caps only ever _shorten_ -the interval: **tracker / TAK tracker → 1 h**, **lost-and-found → 15 min**. - -### Per-sender rate limit - -`rate_limit_window_secs` + `rate_limit_max_packets` (default off; either `0` disables). Drops a -sender's transit packets once it exceeds the budget within the window. - -### Unknown-packet filter - -`unknown_packet_threshold` (default `0` = off). Drops undecryptable traffic from a sender once it -passes the threshold within a ~5 min window. - -### NodeInfo direct response - -`nodeinfo_direct_response_max_hops` (default `0` = off). When set, a neighbour that already -holds the target's identity answers a unicast NodeInfo request on its behalf, saving the full -round trip. This is TMM's most security-sensitive feature; the serve gates and the throttle -that bounds it are covered in the two dedicated sections below. - -### Position precision clamp - -Driven by the channel's `position_precision` ceiling (else the 19-bit firmware default). -`alterReceived()` truncates relayed position coordinates to that precision. - -### Shelved - -Present in the config surface but currently no-ops in the module, deferred until the right -heuristics are settled: hop exhaustion for position/telemetry (`exhaust_hop_position` / -`exhaust_hop_telemetry`) and `router_preserve_hops`. `alterReceived()` leaves rebroadcast hop -handling untouched. - ---- - -## NodeInfo direct response (direct-serve) - -Normally a unicast NodeInfo request travels all the way to the target and the reply travels -all the way back. On a large mesh that is several hops of airtime per lookup. When -`nodeinfo_direct_response_max_hops > 0`, a neighbour that already holds the target's identity -answers **on the target's behalf** with a spoofed reply, cutting the round trip to one hop. - -**Data source.** The reply payload comes from the TMM NodeInfo payload cache (PSRAM-backed; -full cached `User` plus provenance metadata) or, on builds without that cache, from the -NodeDB fallback. Both are described in [node_info_stores.md §4](node_info_stores.md); this -feature is a _consumer_ of them. - -**Decision pipeline** (`shouldRespondToNodeInfo()`), in order - any failure returns `false` -and the request is left to propagate normally: - -1. **Eligibility** (checked by the caller): `nodeinfo_direct_response_max_hops > 0`, - `NODEINFO_APP` portnum, `want_response`, and the packet is unicast, not to us, not from us. -2. **Hop clamp** (`isMinHopsFromRequestor()`): respond only when the requester is within the - role-clamped hop ceiling - **routers up to 3 hops** (`kRouterDefaultMaxHops`, may be - lowered by config), **clients direct-only, 0 hops** (`kClientDefaultMaxHops`). -3. **Identity lookup**: NodeInfo cache hit (cache path) or NodeDB fallback (fallback path). -4. **Staleness gate (6 h)**: never vouch for a node not genuinely _heard_ within the serve - window. Only a real observed frame stamps the recency bit - seeding and write-through are - knowledge, not observation, so a silent node can never look alive to this path. -5. **Key-provenance gate** (`TMM_NODEINFO_REPLAY_SIGNED_GATE`, default on): vouch only for - an identity whose key is proven - XEdDSA-verified (directly or inherited from NodeDB) **or** - manually verified out-of-band. Both paths honour both channels: the cache path via - `keyProven()`, the NodeDB fallback path via `HAS_XEDDSA_SIGNED | IS_KEY_MANUALLY_VERIFIED`. A - trust-on-first-use identity is left for the genuine node - or another cache-holder that _has_ - proof - to answer. Bypassed when PKI is compiled out. -6. **Throttle** (`directResponseAllowed()`): see the next section. - -**The spoofed reply.** On success TMM emits a NodeInfo reply with `from` set to the _target_ -(so the requester sees a valid answer), `to` the requester, `hop_limit = 0` (one hop only), -`request_id` the original packet id, and the OK_TO_MQTT bit set from local -`config.lora.config_ok_to_mqtt` policy. The requester's own identity claim in the request is -**not** written back to NodeDB - a unicast NodeInfo is unsigned, so treating it as an -identity update would be unauthenticated. `nodeinfo_cache_hits` counts only replies actually -sent. - ---- - -## Throttling direct responses - -A direct reply is addressed to the requesting packet's `from` and spoofs the requested -target - and **both fields are unauthenticated header data**. Without a bound, an attacker -crafts requests carrying a victim's address as `from`, and every neighbour holding the target -transmits at the victim: a reflector-amplification primitive. The throttle is the security -core of this feature, checked immediately before a reply would go out so requests declined for -other reasons never consume the budget. - -**Three bounds**, all keyed off `clockMs()` and evaluated under `cacheLock`: - -| Bound | Window | Bounds | -| ------------------------------------------------ | ------ | ------------------------------------------------ | -| Per requester (`kDirectResponsePerRequesterMs`) | 60 s | how much any single node can be made to receive | -| Per target (`kDirectResponsePerTargetMs`) | 60 s | how often we vouch for the same identity | -| Global airtime floor (`kDirectResponseGlobalMs`) | 1 s | total spoofed TX, regardless of key distribution | - -**Mechanism.** The two per-key bounds are fixed **8-slot LRU tables in internal RAM** -(`directRequesterSeen`, `directTargetSeen`) - _not_ the PSRAM NodeInfo cache - so they behave -identically with and without PSRAM, on the cache path and the NodeDB-fallback path alike. -Timestamps are full `uint32` milliseconds compared by wrap-safe subtraction, so there is no -tick clock and no maintenance sweep to keep them honest. `directResponseAllowed(requester, -target, now)` resolves a slot in _both_ tables before stamping either - so a reply one axis -throttles never consumes the other axis's budget - then records the send on all three bounds. -The global floor is a single stamp, checked first as the cheap common case. - -**When a table fills.** For an unseen key with no free slot, `directResponseSlot()` evicts the -**least-recently-used** entry (smallest last-reply time) and admits the new key. The LRU -victim is by construction the entry closest to expiring anyway, so eviction is the -lowest-cost choice. An attacker who cycles more than 8 distinct requesters or targets - easy, -since both are unauthenticated - evicts entries and defeats _per-key_ throttling for the -cycled keys; that is expected, and why the **global 1 s floor is the hard backstop**. It is a -single stamp, cannot fill, and caps total spoofed replies at ~1/s no matter what. Per-key -throttling degrades gracefully to the floor under pressure. - -**Throttled is not dropped.** A throttled request returns `false`, which lets -`handleReceived()` `CONTINUE`: the request forwards toward the genuine target (which can -answer itself) rather than being black-holed. A requester whose first reply was lost on a -noisy link would otherwise get silence for the whole window; repeats of the same packet id -are already absorbed by the router's duplicate detection. - -**Evolution.** The original design split throttling by path: a per-entry `respTick` stamp in -each NodeInfo cache slot (cache path, 30 s, swept for wrap-safety) plus a single module-global -stamp for the NodeDB fallback (30 s, neither per-requester nor per-target). Those two routes -were unified into the symmetric per-requester + per-target RAM tables above, aligned to a -single 60 s window, so both axes hold with and without PSRAM and the cache entry no longer -carries throttle state. - ---- - -## Tick clocks and wrap safety - -Every per-node timestamp in TMM's caches is a free-running modular tick (uint8 or nibble) taken -from `clockMs()` - never an absolute time. That is what keeps `UnifiedCacheEntry` at 10 bytes -across up to 2048 entries. The cost is that modular subtraction is only correct while the true age -stays below the counter's period, so every clock needs something to clear expired state before it -aliases. (The direct-serve throttle above is the deliberate exception: full `uint32` milliseconds -compared by wrap-safe subtraction, hence no tick and no sweep.) - -| Clock | Tick / period | Window | Kept honest by | -| ------------------ | -------------- | --------------- | -------------------------------------------------- | -| pos | 6 min / 25.6 h | <=255 ticks | 60 s sweep (margin as low as 1 tick at the clamp) | -| rate | 5 min / 80 min | <=15 ticks | sweep + read-time window reset (`isRateLimited()`) | -| unknown | 1 min / 16 min | 12 ticks | sweep + read-time window reset | -| NodeInfo `obsTick` | 3 min / 12.8 h | 120 ticks (6 h) | sweep only | - -`obsTick` is the sharp case: `maintainNodeInfoCacheLocked()` clearing `hasObserved` is the -_sole_ guarantee the 6 h serve gate never reads an aliased stamp. That makes the sweep a -compile-time invariant - guarded by `TMM_HAS_NODEINFO_CACHE` **alone** (never -`TRAFFIC_MANAGEMENT_CACHE_SIZE`, which a variant may zero independently), mirroring `purgeAll()`: -a build that has the cache always has its sweep. - -The stores these clocks stamp, and the warm tier's contrasting absolute timestamps, are described -in [node_info_stores.md](node_info_stores.md). - ---- - -## Configuration - -All tunables live under `moduleConfig.traffic_management`; the whole module is gated by the -`has_traffic_management` presence flag, and each per-feature section above lists its own -field(s) and default. Two related sets of knobs are **firmware constants, not config**: the -role-based position caps `default_traffic_mgmt_tracker_position_min_interval_secs` (1 h) and -`default_traffic_mgmt_lost_and_found_position_min_interval_secs` (15 min), and the direct-serve -throttle windows (the `kDirectResponse*Ms` constants). - -## See also - -- [node_info_stores.md](node_info_stores.md) - the NodeDB hot store, warm tier, TMM NodeInfo - payload cache, and unified cache that the direct-serve path reads from, plus their trust, - provenance, and anti-entropy model. diff --git a/src/modules/TrafficManagementModule.cpp b/src/modules/TrafficManagementModule.cpp index b4c5fae98e5..0fdd8c72226 100644 --- a/src/modules/TrafficManagementModule.cpp +++ b/src/modules/TrafficManagementModule.cpp @@ -586,7 +586,8 @@ void TrafficManagementModule::reconcileNodeInfoFromNodeDBLocked() // Membership refresh (this hourly pass owns it): clear every isMember bit, then re-mark from // both NodeDB tiers. Runs AFTER seeding so the upsert still sees last pass's bits (spareMembers). - // Cost/lag rationale in docs/node_info_stores.md "Consistency with NodeDB (anti-entropy)". + // Cost/lag rationale in https://meshtastic.org/docs/development/reference/node-info-stores "Consistency with NodeDB + // (anti-entropy)". for (uint16_t i = 0; i < nodeInfoTargetEntries(); i++) nodeInfoPayload[i].isMember = false; for (size_t i = 0; i < nodeDB->getNumMeshNodes(); i++) { @@ -729,7 +730,8 @@ bool TrafficManagementModule::copyPublicKey(NodeNum node, uint8_t out[32], bool { // Same enable gate as the write-through hooks and maintenance: a disabled module stops // updating and sweeping the cache, so its frozen contents must not keep feeding PKI key - // resolution either. Enforces the "superset only while enabled" corollary (node_info_stores.md). + // resolution either. Enforces the "superset only while enabled" corollary + // (https://meshtastic.org/docs/development/reference/node-info-stores). if (!moduleConfig.has_traffic_management) return false; if (!nodeInfoPayload || node == 0 || !out) @@ -1514,7 +1516,8 @@ bool TrafficManagementModule::shouldRespondToNodeInfo(const meshtastic_MeshPacke // Throttle the spoofed reply (per requester + per target + 1 s global floor; checked here so a // request declined above never spends the budget). false forwards the request instead of consuming - // it. Rationale in docs/traffic_management_module.md "Throttling direct responses". + // it. Rationale in https://meshtastic.org/docs/development/reference/traffic-management-internals "Throttling direct + // responses". if (!directResponseAllowed(getFrom(p), p->to, clockMs())) { TM_LOG_DEBUG("NodeInfo direct response throttled for 0x%08x; forwarding request", getFrom(p)); return false; diff --git a/src/modules/TrafficManagementModule.h b/src/modules/TrafficManagementModule.h index e01cdefdb96..631673d7577 100644 --- a/src/modules/TrafficManagementModule.h +++ b/src/modules/TrafficManagementModule.h @@ -33,7 +33,8 @@ /// Packet inspection and traffic shaping: position dedup, per-node rate limiting, unknown-packet /// filtering, NodeInfo direct response, and the next-hop/role overflow caches. One flat 10-byte -/// unified cache backs all per-node features; see docs/node_info_stores.md for the store overview. +/// unified cache backs all per-node features; see https://meshtastic.org/docs/development/reference/node-info-stores for the +/// store overview. class TrafficManagementModule : public MeshModule, private concurrency::OSThread { public: @@ -144,7 +145,8 @@ class TrafficManagementModule : public MeshModule, private concurrency::OSThread private: // 10-byte packed entry, all platforms. Tick stamps are free-running modular counters with // non-zero presence sentinels; the 4-bit cached role rides the top bits of the two count - // bytes (tier-3 role fallback). Full layout and rationale: docs/node_info_stores.md. + // bytes (tier-3 role fallback). Full layout and rationale: + // https://meshtastic.org/docs/development/reference/node-info-stores. #if _meshtastic_Config_DeviceConfig_Role_MAX > 15 #warning "Device role enum max exceeds 15 - TMM 4-bit role cache (rate_count[7:6]/unknown_count[7:6]) will truncate new values" #endif @@ -347,12 +349,14 @@ class TrafficManagementModule : public MeshModule, private concurrency::OSThread /// 60 s NodeInfo-cache maintenance under cacheLock: saturate the expired obsTick stamp (wrap-safety /// for the modular clock) and run the boot/hourly reconcile. Guarded by TMM_HAS_NODEINFO_CACHE alone - /// (never the unified cache size); see docs/node_info_stores.md "Tick clocks and wrap safety". + /// (never the unified cache size); see https://meshtastic.org/docs/development/reference/node-info-stores "Tick clocks and + /// wrap safety". void maintainNodeInfoCacheLocked(); /// Anti-entropy under cacheLock: upsert hot-store + warm-tier records this cache lacks (never sets /// hasObserved - seeding is knowledge, not observation), and refresh isMember from both NodeDB - /// tiers. Cost/lag: docs/node_info_stores.md "Consistency with NodeDB (anti-entropy)". + /// tiers. Cost/lag: https://meshtastic.org/docs/development/reference/node-info-stores "Consistency with NodeDB + /// (anti-entropy)". void reconcileNodeInfoFromNodeDBLocked(); /// Learn an observed NODEINFO frame into the cache (key hygiene + provenance rules apply). void cacheNodeInfoPacket(const meshtastic_MeshPacket &mp); @@ -368,7 +372,8 @@ class TrafficManagementModule : public MeshModule, private concurrency::OSThread // Direct-response throttles bounding the reflector risk of spoofed replies: three fixed bounds // (per requester, per target, 1 s global airtime floor) via 8-slot LRU RAM tables, wrap-safe and - // PSRAM-agnostic. Design & rationale: docs/traffic_management_module.md "Throttling direct responses". + // PSRAM-agnostic. Design & rationale: https://meshtastic.org/docs/development/reference/traffic-management-internals + // "Throttling direct responses". static constexpr uint32_t kDirectResponsePerRequesterMs = 60'000UL; static constexpr uint32_t kDirectResponsePerTargetMs = 60'000UL; static constexpr uint32_t kDirectResponseGlobalMs = 1'000UL; diff --git a/test/test_nexthop_routing/test_main.cpp b/test/test_nexthop_routing/test_main.cpp index 45dfa8c4aca..60afed5ce0b 100644 --- a/test/test_nexthop_routing/test_main.cpp +++ b/test/test_nexthop_routing/test_main.cpp @@ -1,4 +1,4 @@ -// Unit tests for NextHop direct-message reliability mitigations (see docs/nexthop-routing-reliability.md): +// Unit tests for NextHop direct-message reliability mitigations (landed in meshtastic/firmware#10745): // M1 - NodeDB::resolveLastByte / resolveUniqueLastByte (ambiguity-aware last-byte resolution) // M2 - NextHopRouter::getNextHop strict-neighbor gate + Router::shouldDecrementHopLimit favorite check // M3 - NextHopRouter route-health freshness / failure decay