From 09fe1890cce061647f3a238c9ba59ff257e2fd25 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:26:44 +0100 Subject: [PATCH 01/10] docs(adr): canonical MeshCore channels and per-feeder slot links Amend ADR-0002 and text-message-channels guide for #379: logical MessageChannel identity by name/hashtag, device index on link table only. --- .../meshcore/text-message-channels.md | 22 ++++++++----- .../adr/0002-mc-channel-modelling.md | 31 ++++++++++--------- 2 files changed, 31 insertions(+), 22 deletions(-) diff --git a/docs/features/meshcore/text-message-channels.md b/docs/features/meshcore/text-message-channels.md index ef75a3bf..0880586f 100644 --- a/docs/features/meshcore/text-message-channels.md +++ b/docs/features/meshcore/text-message-channels.md @@ -229,12 +229,20 @@ The bot does **not** pull API channel config and push to device on startup. If t | `name` | Operator-facing label (UI/admin); not on wire | | `constellation` | FK | | `protocol` | `MESHCORE` for MC rows | -| `mc_channel_idx` | Device slot `0..63`; unique per `(constellation, protocol)` | | `mc_channel_type` | `PUBLIC` / `HASHTAG` | -| `mc_hashtag` | Hashtag string when type is `HASHTAG` (no leading `#` in DB) | +| `mc_hashtag` | Hashtag string when type is `HASHTAG` (no leading `#` in DB); unique per constellation for HASHTAG | +| *(no `mc_channel_idx`)* | Logical identity is name/hashtag, not device slot ([#379](https://github.com/pskillen/meshflow-api/issues/379)) | Meshtastic channels use PSK-backed slots on the managed node (`meshtastic_channel_0..7`). MeshCore does **not** use those slots. +### `ManagedNodeMcChannelLink` (per-feeder device slot) + +| Field | Role | +|-------|------| +| `managed_node` | Feeder | +| `message_channel` | Canonical `MessageChannel` for this slot’s logical channel | +| `mc_channel_idx` | Device slot `0..63` (unique per managed node) | + ### `ManagedNode` (feeder) | Field | Role | @@ -242,7 +250,7 @@ Meshtastic channels use PSK-backed slots on the managed node (`meshtastic_channe | `protocol` | `MESHCORE` for MC feeders | | `mc_pubkey` | Full 64-hex feeder identity ([#295](https://github.com/pskillen/meshflow-api/issues/295)) | | `meshtastic_channel_0..7` | Meshtastic only | -| `mc_channels` | M2M mirror of device slots | +| `mc_channels` | M2M to canonical channels **through** `ManagedNodeMcChannelLink` | | `mc_channels_synced_at` | Last successful bot `mc-channel-sync` | Authentication: Node API key via `NodeAuth`; feeder resolved by URL **`{feeder_pubkey_prefix}`** + optional **`X-MeshCore-Feeder-Pubkey`** header ([feeder-bootstrap.md](feeder-bootstrap.md)). @@ -298,10 +306,10 @@ Flow for text packets: ### `resolve_mc_channel` 1. Clamp `channel_idx` to `0..63`. -2. Prefer `observer.mc_channels.filter(mc_channel_idx=idx).first()` (feeder mirror). -3. If missing, auto-create placeholder `MessageChannel`, attach to `observer.mc_channels`. +2. Look up `ManagedNodeMcChannelLink` for `(observer, channel_idx)` → canonical `MessageChannel`. +3. If missing, auto-create placeholder canonical + link (overwritten on next device sync). -Heard traffic links to configured channels without names on the wire. +Heard traffic links to **logical** channels without names on the wire; multiple feeders with the same hashtag at different indices share one canonical row after sync. ### Signals @@ -320,7 +328,7 @@ Identity receiver **skips** channel text (no `from_pubkey` / prefix). Contact te - **Caller:** meshflow-bot with Node API key, after reading the device. - **Body:** full channel list (`mc_channel_idx`, `name`, `mc_channel_type`, `mc_hashtag`); optional `synced_at`. -- **Effect:** upsert `MessageChannel` rows; set `ManagedNode.mc_channels` to match snapshot ([`reconcile_mc_channels`](../../../Meshflow/meshcore_packets/services/channel_sync.py)). +- **Effect:** upsert canonical `MessageChannel` rows by logical identity; set `ManagedNodeMcChannelLink` rows to match snapshot ([`reconcile_mc_channels`](../../../Meshflow/meshcore_packets/services/channel_sync.py)). - **Read:** managed node API returns nested `mc_channels` for UI. ### Secondary: push UI / admin intent → device diff --git a/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md b/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md index b9ab0f13..8c6fa200 100644 --- a/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md +++ b/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md @@ -1,8 +1,8 @@ # ADR-0002 — MeshCore channel modelling -**Status:** Proposed +**Status:** Accepted (amended 2026-06-01) **Date:** 2026-05-12 -**Tracking:** [meshflow-api#276](https://github.com/pskillen/meshflow-api/issues/276) +**Tracking:** [meshflow-api#276](https://github.com/pskillen/meshflow-api/issues/276), [meshflow-api#379](https://github.com/pskillen/meshflow-api/issues/379) ## Context @@ -24,18 +24,19 @@ We still need a way to: ## Decision 1. **Add `MessageChannel.protocol`** (same enum as `ObservedNode.protocol`). Channels are single-protocol; an MT channel and an MC channel are distinct rows even if their names happen to match. -2. **MC channel identifiers on `MessageChannel`:** - - `mc_channel_idx` — `PositiveSmallIntegerField`, nullable. Zero-based index as seen in `channel_message.channel_idx`. - - `name` — existing operator-set string (no change). For MC, populated out of band (admin UI / config) since the wire carries no name. +2. **Canonical `MessageChannel` rows (constellation-scoped logical identity):** + - `name`, `mc_channel_type` (`PUBLIC` / `HASHTAG`), `mc_hashtag` (when HASHTAG). + - **No `mc_channel_idx` on `MessageChannel`.** Device slot index is per-feeder, not global. + - Uniqueness: `(constellation, protocol, mc_hashtag)` for HASHTAG rows; `(constellation, protocol, name)` for PUBLIC rows (normalized in services). 3. **Defer `mc_channel_hash`.** Revisit if/when a firmware revision starts emitting one on the wire. No column added now. -4. **Replace the fixed `channel_0..channel_7` slots, for MC managed nodes, with an M2M:** - - `ManagedNode.mc_channels` — `ManyToManyField(MessageChannel, related_name='managed_nodes_mc', blank=True)`. - - The existing `channel_0..channel_7` FKs stay MT-only and untouched. Single-protocol managed nodes only use one or the other side; the model does not need a CHECK constraint here because the existing `ManagedNode.protocol` (Phase 1) already determines which side is populated. -5. **Channel resolution on `PacketObservation` stays an FK to `MessageChannel`.** For MC packets, the serializer resolves the channel as: - - look up the observer `ManagedNode`, - - filter `observer.mc_channels.filter(mc_channel_idx=)`, - - if not found, auto-create a `MessageChannel` row with `protocol=MESHCORE`, `mc_channel_idx=`, `name=f"MC channel {idx}"`, and attach it to the observer's `mc_channels`. An operator can rename it later. -6. **Uniqueness:** `UniqueConstraint(fields=['protocol', 'mc_channel_idx', 'constellation'])` where applicable (matches the existing constellation-scoped uniqueness pattern on MT channels). MC channels are scoped to a constellation just like MT ones — channel index 0 in Scotland is not the same row as channel index 0 in another constellation. +4. **Per-feeder slot mapping via `ManagedNodeMcChannelLink` (M2M through table):** + - `managed_node`, `message_channel` (canonical), `mc_channel_idx` (`0..63`, unique per managed node). + - `ManagedNode.mc_channels` — `ManyToManyField(MessageChannel, through='ManagedNodeMcChannelLink', ...)`. + - Meshtastic `meshtastic_channel_0..7` FKs stay MT-only and untouched. +5. **Channel resolution on ingest** (`resolve_mc_channel(observer, channel_idx)`): + - Look up `ManagedNodeMcChannelLink` for `(observer, channel_idx)` → canonical `MessageChannel`. + - If missing (heard before sync), create placeholder canonical + link; overwritten when device sync supplies real metadata. +6. **`TextMessage.channel` and packet FKs point at the canonical row** so the same logical channel (`#test`) is one row even when two feeders use different device indices. ## Consequences @@ -48,9 +49,9 @@ We still need a way to: ## Supplement (2026-05-20) — device as source of truth for operator metadata -ADR §5–6 still govern **ingest resolution** (`mc_channel_idx` → `MessageChannel` via feeder `mc_channels`). For **name**, **type** (public/hashtag), and **hashtag string**, Phase 2 ([#297](https://github.com/pskillen/meshflow-api/issues/297)) treats the **MeshCore companion channel table** as authoritative: +ADR §5–6 still govern **ingest resolution** (`channel_idx` on wire → canonical `MessageChannel` via `ManagedNodeMcChannelLink`). For **name**, **type** (public/hashtag), and **hashtag string**, Phase 2 ([#297](https://github.com/pskillen/meshflow-api/issues/297)) treats the **MeshCore companion channel table** as authoritative: -- On bot connect, the bot uploads a full device snapshot; the API **reconciles** `MessageChannel` rows and `ManagedNode.mc_channels` (see [`text-message-channels.md`](../../meshcore/text-message-channels.md)). +- On bot connect, the bot uploads a full device snapshot; the API **reconciles** canonical `MessageChannel` rows and per-feeder links (see [`text-message-channels.md`](../../meshcore/text-message-channels.md)). - UI edits push to the device (WebSocket), then the bot re-syncs; the API does not rely on API-only CRUD as the long-term source of names. - Auto-created `"MC channel N"` placeholders at ingest (before first sync) remain; they are overwritten when sync supplies device metadata. From a3958798c09727b86c15cf068ab160f1c4f87f72 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:26:47 +0100 Subject: [PATCH 02/10] feat(constellations): add ManagedNodeMcChannelLink and migrate MC channel identity Introduce per-feeder slot mapping; merge duplicate canonical channels on migrate; drop mc_channel_idx from MessageChannel. Closes data model for #379. --- .../0012_remove_mc_idx_constraint.py | 17 ++++ ...13_mc_canonical_channels_backfill_links.py | 13 +++ .../0014_mc_canonical_channels_finalize.py | 96 +++++++++++++++++++ Meshflow/constellations/models.py | 23 +++-- .../migrations/0051_mc_canonical_channels.py | 96 +++++++++++++++++++ Meshflow/nodes/models.py | 32 +++++++ 6 files changed, 269 insertions(+), 8 deletions(-) create mode 100644 Meshflow/constellations/migrations/0012_remove_mc_idx_constraint.py create mode 100644 Meshflow/constellations/migrations/0013_mc_canonical_channels_backfill_links.py create mode 100644 Meshflow/constellations/migrations/0014_mc_canonical_channels_finalize.py create mode 100644 Meshflow/nodes/migrations/0051_mc_canonical_channels.py diff --git a/Meshflow/constellations/migrations/0012_remove_mc_idx_constraint.py b/Meshflow/constellations/migrations/0012_remove_mc_idx_constraint.py new file mode 100644 index 00000000..7ebe2b69 --- /dev/null +++ b/Meshflow/constellations/migrations/0012_remove_mc_idx_constraint.py @@ -0,0 +1,17 @@ +# Manual split for #379 — drop index uniqueness before link backfill + +from django.db import migrations + + +class Migration(migrations.Migration): + + dependencies = [ + ("constellations", "0011_remove_constellationusermembership"), + ] + + operations = [ + migrations.RemoveConstraint( + model_name="messagechannel", + name="messagechannel_mc_idx_constellation_unique", + ), + ] diff --git a/Meshflow/constellations/migrations/0013_mc_canonical_channels_backfill_links.py b/Meshflow/constellations/migrations/0013_mc_canonical_channels_backfill_links.py new file mode 100644 index 00000000..48261aaf --- /dev/null +++ b/Meshflow/constellations/migrations/0013_mc_canonical_channels_backfill_links.py @@ -0,0 +1,13 @@ +# Placeholder dependency anchor between link backfill and finalize + +from django.db import migrations + + +class Migration(migrations.Migration): + + dependencies = [ + ("constellations", "0012_remove_mc_idx_constraint"), + ("nodes", "0051_mc_canonical_channels"), + ] + + operations = [] diff --git a/Meshflow/constellations/migrations/0014_mc_canonical_channels_finalize.py b/Meshflow/constellations/migrations/0014_mc_canonical_channels_finalize.py new file mode 100644 index 00000000..b9061df5 --- /dev/null +++ b/Meshflow/constellations/migrations/0014_mc_canonical_channels_finalize.py @@ -0,0 +1,96 @@ +# Merge duplicate MC channels and drop per-row device index from MessageChannel + +from django.db import migrations, models + + +MC_PROTOCOL = 2 +MC_PUBLIC = 1 +MC_HASHTAG = 2 + + +def _logical_key(channel): + if channel.mc_channel_type == MC_HASHTAG and channel.mc_hashtag: + tag = str(channel.mc_hashtag).strip().lstrip("#").lower() + if tag: + return ("hashtag", tag) + name = str(channel.name or "").strip().lower() + return ("public", name) + + +def _repoint_channel_fks(apps, old_id, new_id): + if old_id == new_id: + return + TextMessage = apps.get_model("text_messages", "TextMessage") + TextMessage.objects.filter(channel_id=old_id).update(channel_id=new_id) + + try: + MeshCoreTextPacket = apps.get_model("meshcore_packets", "MeshCoreTextPacket") + MeshCoreTextPacket.objects.filter(channel_id=old_id).update(channel_id=new_id) + except LookupError: + pass + + try: + MeshCorePacketObservation = apps.get_model("meshcore_packets", "MeshCorePacketObservation") + MeshCorePacketObservation.objects.filter(channel_id=old_id).update(channel_id=new_id) + except LookupError: + pass + + Link = apps.get_model("nodes", "ManagedNodeMcChannelLink") + Link.objects.filter(message_channel_id=old_id).update(message_channel_id=new_id) + + +def merge_duplicate_mc_channels(apps, schema_editor): + MessageChannel = apps.get_model("constellations", "MessageChannel") + channels = list( + MessageChannel.objects.filter(protocol=MC_PROTOCOL).exclude(mc_channel_idx__isnull=True) + ) + groups = {} + for ch in channels: + key = (ch.constellation_id, _logical_key(ch)) + groups.setdefault(key, []).append(ch) + + for _key, group in groups.items(): + if len(group) < 2: + continue + group.sort(key=lambda c: c.id) + survivor = group[0] + for dup in group[1:]: + _repoint_channel_fks(apps, dup.id, survivor.id) + MessageChannel.objects.filter(id=dup.id).delete() + + +class Migration(migrations.Migration): + + dependencies = [ + ("constellations", "0013_mc_canonical_channels_backfill_links"), + ("text_messages", "0008_textmessage_mc_provenance"), + ("meshcore_packets", "0004_observation_path_hash_size_mode"), + ] + + operations = [ + migrations.RunPython(merge_duplicate_mc_channels, migrations.RunPython.noop), + migrations.RemoveField( + model_name="messagechannel", + name="mc_channel_idx", + ), + migrations.AddConstraint( + model_name="messagechannel", + constraint=models.UniqueConstraint( + condition=models.Q( + ("mc_channel_type", MC_HASHTAG), + ("mc_hashtag__isnull", False), + ("protocol", MC_PROTOCOL), + ), + fields=("constellation", "protocol", "mc_hashtag"), + name="messagechannel_mc_hashtag_constellation_unique", + ), + ), + migrations.AddConstraint( + model_name="messagechannel", + constraint=models.UniqueConstraint( + condition=models.Q(("mc_channel_type", MC_PUBLIC), ("protocol", MC_PROTOCOL)), + fields=("constellation", "protocol", "name"), + name="messagechannel_mc_public_name_constellation_unique", + ), + ), + ] diff --git a/Meshflow/constellations/models.py b/Meshflow/constellations/models.py index a8f9f6ff..466d1908 100644 --- a/Meshflow/constellations/models.py +++ b/Meshflow/constellations/models.py @@ -47,11 +47,6 @@ class MessageChannel(models.Model): db_index=True, help_text=_("Mesh protocol for this channel row."), ) - mc_channel_idx = models.PositiveSmallIntegerField( - null=True, - blank=True, - help_text=_("MeshCore channel index when protocol is MeshCore; null for Meshtastic."), - ) mc_channel_type = models.PositiveSmallIntegerField( choices=MeshCoreChannelType.choices, null=True, @@ -70,9 +65,21 @@ class Meta: verbose_name_plural = _("Message channels") constraints = [ models.UniqueConstraint( - fields=["constellation", "protocol", "mc_channel_idx"], - condition=models.Q(protocol=Protocol.MESHCORE, mc_channel_idx__isnull=False), - name="messagechannel_mc_idx_constellation_unique", + fields=["constellation", "protocol", "mc_hashtag"], + condition=models.Q( + protocol=Protocol.MESHCORE, + mc_channel_type=MeshCoreChannelType.HASHTAG, + mc_hashtag__isnull=False, + ), + name="messagechannel_mc_hashtag_constellation_unique", + ), + models.UniqueConstraint( + fields=["constellation", "protocol", "name"], + condition=models.Q( + protocol=Protocol.MESHCORE, + mc_channel_type=MeshCoreChannelType.PUBLIC, + ), + name="messagechannel_mc_public_name_constellation_unique", ), ] diff --git a/Meshflow/nodes/migrations/0051_mc_canonical_channels.py b/Meshflow/nodes/migrations/0051_mc_canonical_channels.py new file mode 100644 index 00000000..df66e620 --- /dev/null +++ b/Meshflow/nodes/migrations/0051_mc_canonical_channels.py @@ -0,0 +1,96 @@ +# Create feeder slot links and switch mc_channels M2M to explicit through table + +import django.db.models.deletion +from django.db import migrations, models + + +def populate_mc_channel_links(apps, schema_editor): + MessageChannel = apps.get_model("constellations", "MessageChannel") + Link = apps.get_model("nodes", "ManagedNodeMcChannelLink") + Through = apps.get_model("nodes", "ManagedNode_mc_channels") + + for row in Through.objects.all().iterator(): + try: + channel = MessageChannel.objects.get(id=row.messagechannel_id) + except MessageChannel.DoesNotExist: + continue + idx = channel.mc_channel_idx + if idx is None: + continue + Link.objects.update_or_create( + managed_node_id=row.managednode_id, + mc_channel_idx=idx, + defaults={"message_channel_id": channel.id}, + ) + + +def remove_legacy_mc_channels_m2m(apps, schema_editor): + """Drop implicit M2M table before re-adding mc_channels with through=.""" + schema_editor.execute("DROP TABLE IF EXISTS nodes_managednode_mc_channels") + + +class Migration(migrations.Migration): + + dependencies = [ + ("constellations", "0012_remove_mc_idx_constraint"), + ("nodes", "0050_managednode_protocol_identity"), + ] + + operations = [ + migrations.CreateModel( + name="ManagedNodeMcChannelLink", + fields=[ + ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")), + ( + "mc_channel_idx", + models.PositiveSmallIntegerField( + help_text="MeshCore device channel index (0–63) for this feeder." + ), + ), + ( + "managed_node", + models.ForeignKey( + on_delete=django.db.models.deletion.CASCADE, + related_name="mc_channel_links", + to="nodes.managednode", + ), + ), + ( + "message_channel", + models.ForeignKey( + on_delete=django.db.models.deletion.CASCADE, + related_name="feeder_links", + to="constellations.messagechannel", + ), + ), + ], + options={ + "verbose_name": "MeshCore feeder channel link", + "verbose_name_plural": "MeshCore feeder channel links", + }, + ), + migrations.RunPython(populate_mc_channel_links, migrations.RunPython.noop), + migrations.RemoveField( + model_name="managednode", + name="mc_channels", + ), + migrations.RunPython(remove_legacy_mc_channels_m2m, migrations.RunPython.noop), + migrations.AddField( + model_name="managednode", + name="mc_channels", + field=models.ManyToManyField( + blank=True, + help_text="MeshCore channels mirrored from the feeder device (protocol=MeshCore only).", + related_name="managed_nodes_mc", + through="nodes.ManagedNodeMcChannelLink", + to="constellations.messagechannel", + ), + ), + migrations.AddConstraint( + model_name="managednodemcchannellink", + constraint=models.UniqueConstraint( + fields=("managed_node", "mc_channel_idx"), + name="managednode_mc_channel_idx_unique", + ), + ), + ] diff --git a/Meshflow/nodes/models.py b/Meshflow/nodes/models.py index c2cab0e9..4743a037 100644 --- a/Meshflow/nodes/models.py +++ b/Meshflow/nodes/models.py @@ -154,6 +154,7 @@ class ManagedNode(models.Model): mc_channels = models.ManyToManyField( "constellations.MessageChannel", + through="ManagedNodeMcChannelLink", related_name="managed_nodes_mc", blank=True, help_text=_("MeshCore channels mirrored from the feeder device (protocol=MeshCore only)."), @@ -258,6 +259,37 @@ def get_channel(self, channel_idx: int) -> MessageChannel: return getattr(self, f"meshtastic_channel_{channel_idx}") +class ManagedNodeMcChannelLink(models.Model): + """Maps a MeshCore feeder device slot index to a canonical MessageChannel.""" + + managed_node = models.ForeignKey( + ManagedNode, + on_delete=models.CASCADE, + related_name="mc_channel_links", + ) + message_channel = models.ForeignKey( + MessageChannel, + on_delete=models.CASCADE, + related_name="feeder_links", + ) + mc_channel_idx = models.PositiveSmallIntegerField( + help_text=_("MeshCore device channel index (0–63) for this feeder."), + ) + + class Meta: + verbose_name = _("MeshCore feeder channel link") + verbose_name_plural = _("MeshCore feeder channel links") + constraints = [ + models.UniqueConstraint( + fields=["managed_node", "mc_channel_idx"], + name="managednode_mc_channel_idx_unique", + ), + ] + + def __str__(self): + return f"{self.managed_node_id} slot {self.mc_channel_idx} → {self.message_channel_id}" + + class ManagedNodeStatus(models.Model): """Denormalized feeder/API ingestion status for a managed node (not RF mesh liveness).""" From 027772ad80a81426bc3833afd427db06e5121df0 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:26:47 +0100 Subject: [PATCH 03/10] fix(meshcore_packets): reconcile and resolve canonical MC channels Resolve ingest and device sync via ManagedNodeMcChannelLink; upsert MessageChannel by logical hashtag or public name. --- Meshflow/common/mc_channel_labels.py | 47 +++++++++-- Meshflow/meshcore_packets/services/channel.py | 43 +++++----- .../services/channel_apply.py | 11 ++- .../services/channel_identity.py | 81 +++++++++++++++++++ .../meshcore_packets/services/channel_sync.py | 59 ++++---------- 5 files changed, 170 insertions(+), 71 deletions(-) create mode 100644 Meshflow/meshcore_packets/services/channel_identity.py diff --git a/Meshflow/common/mc_channel_labels.py b/Meshflow/common/mc_channel_labels.py index ff63b808..456e5bc7 100644 --- a/Meshflow/common/mc_channel_labels.py +++ b/Meshflow/common/mc_channel_labels.py @@ -3,6 +3,7 @@ from __future__ import annotations from constellations.models import MeshCoreChannelType, MessageChannel +from nodes.models import ManagedNode, ManagedNodeMcChannelLink def mc_channel_admin_label(channel: MessageChannel) -> str: @@ -14,24 +15,42 @@ def mc_channel_admin_label(channel: MessageChannel) -> str: name = (channel.name or "").strip() if name: return name - if channel.mc_channel_idx is not None: - return f"slot {channel.mc_channel_idx}" return str(channel.pk) +def mc_channel_display_label(channel: MessageChannel) -> str: + """Operator-facing label (Messages UI, constellation channel lists).""" + return mc_channel_admin_label(channel) + + def mc_channel_type_name(channel: MessageChannel) -> str: if channel.mc_channel_type is None: return "—" return MeshCoreChannelType(channel.mc_channel_type).name -def message_channel_to_apply_entry(channel: MessageChannel) -> dict: - """Build one apply_mc_channel_config entry from a MessageChannel row.""" +def message_channel_to_apply_entry( + channel: MessageChannel, + *, + managed_node: ManagedNode | None = None, + mc_channel_idx: int | None = None, +) -> dict: + """Build one apply_mc_channel_config entry from a canonical channel and feeder slot.""" + if mc_channel_idx is None and managed_node is not None: + link = ManagedNodeMcChannelLink.objects.filter( + managed_node=managed_node, + message_channel=channel, + ).first() + if link is not None: + mc_channel_idx = link.mc_channel_idx + if mc_channel_idx is None: + raise ValueError("mc_channel_idx is required to apply channel config to a feeder device") + ch_type = mc_channel_type_name(channel) if ch_type == "—": ch_type = "PUBLIC" entry = { - "mc_channel_idx": channel.mc_channel_idx, + "mc_channel_idx": mc_channel_idx, "name": channel.name, "mc_channel_type": ch_type, } @@ -43,8 +62,22 @@ def message_channel_to_apply_entry(channel: MessageChannel) -> dict: return entry +def managed_node_mc_channel_links(managed_node: ManagedNode): + """Feeder slot links with canonical channels, ordered by device index.""" + from common.protocol import Protocol + + return ( + managed_node.mc_channel_links.filter(message_channel__protocol=Protocol.MESHCORE) + .select_related("message_channel") + .order_by("mc_channel_idx") + ) + + def managed_node_mc_channels_queryset(managed_node): - """MC channel rows linked on a MeshCore feeder (device mirror).""" + """Canonical MC channel rows for a MeshCore feeder (legacy queryset helper).""" from common.protocol import Protocol - return managed_node.mc_channels.filter(protocol=Protocol.MESHCORE).order_by("mc_channel_idx") + return MessageChannel.objects.filter( + feeder_links__managed_node=managed_node, + protocol=Protocol.MESHCORE, + ).order_by("feeder_links__mc_channel_idx") diff --git a/Meshflow/meshcore_packets/services/channel.py b/Meshflow/meshcore_packets/services/channel.py index 2264ad6e..dfa62d60 100644 --- a/Meshflow/meshcore_packets/services/channel.py +++ b/Meshflow/meshcore_packets/services/channel.py @@ -1,34 +1,37 @@ -"""Resolve MeshCore MessageChannel rows per ADR-0002.""" +"""Resolve MeshCore MessageChannel rows per ADR-0002 (canonical + per-feeder link).""" -from common.protocol import Protocol -from constellations.models import MeshCoreChannelType, MessageChannel -from nodes.models import ManagedNode - -MC_CHANNEL_IDX_MAX = 63 +from constellations.models import MessageChannel +from meshcore_packets.services.channel_identity import ( + MC_CHANNEL_IDX_MAX, + placeholder_canonical_mc_channel, +) +from nodes.models import ManagedNode, ManagedNodeMcChannelLink def resolve_mc_channel(observer: ManagedNode, channel_idx: int | None) -> MessageChannel | None: - """Map (observer constellation, mc_channel_idx) to a MessageChannel; prefer feeder M2M.""" + """Map (observer, wire channel_idx) to a canonical MessageChannel via feeder slot link.""" if channel_idx is None: return None channel_idx = int(channel_idx) if channel_idx < 0 or channel_idx > MC_CHANNEL_IDX_MAX: return None - existing = observer.mc_channels.filter(mc_channel_idx=channel_idx).first() - if existing: - return existing + link = ( + ManagedNodeMcChannelLink.objects.filter( + managed_node=observer, + mc_channel_idx=channel_idx, + ) + .select_related("message_channel") + .first() + ) + if link: + return link.message_channel constellation = observer.constellation - channel, created = MessageChannel.objects.get_or_create( - constellation=constellation, - protocol=Protocol.MESHCORE, + canonical = placeholder_canonical_mc_channel(constellation, channel_idx) + ManagedNodeMcChannelLink.objects.get_or_create( + managed_node=observer, mc_channel_idx=channel_idx, - defaults={ - "name": f"MC channel {channel_idx}", - "mc_channel_type": MeshCoreChannelType.PUBLIC, - }, + defaults={"message_channel": canonical}, ) - if created: - observer.mc_channels.add(channel) - return channel + return canonical diff --git a/Meshflow/meshcore_packets/services/channel_apply.py b/Meshflow/meshcore_packets/services/channel_apply.py index bf086439..06ea7166 100644 --- a/Meshflow/meshcore_packets/services/channel_apply.py +++ b/Meshflow/meshcore_packets/services/channel_apply.py @@ -12,7 +12,7 @@ dispatch_node_command, feeder_ws_group_has_subscribers, ) -from common.mc_channel_labels import managed_node_mc_channels_queryset, message_channel_to_apply_entry +from common.mc_channel_labels import managed_node_mc_channel_links, message_channel_to_apply_entry from common.protocol import Protocol from common.ws_groups import managed_node_ws_group from nodes.models import ManagedNode @@ -24,7 +24,14 @@ def build_apply_channels_for_managed_node(managed_node: ManagedNode) -> list[dic """Snapshot entries for apply_mc_channel_config from the feeder mirror.""" if managed_node.protocol != Protocol.MESHCORE: return [] - return [message_channel_to_apply_entry(ch) for ch in managed_node_mc_channels_queryset(managed_node)] + return [ + message_channel_to_apply_entry( + link.message_channel, + managed_node=managed_node, + mc_channel_idx=link.mc_channel_idx, + ) + for link in managed_node_mc_channel_links(managed_node) + ] def dispatch_mc_channel_apply(managed_node: ManagedNode, channels: list[dict]) -> str: diff --git a/Meshflow/meshcore_packets/services/channel_identity.py b/Meshflow/meshcore_packets/services/channel_identity.py new file mode 100644 index 00000000..aee4fc4c --- /dev/null +++ b/Meshflow/meshcore_packets/services/channel_identity.py @@ -0,0 +1,81 @@ +"""Canonical MeshCore MessageChannel identity (logical name/hashtag, not device index).""" + +from __future__ import annotations + +from common.protocol import Protocol +from constellations.models import MeshCoreChannelType, MessageChannel + +MC_CHANNEL_IDX_MAX = 63 + + +def normalize_mc_hashtag(value: str | None) -> str | None: + if value is None: + return None + tag = str(value).strip().lstrip("#").lower() + return tag[:64] if tag else None + + +def normalize_mc_public_name(value: str | None) -> str: + name = str(value or "Public").strip() + return name[:100] if name else "Public" + + +def _parse_channel_type(value: str | int | None) -> int: + if value is None: + return MeshCoreChannelType.PUBLIC + if isinstance(value, int): + if value in MeshCoreChannelType.values: + return value + return MeshCoreChannelType.PUBLIC + key = str(value).strip().upper() + if key in MeshCoreChannelType.names: + return MeshCoreChannelType[key] + return MeshCoreChannelType.PUBLIC + + +def upsert_canonical_mc_channel(constellation, entry: dict) -> MessageChannel: + """ + Resolve or create a constellation-scoped canonical MessageChannel from a device snapshot entry. + + HASHTAG rows are keyed by normalized mc_hashtag; PUBLIC rows by normalized name. + """ + ch_type = _parse_channel_type(entry.get("mc_channel_type")) + + if ch_type == MeshCoreChannelType.HASHTAG: + hashtag = normalize_mc_hashtag(entry.get("mc_hashtag") or entry.get("name")) + if not hashtag: + raise ValueError("Hashtag channels require a non-empty hashtag.") + display_name = str(entry.get("name") or hashtag).strip().lstrip("#")[:100] or hashtag + channel, _created = MessageChannel.objects.update_or_create( + constellation=constellation, + protocol=Protocol.MESHCORE, + mc_channel_type=MeshCoreChannelType.HASHTAG, + mc_hashtag=hashtag, + defaults={ + "name": display_name, + }, + ) + return channel + + name = normalize_mc_public_name(entry.get("name")) + channel, _created = MessageChannel.objects.update_or_create( + constellation=constellation, + protocol=Protocol.MESHCORE, + mc_channel_type=MeshCoreChannelType.PUBLIC, + name=name, + defaults={ + "mc_hashtag": None, + }, + ) + return channel + + +def placeholder_canonical_mc_channel(constellation, channel_idx: int) -> MessageChannel: + """Placeholder until device sync supplies real metadata.""" + return MessageChannel.objects.get_or_create( + constellation=constellation, + protocol=Protocol.MESHCORE, + mc_channel_type=MeshCoreChannelType.PUBLIC, + name=f"MC channel {channel_idx}", + defaults={"mc_hashtag": None}, + )[0] diff --git a/Meshflow/meshcore_packets/services/channel_sync.py b/Meshflow/meshcore_packets/services/channel_sync.py index 370a6bf0..7758800f 100644 --- a/Meshflow/meshcore_packets/services/channel_sync.py +++ b/Meshflow/meshcore_packets/services/channel_sync.py @@ -7,23 +7,9 @@ from django.utils.dateparse import parse_datetime from common.protocol import Protocol -from constellations.models import MeshCoreChannelType, MessageChannel -from nodes.models import ManagedNode - -MC_CHANNEL_IDX_MAX = 63 - - -def _parse_channel_type(value: str | int | None) -> int | None: - if value is None: - return MeshCoreChannelType.PUBLIC - if isinstance(value, int): - if value in MeshCoreChannelType.values: - return value - return None - key = str(value).strip().upper() - if key in MeshCoreChannelType.names: - return MeshCoreChannelType[key] - return None +from constellations.models import MessageChannel +from meshcore_packets.services.channel_identity import MC_CHANNEL_IDX_MAX, upsert_canonical_mc_channel +from nodes.models import ManagedNode, ManagedNodeMcChannelLink def reconcile_mc_channels( @@ -32,20 +18,21 @@ def reconcile_mc_channels( synced_at=None, ) -> list[MessageChannel]: """ - Upsert constellation MC MessageChannel rows and set managed_node.mc_channels to match snapshot. + Upsert canonical MC MessageChannel rows and feeder slot links from a device snapshot. Each channel dict: mc_channel_idx, name, mc_channel_type (PUBLIC|HASHTAG), mc_hashtag (optional). """ if managed_node.protocol != Protocol.MESHCORE: raise ValueError("mc-channel-sync is only valid for MeshCore managed nodes") - constellation = managed_node.constellation synced_dt = synced_at if synced_at is not None and not hasattr(synced_at, "isoformat"): synced_dt = parse_datetime(str(synced_at)) or timezone.now() elif synced_at is None: synced_dt = timezone.now() + constellation = managed_node.constellation + seen_indices: set[int] = set() attached: list[MessageChannel] = [] with transaction.atomic(): @@ -57,32 +44,20 @@ def reconcile_mc_channels( if idx < 0 or idx > MC_CHANNEL_IDX_MAX: raise ValueError(f"mc_channel_idx out of range: {idx}") - name = str(entry.get("name") or f"MC channel {idx}")[:100] - ch_type = _parse_channel_type(entry.get("mc_channel_type")) - if ch_type is None: - raise ValueError(f"invalid mc_channel_type: {entry.get('mc_channel_type')}") - - hashtag = entry.get("mc_hashtag") - if hashtag is not None: - hashtag = str(hashtag).strip().lstrip("#")[:64] or None - - if ch_type == MeshCoreChannelType.HASHTAG and not hashtag: - name_for_hash = name if name.startswith("#") else f"#{name}" - hashtag = name_for_hash.lstrip("#")[:64] - - channel, _created = MessageChannel.objects.update_or_create( - constellation=constellation, - protocol=Protocol.MESHCORE, + canonical = upsert_canonical_mc_channel(constellation, entry) + ManagedNodeMcChannelLink.objects.update_or_create( + managed_node=managed_node, mc_channel_idx=idx, - defaults={ - "name": name, - "mc_channel_type": ch_type, - "mc_hashtag": hashtag if ch_type == MeshCoreChannelType.HASHTAG else None, - }, + defaults={"message_channel": canonical}, ) - attached.append(channel) + seen_indices.add(idx) + attached.append(canonical) + + if seen_indices: + managed_node.mc_channel_links.exclude(mc_channel_idx__in=seen_indices).delete() + else: + managed_node.mc_channel_links.all().delete() - managed_node.mc_channels.set(attached) managed_node.mc_channels_synced_at = synced_dt managed_node.save(update_fields=["mc_channels_synced_at"]) From 24ece1b68e6bd603ad33db5c178e8cf049160def Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:26:49 +0100 Subject: [PATCH 04/10] feat(api): expose display_label and feeder MC channel mirror shape Update serializers, admin, OpenAPI, and sync response for canonical channels with per-feeder mc_channel_idx on link rows. --- Meshflow/constellations/admin.py | 5 ++- Meshflow/constellations/serializers.py | 8 +++-- .../meshcore_packets/serializers_channel.py | 33 +++++++++++++++++- Meshflow/meshcore_packets/views.py | 7 ++-- Meshflow/nodes/admin.py | 14 ++++---- Meshflow/nodes/serializers.py | 8 +++-- openapi.yaml | 34 +++++++++++++++---- 7 files changed, 85 insertions(+), 24 deletions(-) diff --git a/Meshflow/constellations/admin.py b/Meshflow/constellations/admin.py index 14718d28..5e177695 100644 --- a/Meshflow/constellations/admin.py +++ b/Meshflow/constellations/admin.py @@ -107,7 +107,6 @@ class MeshCoreMessageChannelAdmin(admin.ModelAdmin): """Constellation MC channel catalog (device slots). Push to radio from Managed node admin.""" list_display = ( - "mc_channel_idx", "admin_label", "mc_channel_type_display", "constellation", @@ -118,10 +117,10 @@ class MeshCoreMessageChannelAdmin(admin.ModelAdmin): "constellation", ) search_fields = ("name", "mc_hashtag", "constellation__name") - ordering = ("constellation__name", "mc_channel_idx") + ordering = ("constellation__name", "name") list_select_related = ("constellation",) fieldsets = ( - (None, {"fields": ("constellation", "mc_channel_idx")}), + (None, {"fields": ("constellation",)}), ( _("Channel"), { diff --git a/Meshflow/constellations/serializers.py b/Meshflow/constellations/serializers.py index 0164b10b..f780be79 100644 --- a/Meshflow/constellations/serializers.py +++ b/Meshflow/constellations/serializers.py @@ -1,5 +1,7 @@ from rest_framework import serializers +from common.mc_channel_labels import mc_channel_display_label +from common.protocol import Protocol from constellations.models import Constellation, MeshCoreChannelType, MessageChannel @@ -8,15 +10,17 @@ def message_channel_payload(channel: MessageChannel) -> dict: mc_type = None if channel.mc_channel_type is not None: mc_type = MeshCoreChannelType(channel.mc_channel_type).label - return { + payload = { "id": channel.id, "name": channel.name, "protocol": channel.protocol, - "mc_channel_idx": channel.mc_channel_idx, "mc_channel_type": mc_type, "mc_hashtag": channel.mc_hashtag, "constellation": channel.constellation_id, } + if channel.protocol == Protocol.MESHCORE: + payload["display_label"] = mc_channel_display_label(channel) + return payload class ConstellationSerializer(serializers.ModelSerializer): diff --git a/Meshflow/meshcore_packets/serializers_channel.py b/Meshflow/meshcore_packets/serializers_channel.py index 1c59902d..27f28672 100644 --- a/Meshflow/meshcore_packets/serializers_channel.py +++ b/Meshflow/meshcore_packets/serializers_channel.py @@ -3,6 +3,7 @@ from rest_framework import serializers from constellations.models import MeshCoreChannelType, MessageChannel +from nodes.models import ManagedNodeMcChannelLink # Wire/API strings (not gettext labels — lazy __proxy__ breaks Channels msgpack). MC_CHANNEL_TYPE_API_CHOICES = [ @@ -24,6 +25,8 @@ class McChannelSyncSerializer(serializers.Serializer): class MessageChannelMcSerializer(serializers.ModelSerializer): + """Canonical MessageChannel fields (no device index).""" + mc_channel_type = serializers.SerializerMethodField() class Meta: @@ -31,7 +34,6 @@ class Meta: fields = [ "id", "name", - "mc_channel_idx", "mc_channel_type", "mc_hashtag", ] @@ -42,6 +44,35 @@ def get_mc_channel_type(self, obj): return MeshCoreChannelType(obj.mc_channel_type).name +class FeederMcChannelMirrorSerializer(serializers.ModelSerializer): + """Feeder device mirror: canonical channel plus slot index from the link row.""" + + id = serializers.IntegerField(source="message_channel.id", read_only=True) + name = serializers.CharField(source="message_channel.name", read_only=True) + mc_channel_type = serializers.SerializerMethodField() + mc_hashtag = serializers.CharField( + source="message_channel.mc_hashtag", + read_only=True, + allow_null=True, + ) + + class Meta: + model = ManagedNodeMcChannelLink + fields = [ + "id", + "name", + "mc_channel_idx", + "mc_channel_type", + "mc_hashtag", + ] + + def get_mc_channel_type(self, obj): + ch = obj.message_channel + if ch.mc_channel_type is None: + return None + return MeshCoreChannelType(ch.mc_channel_type).name + + class McChannelApplyEntrySerializer(serializers.Serializer): mc_channel_idx = serializers.IntegerField(min_value=0, max_value=63, required=False) name = serializers.CharField(max_length=100) diff --git a/Meshflow/meshcore_packets/views.py b/Meshflow/meshcore_packets/views.py index 8b9c1301..4cb4e6cc 100644 --- a/Meshflow/meshcore_packets/views.py +++ b/Meshflow/meshcore_packets/views.py @@ -13,9 +13,9 @@ from meshcore_packets.permissions import MeshCoreFeederPermission from meshcore_packets.serializers import MeshCorePacketIngestSerializer, MeshCorePacketListSerializer from meshcore_packets.serializers_channel import ( + FeederMcChannelMirrorSerializer, McChannelApplySerializer, McChannelSyncSerializer, - MessageChannelMcSerializer, ) from meshcore_packets.services.channel_apply import apply_mc_channels_to_feeder from meshcore_packets.services.channel_sync import reconcile_mc_channels @@ -181,7 +181,10 @@ def post(self, request, feeder_pubkey_prefix, format=None): { "status": "success", "synced_at": managed_node.mc_channels_synced_at, - "mc_channels": MessageChannelMcSerializer(channels, many=True).data, + "mc_channels": FeederMcChannelMirrorSerializer( + managed_node.mc_channel_links.select_related("message_channel").order_by("mc_channel_idx"), + many=True, + ).data, }, status=status.HTTP_200_OK, ) diff --git a/Meshflow/nodes/admin.py b/Meshflow/nodes/admin.py index b219bf5a..568ff696 100644 --- a/Meshflow/nodes/admin.py +++ b/Meshflow/nodes/admin.py @@ -9,7 +9,7 @@ from common.feeder_ws import COMMAND_DISPATCH_UNAVAILABLE, FEEDER_BOT_NOT_CONNECTED from common.mc_channel_labels import ( - managed_node_mc_channels_queryset, + managed_node_mc_channel_links, mc_channel_admin_label, mc_channel_type_name, ) @@ -563,8 +563,8 @@ def mc_channel_count(self, obj): def mc_channels_mirror(self, obj): if obj is None or obj.protocol != Protocol.MESHCORE: return "—" - rows = list(managed_node_mc_channels_queryset(obj)) - if not rows: + links = list(managed_node_mc_channel_links(obj)) + if not links: return format_html( "

{}

", _("No channels synced from device yet. Connect the bot to populate this mirror."), @@ -574,11 +574,11 @@ def mc_channels_mirror(self, obj): "{}{}{}", ( ( - ch.mc_channel_idx if ch.mc_channel_idx is not None else "—", - mc_channel_type_name(ch), - mc_channel_admin_label(ch), + link.mc_channel_idx, + mc_channel_type_name(link.message_channel), + mc_channel_admin_label(link.message_channel), ) - for ch in rows + for link in links ), ) return format_html( diff --git a/Meshflow/nodes/serializers.py b/Meshflow/nodes/serializers.py index 1c5f4232..b87f7238 100644 --- a/Meshflow/nodes/serializers.py +++ b/Meshflow/nodes/serializers.py @@ -8,7 +8,7 @@ from common.mesh_node_helpers import observed_node_id_str from common.protocol import Protocol from constellations.models import Constellation, MessageChannel -from meshcore_packets.serializers_channel import MessageChannelMcSerializer +from meshcore_packets.serializers_channel import FeederMcChannelMirrorSerializer from users.models import User from .models import ( @@ -872,8 +872,10 @@ def to_representation(self, instance): else: rep["meshtastic_channel_7"] = None - rep["mc_channels"] = MessageChannelMcSerializer( - instance.mc_channels.order_by("mc_channel_idx"), + from common.mc_channel_labels import managed_node_mc_channel_links + + rep["mc_channels"] = FeederMcChannelMirrorSerializer( + managed_node_mc_channel_links(instance), many=True, ).data diff --git a/openapi.yaml b/openapi.yaml index 994f29ac..65e199c5 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -2115,7 +2115,7 @@ components: type: array description: MeshCore channels mirrored from device (protocol=MeshCore only) items: - $ref: '#/components/schemas/MessageChannel' + $ref: '#/components/schemas/FeederMcChannelMirror' mc_channels_synced_at: type: string format: date-time @@ -2956,11 +2956,11 @@ components: protocol: $ref: '#/components/schemas/MeshProtocol' description: Mesh protocol for this channel row - mc_channel_idx: - type: integer + display_label: + type: string nullable: true - minimum: 0 - description: MeshCore channel index when protocol is MeshCore; null for Meshtastic + readOnly: true + description: Operator-facing label (#hashtag for HASHTAG, plain name for PUBLIC). MeshCore only. mc_channel_type: type: string nullable: true @@ -2974,6 +2974,28 @@ components: type: integer description: The ID of the constellation this channel belongs to + FeederMcChannelMirror: + type: object + description: Canonical MessageChannel plus device slot index for a MeshCore feeder. + properties: + id: + type: integer + description: Canonical MessageChannel ID + name: + type: string + mc_channel_idx: + type: integer + minimum: 0 + maximum: 63 + description: Device channel slot index on this feeder + mc_channel_type: + type: string + nullable: true + enum: [PUBLIC, HASHTAG] + mc_hashtag: + type: string + nullable: true + McChannelSyncEntry: type: object required: [mc_channel_idx, name, mc_channel_type] @@ -3016,7 +3038,7 @@ components: mc_channels: type: array items: - $ref: '#/components/schemas/MessageChannel' + $ref: '#/components/schemas/FeederMcChannelMirror' ConstellationMember: type: object From 419b9a4e811b1b5a2376e4638c723b9845ca7868 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:26:50 +0100 Subject: [PATCH 05/10] test(meshcore): two-feeder canonical channel and ingest coverage Regression tests for shared hashtag across device indices and matching TextMessage.channel_id across feeders. --- .../common/tests/test_mc_channel_labels.py | 5 +- .../tests/test_constellation_views.py | 6 +- .../tests/test_canonical_channels.py | 113 ++++++++++++++++++ .../tests/test_channel_apply_service.py | 8 +- .../tests/test_channel_sync.py | 9 +- 5 files changed, 129 insertions(+), 12 deletions(-) create mode 100644 Meshflow/meshcore_packets/tests/test_canonical_channels.py diff --git a/Meshflow/common/tests/test_mc_channel_labels.py b/Meshflow/common/tests/test_mc_channel_labels.py index f3c7cfb1..463c255d 100644 --- a/Meshflow/common/tests/test_mc_channel_labels.py +++ b/Meshflow/common/tests/test_mc_channel_labels.py @@ -17,7 +17,6 @@ def test_mc_channel_admin_label_public(create_constellation): name="Public", constellation=constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=0, mc_channel_type=MeshCoreChannelType.PUBLIC, ) assert mc_channel_admin_label(ch) == "Public" @@ -30,7 +29,6 @@ def test_mc_channel_admin_label_hashtag_prefix(create_constellation): name="galloway", constellation=constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=1, mc_channel_type=MeshCoreChannelType.HASHTAG, mc_hashtag="galloway", ) @@ -44,11 +42,10 @@ def test_message_channel_to_apply_entry_hashtag(create_constellation): name="galloway", constellation=constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=1, mc_channel_type=MeshCoreChannelType.HASHTAG, mc_hashtag="galloway", ) - entry = message_channel_to_apply_entry(ch) + entry = message_channel_to_apply_entry(ch, mc_channel_idx=1) assert entry["mc_channel_type"] == "HASHTAG" assert entry["mc_hashtag"] == "galloway" assert entry["name"] == "galloway" diff --git a/Meshflow/constellations/tests/test_constellation_views.py b/Meshflow/constellations/tests/test_constellation_views.py index e3fbd4b9..596281df 100644 --- a/Meshflow/constellations/tests/test_constellation_views.py +++ b/Meshflow/constellations/tests/test_constellation_views.py @@ -5,7 +5,7 @@ from rest_framework.test import APIClient from common.protocol import Protocol -from constellations.models import Constellation, MessageChannel +from constellations.models import Constellation, MeshCoreChannelType, MessageChannel @pytest.mark.django_db @@ -45,7 +45,7 @@ def test_constellation_list_protocol_filter(create_constellation, create_user): name="MC Ch", constellation=mc, protocol=Protocol.MESHCORE, - mc_channel_idx=1, + mc_channel_type=MeshCoreChannelType.PUBLIC, ) response = client.get(reverse("constellation-list"), {"protocol": "meshcore"}) @@ -118,7 +118,7 @@ def test_guest_can_list_channels(create_constellation, create_user): name="MC", constellation=constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=0, + mc_channel_type=MeshCoreChannelType.PUBLIC, ) url = reverse("constellation-channels-list-create", kwargs={"constellation_id": constellation.id}) diff --git a/Meshflow/meshcore_packets/tests/test_canonical_channels.py b/Meshflow/meshcore_packets/tests/test_canonical_channels.py new file mode 100644 index 00000000..19afb35c --- /dev/null +++ b/Meshflow/meshcore_packets/tests/test_canonical_channels.py @@ -0,0 +1,113 @@ +"""Regression tests for canonical MC channels across feeders (#379).""" + +from django.utils import timezone + +import pytest + +from common.protocol import Protocol +from constellations.models import MeshCoreChannelType, MessageChannel +from meshcore_packets.models import MeshCorePayloadType, MeshCoreTextPacket +from meshcore_packets.services.channel import resolve_mc_channel +from meshcore_packets.services.channel_sync import reconcile_mc_channels +from meshcore_packets.services.text_message import MeshCoreTextMessageService +from nodes.models import NodeAuth +from text_messages.models import TextMessage + + +@pytest.mark.django_db +def test_two_feeders_same_hashtag_different_indices_one_canonical(meshcore_feeder, create_managed_node, create_node_api_key): + """Same logical hashtag on different device slots → one MessageChannel, two links.""" + constellation = meshcore_feeder["node"].constellation + feeder_b = create_managed_node( + meshtastic_node_id=None, + protocol=Protocol.MESHCORE, + name="MC Feeder B", + mc_pubkey="c" * 64, + constellation=constellation, + ) + api_key_b = create_node_api_key(constellation=constellation) + NodeAuth.objects.create(api_key=api_key_b, node=feeder_b) + + reconcile_mc_channels( + meshcore_feeder["node"], + [{"mc_channel_idx": 1, "name": "test", "mc_channel_type": "HASHTAG", "mc_hashtag": "test"}], + ) + reconcile_mc_channels( + feeder_b, + [{"mc_channel_idx": 2, "name": "test", "mc_channel_type": "HASHTAG", "mc_hashtag": "test"}], + ) + + canonical = MessageChannel.objects.filter( + constellation=constellation, + protocol=Protocol.MESHCORE, + mc_hashtag="test", + ) + assert canonical.count() == 1 + + ch_a = resolve_mc_channel(meshcore_feeder["node"], 1) + ch_b = resolve_mc_channel(feeder_b, 2) + assert ch_a.id == ch_b.id + + +@pytest.mark.django_db +def test_two_feeders_ingest_same_text_same_channel_id(meshcore_feeder, create_managed_node, create_node_api_key): + constellation = meshcore_feeder["node"].constellation + feeder_b = create_managed_node( + meshtastic_node_id=None, + protocol=Protocol.MESHCORE, + name="MC Feeder B", + mc_pubkey="c" * 64, + constellation=constellation, + ) + reconcile_mc_channels( + meshcore_feeder["node"], + [{"mc_channel_idx": 1, "name": "test", "mc_channel_type": "HASHTAG", "mc_hashtag": "test"}], + ) + reconcile_mc_channels( + feeder_b, + [{"mc_channel_idx": 2, "name": "test", "mc_channel_type": "HASHTAG", "mc_hashtag": "test"}], + ) + + now = timezone.now() + channel_a = resolve_mc_channel(meshcore_feeder["node"], 1) + packet_a = MeshCoreTextPacket.objects.create( + observer=meshcore_feeder["node"], + payload_type=MeshCorePayloadType.CHANNEL_TEXT, + event_type="channel_message", + rx_time=now, + raw_json={}, + text="hello #test", + channel=channel_a, + pkt_hash=111, + ) + from meshcore_packets.models import MeshCorePacketObservation + + obs_a = MeshCorePacketObservation.objects.create( + packet=packet_a, + observer=meshcore_feeder["node"], + channel=channel_a, + rx_time=now, + ) + msg_a = MeshCoreTextMessageService().process_packet(packet_a, meshcore_feeder["node"], obs_a) + assert msg_a is not None + + channel_b = resolve_mc_channel(feeder_b, 2) + packet_b = MeshCoreTextPacket.objects.create( + observer=feeder_b, + payload_type=MeshCorePayloadType.CHANNEL_TEXT, + event_type="channel_message", + rx_time=now, + raw_json={}, + text="hello #test", + channel=channel_b, + pkt_hash=222, + ) + obs_b = MeshCorePacketObservation.objects.create( + packet=packet_b, + observer=feeder_b, + channel=channel_b, + rx_time=now, + ) + msg_b = MeshCoreTextMessageService().process_packet(packet_b, feeder_b, obs_b) + assert msg_b is not None + assert msg_a.channel_id == msg_b.channel_id diff --git a/Meshflow/meshcore_packets/tests/test_channel_apply_service.py b/Meshflow/meshcore_packets/tests/test_channel_apply_service.py index 60653725..adc06dde 100644 --- a/Meshflow/meshcore_packets/tests/test_channel_apply_service.py +++ b/Meshflow/meshcore_packets/tests/test_channel_apply_service.py @@ -7,6 +7,7 @@ from common.protocol import Protocol from constellations.models import MeshCoreChannelType, MessageChannel from meshcore_packets.services.channel_apply import build_apply_channels_for_managed_node +from nodes.models import ManagedNodeMcChannelLink @pytest.mark.django_db @@ -17,11 +18,14 @@ def test_build_apply_channels_for_managed_node(meshcore_feeder): name="tag", constellation=constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=2, mc_channel_type=MeshCoreChannelType.HASHTAG, mc_hashtag="meshflow", ) - node.mc_channels.add(ch) + ManagedNodeMcChannelLink.objects.create( + managed_node=node, + message_channel=ch, + mc_channel_idx=2, + ) payload = build_apply_channels_for_managed_node(node) assert len(payload) == 1 diff --git a/Meshflow/meshcore_packets/tests/test_channel_sync.py b/Meshflow/meshcore_packets/tests/test_channel_sync.py index b9cdeed3..9223fe3c 100644 --- a/Meshflow/meshcore_packets/tests/test_channel_sync.py +++ b/Meshflow/meshcore_packets/tests/test_channel_sync.py @@ -46,9 +46,10 @@ def test_reconcile_mc_channels_creates_and_links(meshcore_feeder): ch0 = MessageChannel.objects.get( constellation=node.constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=0, + name="Public", + mc_channel_type=MeshCoreChannelType.PUBLIC, ) - assert ch0.mc_channel_type == MeshCoreChannelType.PUBLIC + assert node.mc_channel_links.filter(mc_channel_idx=0, message_channel=ch0).exists() @pytest.mark.django_db @@ -65,7 +66,8 @@ def test_reconcile_updates_name_on_resync(meshcore_feeder): ch = MessageChannel.objects.get( constellation=node.constellation, protocol=Protocol.MESHCORE, - mc_channel_idx=0, + name="Renamed", + mc_channel_type=MeshCoreChannelType.PUBLIC, ) assert ch.name == "Renamed" @@ -103,3 +105,4 @@ def test_resolve_mc_channel_prefers_m2m(meshcore_feeder): ch = resolve_mc_channel(node, 2) assert ch.name == "Synced" assert ch in node.mc_channels.all() + assert node.mc_channel_links.filter(mc_channel_idx=2, message_channel=ch).exists() From 46911a70141c0a21a679facb973c30745415b2df Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 15:37:08 +0100 Subject: [PATCH 06/10] fix(dx_monitoring): point initial migration at packets.MtRawPacket DxEventObservation.raw_packet referenced removed packets.rawpacket model, breaking migrate when building project state after the MT raw packet rename. --- Meshflow/dx_monitoring/migrations/0001_initial.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/Meshflow/dx_monitoring/migrations/0001_initial.py b/Meshflow/dx_monitoring/migrations/0001_initial.py index 54c458f4..07944bcb 100644 --- a/Meshflow/dx_monitoring/migrations/0001_initial.py +++ b/Meshflow/dx_monitoring/migrations/0001_initial.py @@ -12,7 +12,7 @@ class Migration(migrations.Migration): dependencies = [ ('constellations', '0005_add_bot_default_env_vars'), ('nodes', '0032_add_update_managed_node_statuses_periodic_task'), - ('packets', '0016_packetobservation_observer_upload_time_idx'), + ('packets', '0018_rename_packet_metrics_meshtastic_fields'), ] operations = [ @@ -62,7 +62,14 @@ class Migration(migrations.Migration): ('event', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='observations', to='dx_monitoring.dxevent')), ('observer', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='dx_event_observations', to='nodes.managednode')), ('packet_observation', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='dx_event_observations', to='packets.packetobservation')), - ('raw_packet', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='dx_event_observations', to='packets.rawpacket')), + ( + 'raw_packet', + models.ForeignKey( + on_delete=django.db.models.deletion.CASCADE, + related_name='dx_event_observations', + to='packets.mtrawpacket', + ), + ), ], options={ 'verbose_name': 'DX event observation', From 555a0e8a1d0af601c274bdbe857742e2eec5ae76 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 16:52:00 +0100 Subject: [PATCH 07/10] docs(meshcore): path chain findings and ingestion cross-links Document pre-prod message path gap (channel_text vs rx_log_data ADVERT), thin-bot/fat-server direction, and fix meshcore ingestion doc formatting. Gitignore Meshflow/ai-env for pre-prod DB skill. --- .gitignore | 3 + AGENTS.md | 4 + .../packet-path-tracing-outstanding.md | 64 ++++++++++ docs/features/packet-ingestion/meshcore.md | 111 ++++++++++-------- .../traceroute/meshcore-path-outstanding.md | 8 +- 5 files changed, 141 insertions(+), 49 deletions(-) diff --git a/.gitignore b/.gitignore index ac372923..28151529 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,9 @@ ## Mac stuff .DS_Store +### Local secrets (pre-prod DB skill — see .cursor/skills/preprod-database/) +Meshflow/ai-env + ### app outputs state.json nodes.json diff --git a/AGENTS.md b/AGENTS.md index e55ba626..45639732 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,6 +65,10 @@ Make sure to activate the venv `venv/bin/activate` See **tests/TESTING.md** for detailed instructions (unit tests, integration tests via Docker Compose or local Django). +### Pre-production database (opt-in) + +Agents must **not** query pre-prod unless the user explicitly enables it. When enabled, follow **[.cursor/skills/preprod-database/SKILL.md](.cursor/skills/preprod-database/SKILL.md)** (`disable-model-invocation`; copy credentials to local `Meshflow/ai-env`, gitignored). + ## Code Style - **Linting**: black, isort, flake8 diff --git a/docs/features/meshcore/packet-path-tracing/packet-path-tracing-outstanding.md b/docs/features/meshcore/packet-path-tracing/packet-path-tracing-outstanding.md index ae16fd13..66e42901 100644 --- a/docs/features/meshcore/packet-path-tracing/packet-path-tracing-outstanding.md +++ b/docs/features/meshcore/packet-path-tracing/packet-path-tracing-outstanding.md @@ -47,6 +47,70 @@ Until then, operators should assume heard-map paths are **list-order hash eviden --- +## Message path data chain (confirmed — pre-prod Jun 2026) + +**Symptom:** Message “Heard” UI and `GET` message `heard[]` show observers but **no hop chain** for MeshCore channel/contact text on pre-prod, even though M1 edge rollups exist and the heard-map UI (#311) is wired to `path_hashes`. + +**Not the cause:** Single feeder (one observation per `packet_id` is expected). API heard assembly and UI #311 behave correctly when `MeshCorePacketObservation.path_hashes` is set on the **same** packet as `TextMessage.original_mc_packet` (see `text_messages/tests/test_heard_api.py`). + +### Pre-prod evidence (read-only DB, one feeder) + +| Metric | Value | +| --- | --- | +| Observations with non-empty `path_hashes` | 754 | +| Event type for those rows | **100%** `rx_log_data` (stored as `advert` ingest) | +| `channel_text` observations with `path_hashes` | **0** (746 `channel_text` rows without path) | +| MeshCore `TextMessage` rows with path on linked packet observation | **0** | +| Packets with >1 observation | **0** (one feeder) | + +M1 rollups on pre-prod are fed by **overheard ADVERT** frames (`rx_log_data` → `payload_typename == ADVERT`), not by channel-message ingest. + +### End-to-end chain (logic confirmed) + +```text +channel_message (bot) → channel_text (API) → TextMessage.original_mc_packet + → heard[] reads obs.path_hashes on THAT packet → empty today + +rx_log_data ADVERT only (bot) → advert (API) → observation.path_hashes populated + → no TextMessage link → does not appear in message heard[] +``` + +1. **High-level decode (`channel_message`, `contact_message`)** — captures include `path_len` and often `path_hash_mode`, but **usually no `path` hex** (see meshflow-bot `docs/meshcore_packets/channel_messages/*.json`). Without `path`, ingest cannot populate `path_hashes` (bot today only forwards a pre-split list when `path` is present; API persists what the bot sends). + +2. **Low-level decode (`rx_log_data`)** — ADVERT (and PATH) frames **do** include `path` + `path_hash_size` in captures (see `docs/meshcore_packets/rx_log_data/*ADVERT*.json`). Bot uploads **ADVERT `rx_log_data` only**; TEXT_MSG / PATH / etc. are skipped (`MeshCoreSkipUpload`) per [packet-ingestion/meshcore.md](../../packet-ingestion/meshcore.md). + +3. **API** — no payload-type filter on `path_hashes`; M1 segment/edge tasks consume whatever observations exist. + +### Design constraint: thin bot, fat server + +Meshflow bots should stay **deploy-light**: forward captures with minimal transformation. **Maintainable path logic belongs on the API** (ingest normalization, dedup, rollups, heard assembly, resolution). + +Implications for closing this gap (direction only — not scheduled here): + +| Approach | Bot change | Server work | +| --- | --- | --- | +| **A. Ingest more `rx_log_data`** (TEXT_MSG / PATH with `path` on wire) | Thin: upload additional typename(s) as `raw` or mapped payload types; no hash splitting | Ingest serializer splits `path` → `path_hashes`; correlate to `TextMessage` via `pkt_hash` / time / dedup (needs design) | +| **B. Store decode metadata + raw** on observation | Thin: pass through `path_len`, `path_hash_mode`, optional `raw_hex` from envelope | Server decodes or joins to PATH/`rx_log_data` rows if/when ingested | +| **C. Wait for meshcore lib** to add `path` on `channel_message` | None if library starts emitting `path` | Existing ingest path may “just work” once `path` arrives in JSON | + +**Avoid** adding bot-side `_path_hashes()`-style rules, new upload filters, or message↔packet correlation — that duplicates server responsibility and is painful to roll out per feeder. + +### Sample references + +| Source | `path` present? | +| --- | --- | +| `channel_message` dump `20260507_094941_052599.json` | No (`path_len`, `path_hash_mode` only) | +| `rx_log_data` ADVERT `20260506_211819_583174.json` | Yes | +| `rx_log_data` PATH `20260506_211515_351329.json` | Yes (not uploaded today) | + +### Follow-up (tracking) + +- [ ] **Server-led ingest design** — extend API (and optionally bot upload surface) so channel-text messages get `path_hashes` on the observation tied to `original_mc_packet`, without bot-side path logic. Likely depends on [#266](https://github.com/pskillen/meshflow-api/issues/266) / `rx_log_data` TEXT_MSG ingest and/or dedup correlation spike. +- [ ] **Confirm with M2 spike** — whether `path_hash_mode` changes segment identity when we do get text paths. +- [ ] **Optional:** re-run pre-prod queries after deploy (`Meshflow/ai-env` + Django shell; local skill `MeshFlow/.cursor/skills/preprod-database/`) — breakdown by `payload_type` + `event_type`. + +--- + ## Cross-links - [ ] Update [#267](https://github.com/pskillen/meshflow-api/issues/267) epic and this file as milestones land. diff --git a/docs/features/packet-ingestion/meshcore.md b/docs/features/packet-ingestion/meshcore.md index 1912f220..213f6e39 100644 --- a/docs/features/packet-ingestion/meshcore.md +++ b/docs/features/packet-ingestion/meshcore.md @@ -1,6 +1,6 @@ # MeshCore packet ingestion — data path -Reverse-engineered map of how MeshCore (MC) traffic moves from a feeder through **meshflow-bot** into **`meshcore_packets`** and downstream business models. Capture-level field reference: [MESHCORE_PACKET_FIELDS.md](MESHCORE_PACKET_FIELDS.md) (derivative of [meshflow-bot `docs/meshcore_packets/`](https://github.com/pskillen/meshflow-bot/tree/main/docs/meshcore_packets)). +Reverse-engineered map of how MeshCore (MC) traffic moves from a feeder through **meshflow-bot** into **meshcore_packets** and downstream business models. Capture-level field reference: [MESHCORE_PACKET_FIELDS.md](MESHCORE_PACKET_FIELDS.md) (derivative of [meshflow-bot `docs/meshcore_packets/`](https://github.com/pskillen/meshflow-bot/tree/main/docs/meshcore_packets)). Phase 1 MVP is shipped; [epic #266](https://github.com/pskillen/meshflow-api/issues/266) tracks full packet-type parity (advert subclasses, ack/req/resp, telemetry, dual-FK migrations). @@ -10,14 +10,16 @@ Phase 1 MVP is shipped; [epic #266](https://github.com/pskillen/meshflow-api/iss `MeshCorePacketSerializer.UPLOADABLE_PAYLOAD_TYPES` = `advert`, `channel_text`, `contact_text` only. -| Bot `event_type` (envelope) | API `payload_type` | Uploaded? | -| --- | --- | --- | -| `advertisement` | `advert` | Yes | -| `rx_log_data` with `payload_typename == ADVERT` | `advert` (+ optional `adv_lat` / `adv_lon` / `adv_name`) | Yes | -| `channel_message` | `channel_text` | Yes | -| `contact_message` | `contact_text` | Yes | -| `rx_log_data` (non-ADVERT, e.g. TEXT_MSG, PATH, REQ, CONTROL) | — | **No** — `MeshCoreSkipUpload` in bot | -| `control_data`, `discover_response`, `path_update`, `trace_data`, `messages_waiting`, … | — | **No** — capture/dump only unless later phases map them | + +| Bot `event_type` (envelope) | API `payload_type` | Uploaded? | +| --------------------------------------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------- | +| `advertisement` | `advert` | Yes | +| `rx_log_data` with `payload_typename == ADVERT` | `advert` (+ optional `adv_lat` / `adv_lon` / `adv_name`) | Yes | +| `channel_message` | `channel_text` | Yes | +| `contact_message` | `contact_text` | Yes | +| `rx_log_data` (non-ADVERT, e.g. TEXT_MSG, PATH, REQ, CONTROL) | — | **No** — `MeshCoreSkipUpload` in bot | +| `control_data`, `discover_response`, `path_update`, `trace_data`, `messages_waiting`, … | — | **No** — capture/dump only unless later phases map them | + Requires `RADIO_PROTOCOL=meshcore` and `MESHCORE_UPLOAD_ENABLED=true`. Otherwise the bot writes JSON under `data/meshcore_packets/` only. @@ -35,24 +37,28 @@ Code: `meshflow-bot/src/meshcore/serializers.py`, `src/api/StorageAPI.py` (`stor `MeshCorePacketIngestSerializer` (`meshcore_packets/serializers.py`): -| `payload_type` | `MeshCorePayloadType` | Raw model | Business / side effects | -| --- | --- | --- | --- | -| `advert` | `ADVERT` (1) | `MeshCoreRawPacket` | `ObservedNode` upsert (pubkey / prefix); optional `Position` + `NodeLatestStatus` if `adv_lat`/`adv_lon` non-zero; `meshcore_adv_type` on node | -| `channel_text` | `CHANNEL_TEXT` (2) | `MeshCoreTextPacket` | `TextMessage` (`original_mc_packet`, `protocol=MESHCORE`); channel FK via feeder `channel_idx` | -| `contact_text` | `CONTACT_TEXT` (3) | `MeshCoreTextPacket` | `TextMessage` + sender `ObservedNode` from `from_pubkey_prefix`; node-claim path | -| `raw` | `RAW` (99) | `MeshCoreRawPacket` | **Serializer supports it; bot does not upload `raw` today** | -Also creates or updates **`MeshCorePacketObservation`** per `(packet, observer)` (deduped). Repeater **`path_hashes`** are stored on the observation row only (not on deduped `MeshCoreRawPacket`), so two feeders reporting the same `pkt_hash` can keep different paths. See [ADR-0001 (path hash resolution)](../traceroute/adr/0001-mc-path-hash-resolution.md). +| `payload_type` | `MeshCorePayloadType` | Raw model | Business / side effects | +| -------------- | --------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `advert` | `ADVERT` (1) | `MeshCoreRawPacket` | `ObservedNode` upsert (pubkey / prefix); optional `Position` + `NodeLatestStatus` if `adv_lat`/`adv_lon` non-zero; `meshcore_adv_type` on node | +| `channel_text` | `CHANNEL_TEXT` (2) | `MeshCoreTextPacket` | `TextMessage` (`original_mc_packet`, `protocol=MESHCORE`); channel FK via feeder `channel_idx` | +| `contact_text` | `CONTACT_TEXT` (3) | `MeshCoreTextPacket` | `TextMessage` + sender `ObservedNode` from `from_pubkey_prefix`; node-claim path | +| `raw` | `RAW` (99) | `MeshCoreRawPacket` | **Serializer supports it; bot does not upload `raw` today** | + + +Also creates or updates **MeshCorePacketObservation** per `(packet, observer)` (deduped). Repeater **path_hashes** are stored on the observation row only (not on deduped `MeshCoreRawPacket`), so two feeders reporting the same `pkt_hash` can keep different paths. See [ADR-0001 (path hash resolution)](../traceroute/adr/0001-mc-path-hash-resolution.md). Dedup: `meshcore_packets/services/dedup.py` — `pkt_hash` + time window (see [adr/0004-mc-dedup-key.md](adr/0004-mc-dedup-key.md)). ### API short-circuits -| Condition | Response | -| --- | --- | -| Top-level `encrypted` | **304**, no write | -| Invalid body | **400** | -| Non-MC feeder / wrong prefix | **403** | + +| Condition | Response | +| ---------------------------- | ----------------- | +| Top-level `encrypted` | **304**, no write | +| Invalid body | **400** | +| Non-MC feeder / wrong prefix | **403** | + --- @@ -82,15 +88,19 @@ flowchart LR Z --> TXT --> TM ``` + + --- ## API → raw storage -| Model | Role | -| --- | --- | -| `MeshCoreRawPacket` | Common row: observer, `payload_type`, `event_type`, pubkey fields, `pkt_hash`, RF metadata, `raw_json` | -| `MeshCoreTextPacket` | Subclass for channel/contact text (`text`, `channel`, `to_pubkey_prefix`) | -| `MeshCorePacketObservation` | Per-feeder hearing (like MT `PacketObservation`) | + +| Model | Role | +| --------------------------- | ------------------------------------------------------------------------------------------------------ | +| `MeshCoreRawPacket` | Common row: observer, `payload_type`, `event_type`, pubkey fields, `pkt_hash`, RF metadata, `raw_json` | +| `MeshCoreTextPacket` | Subclass for channel/contact text (`text`, `channel`, `to_pubkey_prefix`) | +| `MeshCorePacketObservation` | Per-feeder hearing (like MT `PacketObservation`) | + List API (JWT): `GET /api/meshcore/packets/` — filter `payload_type`, `from_pubkey_prefix`. @@ -100,21 +110,25 @@ List API (JWT): `GET /api/meshcore/packets/` — filter `payload_type`, `from_pu ### `meshcore_packets` app -| Step | Module | Behaviour | -| --- | --- | --- | -| All ingests | `receivers.upsert_observed_node_from_meshcore_packet` | `meshcore_packet_received` → `resolve_or_create_mc_observed_node`, `last_heard` | -| ADVERT + coords | `services/position.apply_advert_position` | `Position.original_mc_packet`, `NodeLatestStatus` | -| Text | `text_messages.receivers` → `MeshCoreTextMessageService` | `TextMessage.original_mc_packet` | + +| Step | Module | Behaviour | +| --------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------- | +| All ingests | `receivers.upsert_observed_node_from_meshcore_packet` | `meshcore_packet_received` → `resolve_or_create_mc_observed_node`, `last_heard` | +| ADVERT + coords | `services/position.apply_advert_position` | `Position.original_mc_packet`, `NodeLatestStatus` | +| Text | `text_messages.receivers` → `MeshCoreTextMessageService` | `TextMessage.original_mc_packet` | + No dedicated services yet for ack/req/resp/telemetry — **#266** scope. ### Cross-app -| App | Role | -| --- | --- | -| `text_messages` | MC text + claims (`try_accept_node_claim` on contact text) | -| `stats` | `mc_packet_volume` snapshots count `MeshCoreRawPacket` / observations (see [packet-stats/meshcore.md](../packet-stats/meshcore.md)) | -| `constellations` | `MessageChannel` resolution for `channel_idx` on text + observations | + +| App | Role | +| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `text_messages` | MC text + claims (`try_accept_node_claim` on contact text) | +| `stats` | `mc_packet_volume` snapshots count `MeshCoreRawPacket` / observations (see [packet-stats/meshcore.md](../packet-stats/meshcore.md)) | +| `constellations` | `MessageChannel` resolution for `channel_idx` on text + observations | + MC does **not** use the Meshtastic `packets` app or `MtRawPacket` tables. @@ -131,12 +145,14 @@ MC does **not** use the Meshtastic `packets` app or `MtRawPacket` tables. Captured on air (see [MESHCORE_PACKET_FIELDS.md](MESHCORE_PACKET_FIELDS.md)) but **not** end-to-end in API yet: -| Wire / dump area | Epic direction | -| --- | --- | -| `rx_log_data` TEXT_MSG, PATH, REQ, RESP, CONTROL | Subclasses + storage-only or business rules | -| Dedicated `ack` / `req` / `resp` events | Storage models | -| Telemetry / CayenneLPP | `NodeLatestStatus` + metric receivers | -| `protocol` + `original_mt_packet` / `original_mc_packet` on shared business tables | Dual-FK migrations with CHECK | + +| Wire / dump area | Epic direction | +| ---------------------------------------------------------------------------------- | ------------------------------------------- | +| `rx_log_data` TEXT_MSG, PATH, REQ, RESP, CONTROL | Subclasses + storage-only or business rules | +| Dedicated `ack` / `req` / `resp` events | Storage models | +| Telemetry / CayenneLPP | `NodeLatestStatus` + metric receivers | +| `protocol` + `original_mt_packet` / `original_mc_packet` on shared business tables | Dual-FK migrations with CHECK | + Track execution: [meshcore/phase-2-outstanding.md](../meshcore/phase-2-outstanding.md). Path/traceroute parity ([#267](https://github.com/pskillen/meshflow-api/issues/267)): [traceroute/meshcore-path-progress.md](../traceroute/meshcore-path-progress.md). @@ -144,10 +160,12 @@ Track execution: [meshcore/phase-2-outstanding.md](../meshcore/phase-2-outstandi ## Provenance FKs (MC) -| Business model | Link to raw | -| --- | --- | -| `TextMessage` | `original_mc_packet` → `MeshCoreTextPacket` | -| `Position` | `original_mc_packet` → `MeshCoreRawPacket` (ADVERT ingest) | + +| Business model | Link to raw | +| -------------- | ---------------------------------------------------------- | +| `TextMessage` | `original_mc_packet` → `MeshCoreTextPacket` | +| `Position` | `original_mc_packet` → `MeshCoreRawPacket` (ADVERT ingest) | + Meshtastic provenance remains `original_packet` on `TextMessage` only (no MC FK on MT-only paths). @@ -159,3 +177,4 @@ Meshtastic provenance remains `original_packet` on `TextMessage` only (no MC FK - [MESHCORE_PACKET_FIELDS.md](MESHCORE_PACKET_FIELDS.md) - [MeshCore phase 1](../meshcore/phase-1-progress.md) - [Packet stats (MC)](../packet-stats/meshcore.md) + diff --git a/docs/features/traceroute/meshcore-path-outstanding.md b/docs/features/traceroute/meshcore-path-outstanding.md index 003dbe74..cd9c446c 100644 --- a/docs/features/traceroute/meshcore-path-outstanding.md +++ b/docs/features/traceroute/meshcore-path-outstanding.md @@ -18,9 +18,11 @@ Items **skipped**, **incomplete**, or **discovered during planning** for [#267]( ## Bot ([meshflow-bot#119](https://github.com/pskillen/meshflow-bot/issues/119)) -- [ ] Unit tests for `_path_hashes()` (1/2/3-byte `path_hash_size`) — separate bot PR. -- [ ] Optional: upload `rx_log_data` PATH-only frames. -- [ ] Document limitation: `path_len > 0` but no `path` on decoded messages → `path_hashes` often null. +Prefer **thin bot / fat server** for path handling — see [packet-path-tracing-outstanding.md § Message path data chain](../meshcore/packet-path-tracing/packet-path-tracing-outstanding.md#message-path-data-chain-confirmed--pre-prod-jun-2026). + +- [ ] Unit tests for `_path_hashes()` (1/2/3-byte `path_hash_size`) — separate bot PR (legacy; new work should move splitting to API where possible). +- [ ] Optional: upload `rx_log_data` PATH-only frames (minimal forward; server ingests `path`). +- [x] Document limitation: `path_len > 0` but no `path` on decoded messages → message `heard[]` has no path — detailed in packet-path-tracing outstanding (pre-prod confirmed). --- From 745c06f0464fa76790ad63ec75e1fc5648e7571d Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 16:55:30 +0100 Subject: [PATCH 08/10] chore: linting --- Meshflow/meshcore_packets/tests/test_canonical_channels.py | 7 ++++--- Meshflow/meshcore_packets/views.py | 2 +- Meshflow/nodes/admin.py | 1 + 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Meshflow/meshcore_packets/tests/test_canonical_channels.py b/Meshflow/meshcore_packets/tests/test_canonical_channels.py index 19afb35c..babbe322 100644 --- a/Meshflow/meshcore_packets/tests/test_canonical_channels.py +++ b/Meshflow/meshcore_packets/tests/test_canonical_channels.py @@ -5,17 +5,18 @@ import pytest from common.protocol import Protocol -from constellations.models import MeshCoreChannelType, MessageChannel +from constellations.models import MessageChannel from meshcore_packets.models import MeshCorePayloadType, MeshCoreTextPacket from meshcore_packets.services.channel import resolve_mc_channel from meshcore_packets.services.channel_sync import reconcile_mc_channels from meshcore_packets.services.text_message import MeshCoreTextMessageService from nodes.models import NodeAuth -from text_messages.models import TextMessage @pytest.mark.django_db -def test_two_feeders_same_hashtag_different_indices_one_canonical(meshcore_feeder, create_managed_node, create_node_api_key): +def test_two_feeders_same_hashtag_different_indices_one_canonical( + meshcore_feeder, create_managed_node, create_node_api_key +): """Same logical hashtag on different device slots → one MessageChannel, two links.""" constellation = meshcore_feeder["node"].constellation feeder_b = create_managed_node( diff --git a/Meshflow/meshcore_packets/views.py b/Meshflow/meshcore_packets/views.py index 4cb4e6cc..4f02e0b8 100644 --- a/Meshflow/meshcore_packets/views.py +++ b/Meshflow/meshcore_packets/views.py @@ -168,7 +168,7 @@ def post(self, request, feeder_pubkey_prefix, format=None): return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST) try: - channels = reconcile_mc_channels( + reconcile_mc_channels( managed_node, serializer.validated_data["channels"], synced_at=serializer.validated_data.get("synced_at"), diff --git a/Meshflow/nodes/admin.py b/Meshflow/nodes/admin.py index 568ff696..966d6ea1 100644 --- a/Meshflow/nodes/admin.py +++ b/Meshflow/nodes/admin.py @@ -10,6 +10,7 @@ from common.feeder_ws import COMMAND_DISPATCH_UNAVAILABLE, FEEDER_BOT_NOT_CONNECTED from common.mc_channel_labels import ( managed_node_mc_channel_links, + managed_node_mc_channels_queryset, mc_channel_admin_label, mc_channel_type_name, ) From 927f6122755be0c8da5bf7df2e78476c9160fe70 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 16:57:48 +0100 Subject: [PATCH 09/10] docs(meshcore): sync progress files for #379 canonical channels Update phase-2 progress/outstanding, text-message-channels status table, and ADR-0002 UI note for ManagedNodeMcChannelLink model. --- docs/features/meshcore/phase-2-outstanding.md | 2 +- docs/features/meshcore/phase-2-progress.md | 21 ++++++++++++++++++- .../meshcore/text-message-channels.md | 3 ++- .../adr/0002-mc-channel-modelling.md | 2 +- 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/docs/features/meshcore/phase-2-outstanding.md b/docs/features/meshcore/phase-2-outstanding.md index f23aa636..5ec4f322 100644 --- a/docs/features/meshcore/phase-2-outstanding.md +++ b/docs/features/meshcore/phase-2-outstanding.md @@ -102,7 +102,7 @@ Follow-up after local UI + pre-prod bot + shared Postgres/Redis. Tracked on [#29 - [ ] **Empty device channel table** — bot can connect and sync while `get_channel` scan finds 0 named channels; other tools may show channels on the same radio. Needs device/firmware/meshcore_py investigation; bot logs per-slot scan ([#107](https://github.com/pskillen/meshflow-bot/pull/107)). - [ ] **Auto-set `mc_pubkey` on first connect** — still manual in admin ([#279](https://github.com/pskillen/meshflow-api/issues/279)). - [ ] **OpenAPI** — confirm all MC feeder paths and apply responses match deployed code after #335 merge (code was ahead of spec in places during staging). -- [ ] **Admin push action** — dispatches mirror only; editing constellation rows in **MeshCore channels** does not auto-link to `ManagedNode.mc_channels` or push (operators must sync mirror via bot connect or understand M2M). +- [ ] **Admin push action** — pushes the feeder’s **linked** channel list (`ManagedNodeMcChannelLink` order/slots); editing constellation **MeshCore channels** admin does not auto-assign links or push (use Node Settings / bot sync). - [ ] **Dual API (`STORAGE_API_2_*`)** — bot POSTs `mc-channel-sync` (and packets) to **both** APIs when upload enabled; **WebSocket / apply** only on primary `STORAGE_API_ROOT`. Documented in [text-message-channels.md](./text-message-channels.md); UI apply against API 2 while bot WS on API 1 will always 503. ### Intentional / by design (document only) diff --git a/docs/features/meshcore/phase-2-progress.md b/docs/features/meshcore/phase-2-progress.md index 776bbfa4..ad1def2a 100644 --- a/docs/features/meshcore/phase-2-progress.md +++ b/docs/features/meshcore/phase-2-progress.md @@ -86,7 +86,7 @@ Deploy api #325 before bot fleet upgrade. **meshflow-api — delivered** -- `MessageChannel.mc_channel_type` / `mc_hashtag`; `ManagedNode.mc_channels` M2M + `mc_channels_synced_at`. +- `MessageChannel.mc_channel_type` / `mc_hashtag`; `ManagedNodeMcChannelLink` + `ManagedNode.mc_channels` (through table) + `mc_channels_synced_at`. - `POST /api/meshcore/feeders/{prefix}/mc-channel-sync/`; `POST …/apply-mc-channel-config/` (WS dispatch). - `MeshCoreTextMessageService` + `text_messages` receiver; `TextMessage.protocol` + `original_mc_packet`. - History API `protocol` query; MC `heard` from `MeshCorePacketObservation`. @@ -106,6 +106,25 @@ Deploy api #325 before bot fleet upgrade. --- +## Canonical MC channels ([#379](https://github.com/pskillen/meshflow-api/issues/379)) + +**Status:** Open PRs — [api #380](https://github.com/pskillen/meshflow-api/pull/380), [ui #313](https://github.com/pskillen/meshflow-ui/pull/313). + +**meshflow-api — delivered** + +- `MessageChannel` unique by constellation + logical identity (name/hashtag); `mc_channel_idx` removed from channel row. +- `ManagedNodeMcChannelLink` holds per-feeder device slot; `reconcile_mc_channels` / `resolve_mc_channel` use links. +- `display_label` on constellation channels; feeder mirror nested on managed node API. + +**meshflow-ui — delivered** + +- `formatMessageChannelLabel` uses `display_label` / `#hashtag` (no device index in picker). +- Node Settings: dual-list channel editor (Available → On this radio), reorder warning, apply confirmation modal. + +**meshflow-bot:** no wire changes. + +--- + ## MeshCore messages UI ([ui#275](https://github.com/pskillen/meshflow-ui/issues/275)) **Status:** In progress (branch `ui-275/paddy/meshcore-messages` on meshflow-ui). diff --git a/docs/features/meshcore/text-message-channels.md b/docs/features/meshcore/text-message-channels.md index 0880586f..c6afea12 100644 --- a/docs/features/meshcore/text-message-channels.md +++ b/docs/features/meshcore/text-message-channels.md @@ -21,7 +21,7 @@ How MeshCore **group text** and **channel configuration** flow from the radio th |---|----------------|--------------| | Channel on radio | Fixed slots 0–7, PSK + name in firmware | Arbitrary list on companion; **index** on wire, **name** only in device config | | Feeder channel config | API slot FKs → `MessageChannel` (operator maps slots in UI) | Device is source of truth; API **mirror** via bot `mc-channel-sync` on connect | -| Feeder channel link in API | `meshtastic_channel_0..7` | `ManagedNode.mc_channels` M2M (reconciled from device snapshot) | +| Feeder channel link in API | `meshtastic_channel_0..7` | `ManagedNodeMcChannelLink` (slot → canonical `MessageChannel`; reconciled from device snapshot) | | What a text packet carries | Channel index + sender node id | `channel_message`: **index + body only** (no sender pubkey); `contact_message`: **12-hex sender prefix** + body | | Broadcast vs DM | `to_int == 0xFFFFFFFF` vs directed node id | Broadcast = **no** `to_pubkey*` on wire; channel text is always broadcast on that index ([ADR-0003](../packet-ingestion/adr/0003-mc-broadcast-semantics.md)) | | UI today | Slot 0–7 mapping on [Node Settings](https://github.com/pskillen/meshflow-ui/blob/main/src/pages/user/NodeSettings.tsx) | MC feeders: mirror + **Apply to radio** on Node Settings | @@ -374,6 +374,7 @@ Identity receiver **skips** channel text (no `from_pubkey` / prefix). Contact te | `TextMessage` + MC provenance ([#296](https://github.com/pskillen/meshflow-api/issues/296)) | **Done** | | Bot sync + WS apply ([#297](https://github.com/pskillen/meshflow-api/issues/297)) | **Done** | | UI mirror + apply-to-radio | **Done** | +| Canonical channels + per-feeder links ([#379](https://github.com/pskillen/meshflow-api/issues/379)) | **Done** — [api #380](https://github.com/pskillen/meshflow-api/pull/380), [ui #313](https://github.com/pskillen/meshflow-ui/pull/313) | | Apply 503 / msgpack / WS group fixes | **Done** on [api #335](https://github.com/pskillen/meshflow-api/pull/335), [bot #108](https://github.com/pskillen/meshflow-bot/pull/108) (merge pending) | | Django admin MC channels + push action | **Done** (#335) | | Dual API channel sync (no WS on API 2) | **Done** (bot behaviour; documented) | diff --git a/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md b/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md index 8c6fa200..b2b271e8 100644 --- a/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md +++ b/docs/features/packet-ingestion/adr/0002-mc-channel-modelling.md @@ -45,7 +45,7 @@ We still need a way to: - **`mc_channels` M2M lets a managed node "subscribe" to an arbitrary subset of MC channels**, which fits the protocol better than 8 fixed slots. UI and admin screens for `ManagedNode` need a small per-protocol branch. - **If MC firmware later adds a channel hash on the wire**, we can add `mc_channel_hash` and a matching unique constraint without touching the dispatch path — `channel_idx` continues to be the primary dispatch key. - **Risk:** auto-creating `MessageChannel` rows on first sight could pollute the table if an attacker sends bogus channel indices via a compromised observer. Mitigation: limit to indices `0..63` (MC's plausible range) and require channel rows to be attached to the **observer's** `mc_channels` only on packets the observer authenticated for — which they already are. -- **Out of scope:** UI for managing `mc_channels` per managed node; covered in Phase 1.7 (UI ticket). +- **UI for managing feeder channels:** Node Settings dual-list editor + apply-to-radio ([#379](https://github.com/pskillen/meshflow-api/issues/379) / [ui #313](https://github.com/pskillen/meshflow-ui/pull/313)). ## Supplement (2026-05-20) — device as source of truth for operator metadata From ee8a4f84cf8a6856af0c0f3a2218aabed90ae8a8 Mon Sep 17 00:00:00 2001 From: Patrick Skillen Date: Mon, 1 Jun 2026 17:00:17 +0100 Subject: [PATCH 10/10] docs(meshcore): add Phase 3 progress/outstanding stubs for #267 Introduce phase-3-progress.md and phase-3-outstanding.md under meshcore/, index them in README, and link from meshcore-path-progress. --- docs/features/meshcore/README.md | 5 +- docs/features/meshcore/phase-3-outstanding.md | 51 +++++++++++++++ docs/features/meshcore/phase-3-progress.md | 62 +++++++++++++++++++ .../traceroute/meshcore-path-progress.md | 2 + 4 files changed, 118 insertions(+), 2 deletions(-) create mode 100644 docs/features/meshcore/phase-3-outstanding.md create mode 100644 docs/features/meshcore/phase-3-progress.md diff --git a/docs/features/meshcore/README.md b/docs/features/meshcore/README.md index a845bb07..d118d49c 100644 --- a/docs/features/meshcore/README.md +++ b/docs/features/meshcore/README.md @@ -1,6 +1,6 @@ # MeshCore / multi-protocol documentation -Cross-repo MeshCore work is tracked on GitHub epics [#264](https://github.com/pskillen/meshflow-api/issues/264) (Phase 0), [#265](https://github.com/pskillen/meshflow-api/issues/265) (Phase 1), [#266](https://github.com/pskillen/meshflow-api/issues/266) (Phase 2). +Cross-repo MeshCore work is tracked on GitHub epics [#264](https://github.com/pskillen/meshflow-api/issues/264) (Phase 0), [#265](https://github.com/pskillen/meshflow-api/issues/265) (Phase 1), [#266](https://github.com/pskillen/meshflow-api/issues/266) (Phase 2), [#267](https://github.com/pskillen/meshflow-api/issues/267) (Phase 3). **Progress and follow-ups** are split by phase (not one monolithic file): @@ -9,9 +9,10 @@ Cross-repo MeshCore work is tracked on GitHub epics [#264](https://github.com/ps | **0** — bot seam, capture, ADRs | [phase-0-progress.md](./phase-0-progress.md) | [phase-0-outstanding.md](./phase-0-outstanding.md) | | **1** — MC ingestion MVP | [phase-1-progress.md](./phase-1-progress.md) | [phase-1-outstanding.md](./phase-1-outstanding.md) | | **2** — parity, rename track, position/text | [phase-2-progress.md](./phase-2-progress.md) | [phase-2-outstanding.md](./phase-2-outstanding.md) | +| **3** — traceroute / path parity ([#267](https://github.com/pskillen/meshflow-api/issues/267)) | [phase-3-progress.md](./phase-3-progress.md) | [phase-3-outstanding.md](./phase-3-outstanding.md) | | **ManagedNode identity** ([#362](https://github.com/pskillen/meshflow-api/issues/362)) | [managed-node-identity-progress.md](./managed-node-identity-progress.md) | [managed-node-identity-outstanding.md](./managed-node-identity-outstanding.md) | | **Feeder enrollment** ([#293](https://github.com/pskillen/meshflow-ui/issues/293)) | [enrollment-progress.md](./enrollment-progress.md) | [enrollment-outstanding.md](./enrollment-outstanding.md) | -| **Passive packet path** ([#267](https://github.com/pskillen/meshflow-api/issues/267)) | [packet-path-tracing/packet-path-tracing-progress.md](./packet-path-tracing/packet-path-tracing-progress.md) | [packet-path-tracing/packet-path-tracing-outstanding.md](./packet-path-tracing/packet-path-tracing-outstanding.md) | +| **Passive packet path** (Phase 3 slice) | [packet-path-tracing/packet-path-tracing-progress.md](./packet-path-tracing/packet-path-tracing-progress.md) | [packet-path-tracing/packet-path-tracing-outstanding.md](./packet-path-tracing/packet-path-tracing-outstanding.md) | **Agent convention:** [progress-tracking skill](../../../.cursor/skills/progress-tracking/SKILL.md) — update progress/outstanding at plan breakpoints and before PRs. diff --git a/docs/features/meshcore/phase-3-outstanding.md b/docs/features/meshcore/phase-3-outstanding.md new file mode 100644 index 00000000..199f240c --- /dev/null +++ b/docs/features/meshcore/phase-3-outstanding.md @@ -0,0 +1,51 @@ +# Phase 3 — outstanding + +Items **skipped**, **incomplete**, or **discovered during Phase 3** ([#267](https://github.com/pskillen/meshflow-api/issues/267)) — not a copy of the full epic backlog. + +**Passive-path detail:** [packet-path-tracing/packet-path-tracing-outstanding.md](./packet-path-tracing/packet-path-tracing-outstanding.md). +**Traceroute-folder mirror:** [meshcore-path-outstanding.md](../traceroute/meshcore-path-outstanding.md). + +--- + +## Passive path + +- [ ] **Channel text `heard[]` without hops** — `channel_message` ingest often has `path_len` but no `path`; ADVERT-only `rx_log_data` has paths but no `TextMessage` link. Server-led ingest design: [packet-path-tracing-outstanding.md § Message path data chain](./packet-path-tracing/packet-path-tracing-outstanding.md#message-path-data-chain-confirmed--pre-prod-jun-2026). +- [ ] **Passive packet path subsystem** — rollups, resolution table, Neo4j export, realtime/history UI ([ADR-0001](./packet-path-tracing/adr/0001-meshcore-packet-path-tracing-subsystem.md)). +- [ ] **Proven hash → node matcher** — per [traceroute ADR §A](../traceroute/adr/0001-mc-path-hash-resolution.md); no unsafe heuristics in v1. +- [ ] **`GET /meshcore/packets/`** — optional `resolved_path` on list/detail (deferred). +- [ ] **[meshflow-ui#311](https://github.com/pskillen/meshflow-ui/issues/311)** — schematic hop chain per feeder in heard dialog. + +### Bot ([meshflow-bot#119](https://github.com/pskillen/meshflow-bot/issues/119)) + +Prefer **thin bot / fat server** — see packet-path-tracing outstanding. + +- [ ] Unit tests for `_path_hashes()` (1/2/3-byte `path_hash_size`). +- [ ] Optional: upload `rx_log_data` PATH (and related) frames for server-side `path` split. + +--- + +## Active traceroute (epic backlog) + +From [#267](https://github.com/pskillen/meshflow-api/issues/267) — not scheduled in passive slice: + +- [ ] Spike + ADR: MC vs MT traceroute semantics (`meshcore_py`). +- [ ] Schema: extend `AutoTraceRoute` or add `MeshCorePathObservation`. +- [ ] `pick_traceroute_target` + Celery: never mix MT sources with MC targets. +- [ ] `traceroute_analytics` / Neo4j: protocol on nodes/edges. +- [ ] UI: heatmap / topology / coverage protocol filter. +- [ ] MC new-node baseline traceroute analog. + +--- + +## Ops / cross-cutting + +- [ ] Update [#267](https://github.com/pskillen/meshflow-api/issues/267) epic checklist when major slices merge. +- [ ] Keep [phase-3-progress.md](./phase-3-progress.md) in sync with [meshcore-path-progress.md](../traceroute/meshcore-path-progress.md) (avoid duplicate narratives — link, don’t fork). + +--- + +## Resolved (do not re-open) + +*(Move items here when closing.)* + +- [x] **#304** — UI heard path map ([meshflow-ui#304](https://github.com/pskillen/meshflow-ui/issues/304)). diff --git a/docs/features/meshcore/phase-3-progress.md b/docs/features/meshcore/phase-3-progress.md new file mode 100644 index 00000000..4a194180 --- /dev/null +++ b/docs/features/meshcore/phase-3-progress.md @@ -0,0 +1,62 @@ +# Phase 3 — progress + +**Epic:** [#267](https://github.com/pskillen/meshflow-api/issues/267) — MeshCore traceroute / path parity (planning + passive path first). + +**Milestone:** MeshCore Support. + +--- + +## Scope (epic summary) + +MC **path/trace** or passive hop accumulation as a Meshtastic traceroute analog: `AutoTraceRoute.protocol` (or sibling model), scheduler guards by protocol, Neo4j edges labelled by protocol, UI filters on traceroute pages, MC new-node baseline. + +**Two tracks** (see [traceroute/meshcore-path-progress.md](../traceroute/meshcore-path-progress.md)): + +| Track | Goal | Detail doc | +| --- | --- | --- | +| **Passive path** | `path_hashes` on observations → resolve → UI / rollups | [packet-path-tracing/packet-path-tracing-progress.md](./packet-path-tracing/packet-path-tracing-progress.md) | +| **Active traceroute** | Bot command + `AutoTraceRoute` (or MC sibling) + analytics | *Not started — spike/ADR backlog* | + +--- + +## Delivered (log here as work lands) + +*(Stub — add sections with PR/issue links when closing slices.)* + +### Passive path — message heard UI + +- [meshflow-ui#304](https://github.com/pskillen/meshflow-ui/issues/304) — heard path map in message UI (closed 2026-05-27). + +### Passive path — API `path_hashes` on observations + +- See [meshcore-path-progress.md](../traceroute/meshcore-path-progress.md) § Passive path slice ([#369](https://github.com/pskillen/meshflow-api/issues/369), [#360](https://github.com/pskillen/meshflow-api/issues/360)). + +### Passive path — bot forward `path` when present + +- [meshflow-bot#119](https://github.com/pskillen/meshflow-bot/issues/119) — `_path_hashes()` on ingest when wire `path` is set. + +--- + +## In flight + +- **Packet path subsystem** — ADR + rollups + resolution: [packet-path-tracing/](./packet-path-tracing/). +- **UI #311** — logical path per feeder in heard dialog ([meshflow-ui#311](https://github.com/pskillen/meshflow-ui/issues/311)). + +--- + +## Cross-repo issue map (passive slice) + +| Repo | Issue | +| --- | --- | +| meshflow-api | [#360](https://github.com/pskillen/meshflow-api/issues/360), [#369](https://github.com/pskillen/meshflow-api/issues/369) | +| meshflow-bot | [#119](https://github.com/pskillen/meshflow-bot/issues/119) | +| meshflow-ui | [#304](https://github.com/pskillen/meshflow-ui/issues/304), [#311](https://github.com/pskillen/meshflow-ui/issues/311) | + +--- + +## References + +- [phase-3-outstanding.md](./phase-3-outstanding.md) +- [traceroute/README.md](../traceroute/README.md) § MeshCore path parity +- [ADR-0001 — MC path hash resolution](../traceroute/adr/0001-mc-path-hash-resolution.md) +- [packet-ingestion/meshcore.md](../packet-ingestion/meshcore.md) — what the bot uploads today diff --git a/docs/features/traceroute/meshcore-path-progress.md b/docs/features/traceroute/meshcore-path-progress.md index ccc5e24b..4df779bc 100644 --- a/docs/features/traceroute/meshcore-path-progress.md +++ b/docs/features/traceroute/meshcore-path-progress.md @@ -2,6 +2,8 @@ **Epic:** [meshflow-api#267](https://github.com/pskillen/meshflow-api/issues/267) — MeshCore Phase 3 traceroute / path parity (MC analog of Meshtastic `AutoTraceRoute` + analytics). +**Phase index (meshcore/):** [phase-3-progress.md](../meshcore/phase-3-progress.md) · [phase-3-outstanding.md](../meshcore/phase-3-outstanding.md) + **Repos:** meshflow-api (primary), meshflow-bot, meshflow-ui. **Related docs**