Skip to content

Persistent graph#241

Open
TeunHuijben wants to merge 43 commits into
mainfrom
persistent-graph
Open

Persistent graph#241
TeunHuijben wants to merge 43 commits into
mainfrom
persistent-graph

Conversation

@TeunHuijben

@TeunHuijben TeunHuijben commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Updates

  • Removed SolutionTracks — track-id logic (get/set track & lineage ids, from_tracks, neighbor lookups) folded into Tracks; TrackAnnotator now registers based on a non-None tracklet_key instead of an isinstance check.
  • Split the graph into graph_full + graph_solutiontracks.graph is gone; graph_solution is the solution=True view, graph_full is its root (every node/edge ever known). All call sites in src/ and tests/ migrated.
  • Reversible soft-deleteDeleteNode/DeleteEdge flip solution=False in graph_full and remove from the view only (remove_*_from_view); AddNode/AddEdge branch on graph_full.has_node/has_edge to revive the same id in place or add genuinely new. Topology is append-only, so delete↔undo is a true inverse across repeated/divergent cycles.
  • Annotator scope splitRegionpropsAnnotator/EdgeAnnotator compute on graph_full (features persist on soft-deleted candidates, ready for re-solving); TrackAnnotator stays on graph_solution. Low-level attr get/set route through graph_full (shared attr dicts keep the view in sync).
  • Depth-1 guardTracks.__init__ raises if graph_solution._root is itself a GraphView, and _get_new_node_ids allocates against graph_full to avoid colliding with soft-deleted ids.
  • tracksdata pinned to TeunHuijben/tracksdata@040ed9f for the view-only add/remove primitives (revert once merged upstream).
  • Import/export + builders updated to construct Tracks (declaring tracklet/lineage intent), preserve solution flags, and backfill Solution() on legacy saves.
  • Tests — new test_soft_delete_roundtrip.py (12 cases) and test_features_on_full_graph.py; full suite green (588 passed).

Breaking changes

  • SolutionTracks is removed — use Tracks (pass tracklet_attr/lineage_attr, or a FeatureDict with a tracklet_key).
  • tracks.graph is removed — use tracks.graph_solution (solution view) or tracks.graph_full (full graph).

@TeunHuijben TeunHuijben requested a review from cmalinmayor as a code owner June 11, 2026 16:47
@TeunHuijben TeunHuijben added the ready for initial review PR ready for first review label Jun 11, 2026
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.08333% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.89%. Comparing base (6baed08) to head (66aa352).

Files with missing lines Patch % Lines
src/funtracks/data_model/tracks.py 95.41% 5 Missing ⚠️
src/funtracks/import_export/geff/_export.py 75.00% 1 Missing ⚠️
...funtracks/user_actions/user_update_segmentation.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #241      +/-   ##
==========================================
+ Coverage   93.78%   93.89%   +0.10%     
==========================================
  Files          59       58       -1     
  Lines        3284     3275       -9     
==========================================
- Hits         3080     3075       -5     
+ Misses        204      200       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TeunHuijben

Copy link
Copy Markdown
Collaborator Author

@cmalinmayor, persistent graph is ready for a first glance 😇

Comment thread src/funtracks/annotators/_graph_annotator.py Outdated

@cmalinmayor cmalinmayor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, some high level comments, some low level suggestions

Comment thread src/funtracks/data_model/tracks.py Outdated
Comment thread src/funtracks/data_model/tracks.py Outdated
Comment thread src/funtracks/data_model/tracks.py
Comment thread src/funtracks/data_model/tracks.py
Comment thread src/funtracks/data_model/tracks.py Outdated
Comment thread src/funtracks/user_actions/user_add_edge.py Outdated
Comment thread src/funtracks/user_actions/user_add_node.py Outdated
self.tracks: SolutionTracks # Narrow type from base class
if not self.tracks.graph.has_edge(*edge):
self.tracks: Tracks # Narrow type from base class
if not self.tracks.graph_solution.has_edge(*edge):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graph_full

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graph_solution, because we can only remove edges that are in the solution, since we only do a soft-delete technically

Comment thread src/funtracks/user_actions/user_delete_edge.py
Comment thread src/funtracks/user_actions/user_update_segmentation.py Outdated
@cmalinmayor cmalinmayor added iterating PR undergoing review iterations and removed ready for initial review PR ready for first review labels Jun 26, 2026
@TeunHuijben TeunHuijben requested a review from cmalinmayor July 1, 2026 18:37
@TeunHuijben

TeunHuijben commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

Hi @cmalinmayor, here we go again, hahah (why are these PRs always so big 😂)

I implemented and answered all of your concerns. Biggest change is that:

  • graph_full is now the main player, and graph_solution merely a view of it, so not the other way around.

Notes:

  • we probably need a lot of extra work again once we switch to a persistent SQL graph (because of the view-out-of-sync issues), but lets park that for later
  • the failing Windows benchmark is solved on tracksdata PR 310, will be merged soon

Let me know what you think 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

iterating PR undergoing review iterations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants