Skip to content

fix(raft): fix election and network issues in raft event loop#126

Merged
slow2342 merged 1 commit into
mainfrom
fix/raft-election-network
Jun 14, 2026
Merged

fix(raft): fix election and network issues in raft event loop#126
slow2342 merged 1 commit into
mainfrom
fix/raft-election-network

Conversation

@slow2342

Copy link
Copy Markdown
Owner

Summary

Fix several issues in the Raft event loop that caused election instability and unreliable message delivery. Replace spin-based ticking with accurate time-based ticking, reorder ready processing to match raft-rs best practices, and add observability for debugging raft issues.

Changes

  • src/raft/node.rs — Replace MAX_SPIN loop with Instant-based tick scheduling to prevent tick drift. Reorder ready processing: send leader messages before persisting entries (parallel path per raft-rs docs). Replace for loop over ready with while node.has_ready(). Add tracing for steps, proposals, and leader state updates.
  • src/main.rs — Increase heartbeat_tick from 1 (100ms) to 2 (200ms) to reduce heartbeat overhead. Set max_committed_size_per_ready to 1MB (0 means no entries returned).
  • src/raft/network.rs — Improve send_message error handling to distinguish RPC errors from task panics. Add structured logging for outgoing raft messages.
  • src/raft/rpc.rs — Add tracing for received Vote RPC and InstallSnapshot messages.
  • src/raft/handle.rs — Add logging for require_leader checks.
  • src/raft/raftrs_store.rs — Add debug logging for entries() and term() store operations.

How to verify

cargo build --release
cargo test
cargo clippy -- -D warnings

Replace spin-based ticking with Instant-based accurate timing to prevent
tick drift. Reorder ready processing to send leader messages before
persisting entries, matching raft-rs best practices. Fix heartbeat interval
from 100ms to 200ms and set max_committed_size_per_ready to ensure entries
are returned from ready(). Add tracing for raft messages, proposals, and
leader state transitions to aid debugging. Improve network sender error
handling to distinguish between RPC errors and task panics.
@slow2342 slow2342 force-pushed the fix/raft-election-network branch from 49e26a2 to 75d571d Compare June 14, 2026 10:30
@slow2342 slow2342 merged commit 797b3b9 into main Jun 14, 2026
2 of 5 checks passed
@slow2342 slow2342 deleted the fix/raft-election-network branch June 14, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant