🚀 Upcoming Feature: True Multi-Writer Transactions in CSharpDB

[MaxAkbar]

Hi everyone 👋

I’m excited to share what’s coming next for CSharpDB — a major architectural upgrade:

True Concurrent Multi-Writer Transactions

Today, CSharpDB supports:

• Multiple concurrent readers (snapshot-based)
• High-throughput auto-commit writes (via WAL pipelining)

However, explicit multi-statement write transactions are still effectively single-writer.

That’s about to change.

---

🔥 What’s Coming

We’re introducing support for:

• Concurrent write transactions
• Conflict detection + automatic retry
• Preserved ACID guarantees
• Full compatibility with existing WAL + snapshot readers

This means multiple writers will be able to:

• Execute transactions in parallel
• Safely commit without corrupting shared state
• Scale much better under real-world workloads

---

🧠 High-Level Design

The implementation will roll out in phases.

Phase 1 (Initial Release)

• Snapshot Isolation (SI)
• Per-transaction Copy-On-Write (COW) pages
• Serialized commit with validation
• Page-level conflict detection (first-committer-wins)

👉 Each transaction works on its own private page copies and only validates at commit time.

---

Phase 2 (Advanced Concurrency)

• Key-range conflict detection (phantom-safe)
• Moves toward Serializable isolation
• Reduces false conflicts for scans and indexed queries

---

⚙️ Why This Approach?

CSharpDB is built around:

• A WAL-based architecture
• Full-page writes
• Snapshot readers

So instead of adding heavy locking, we’re going with:

👉 Optimistic Concurrency Control (OCC)

This keeps:

• Reads fast (no blocking)
• Writes scalable
• Architecture aligned with current design

---

📊 What This Unlocks

• True parallel write workloads

• Better throughput on multi-core systems

• Real-world transactional patterns (not just single statements)

• Foundation for future features like:

  • Advanced query optimization
  • Distributed/sharded scenarios
  • High-performance OLTP workloads

---

⚠️ Tradeoffs (Being Transparent)

• Some transactions may conflict and require retry
• Hot pages (e.g. sequential inserts) can cause contention
• Long-running transactions may delay checkpoints

These are expected tradeoffs for OCC systems and will be improved over time.

---

💬 Feedback Wanted

I’d love input on:

• Transaction API design (developer experience)
• Conflict + retry behavior
• Edge cases you care about (especially real workloads)
• Any concerns around isolation guarantees

---

🙌 Final Thoughts

--
This is still an evolving design, and details may change as we move forward. I’ll keep everyone updated along the way and share the final implementation once it’s ready.

Appreciate all feedback, ideas, and challenges — especially from people who’ve worked on storage engines or high-concurrency systems.

Let’s build something powerful 🚀