Port main.go to C# implementation (main.cs)

[Copilot]

This PR implements a complete C# port of the JetstreamDB server from main.go to main.cs, providing full API compatibility and feature parity with the original Go implementation.

Overview

JetstreamDB is a low-complexity data and code storage solution that provides hash-based file storage with distributed clustering capabilities. This C# implementation maintains all the core functionality while leveraging ASP.NET Core for the HTTP server infrastructure.

Key Features Implemented

Core Storage Engine

• SHA256-based file storage: Files are stored and retrieved using their SHA256 hash, ensuring data integrity
• Dual storage modes:
  • Non-volatile storage (hash-based, immutable)
  • Volatile storage (key-value pairs, mutable)
• Memory pool management: Uses ConcurrentQueue<byte[]> to manage buffer pools and prevent memory allocation bottlenecks

HTTP API Compatibility

• All HTTP methods: GET, PUT, POST, DELETE, HEAD
• Special endpoints: /kv for key-value operations
• Query parameters: Support for apikey, append, setifnot, take, burst, chtimes, format
• Content-Type handling: Proper MIME type handling and cache control headers

Security & Authentication

• API key authentication: Environment variable or file-based API key support
• Quantum-grade rate limiting: Implements the same security delays and mutex locking
• Path validation: Prevents directory traversal attacks
• Immutable storage protection: Hash-based files cannot be modified or deleted

Distributed Features

• Cluster support: Multi-node deployment with configurable topology
• Backup chains: Automatic replication to backup nodes
• Dynamic restore: Fetches missing data from peer nodes during startup
• Load balancing: Random node selection for distributed requests

Background Services

• Automatic cleanup: Configurable retention period with background file deletion
• Health monitoring: Updates file access times to prevent premature cleanup
• Startup restoration: Fetches data from backup nodes during warmup period

Technical Implementation

The C# version uses modern .NET 8 features:

• ASP.NET Core: For HTTP server and middleware pipeline
• Hosted Services: For background cleanup tasks
• HttpClient: For distributed node communication
• System.Security.Cryptography: For SHA256 hashing
• System.IO: For file operations with proper async/await patterns

Testing Verified

All core functionality has been tested and verified to work identically to the Go version:

# Store data with hash-based addressing
echo "Hello World" | curl -X PUT --data-binary @- http://127.0.0.1:7777/?apikey=testkey
# Returns: /d2a84f4b8b650937ec8f73cd8be2c74add5a911ba64df27458ed8229da804a26.dat

# Retrieve stored data
curl http://127.0.0.1:7777/d2a84f4b8b650937ec8f73cd8be2c74add5a911ba64df27458ed8229da804a26.dat
# Returns: Hello World

# Key-value operations
echo "test" | curl -X PUT --data-binary @- http://127.0.0.1:7777/custom_key.dat?apikey=testkey
curl -X DELETE http://127.0.0.1:7777/custom_key.dat?apikey=testkey

Project Structure

• main.cs: Complete C# implementation of JetstreamDB
• JetStreamDB.csproj: .NET 8 project file with ASP.NET Core dependencies
• .gitignore: Excludes C# build artifacts

Compatibility

The C# implementation maintains 100% API compatibility with the Go version, allowing seamless migration or side-by-side deployment in mixed environments.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.