Synth

An online-learning neuroevolution system that evolves neural network topologies using NEAT-style genetic algorithms while training connection weights via per-example stochastic gradient descent. 14 experiments, from 78% to 96% MNIST accuracy, all designed and run by Claude Code.

Read the full write-up — how the research worked, what we found, and why a lab notebook is all you need for LLM-driven research.

Highlights

• 95.87% MNIST accuracy with 2,943 connections (11.4% of a dense equivalent)
• 14 experiments autonomously designed, run, and analyzed by Claude Code
• The notebook method: a journal.md + experiments.md is enough to maintain research coherence across LLM sessions — no vector databases, no RAG, no special memory systems
• Architecture >> hyperparameters: one structural change (+18pp) outweighed 12 experiments of tuning (+8pp combined)

Running

# Place MNIST train-images-idx3-ubyte and train-labels-idx1-ubyte in data/
cargo run --release

Architecture

Genome (source of truth)
  ├── NodeGenes: id, kind (Input/Output/Hidden/Bias), activation (ReLU/Sigmoid/Tanh/Identity)
  └── ConnectionGenes: innovation number, from_node, to_node, weight, enabled
        ↓ compile (topological sort)
Network (disposable phenotype)
  ├── Forward pass: weighted sums through DAG, softmax on outputs
  └── Backward pass: cross-entropy loss, reverse topo order backprop, SGD weight update
        ↓ after training
write_weights_to_genome() → crossover/mutation → new Genome → new Network

Networks start with a seeded hidden layer (784→32→10, sparse input→hidden connections with ReLU) and evolve from there. Evolution adds connections, removes connections, splits connections through new hidden nodes, and changes activation functions. SGD trains the weights. The population is culled and bred every 10,000 training steps.

Project Structure

src/
  main.rs              Entry point, training loop
  config.rs            All hyperparameters
  genome/              Genetic representation
    genome.rs          Genome struct, seeded topology constructor
    mutation.rs        6 mutation operators (add/remove connection, add node, etc.)
    crossover.rs       NEAT-style crossover
    innovation.rs      Global innovation tracker
  network/             Compiled phenotype
    phenotype.rs       Topological sort, connection compilation
    forward.rs         Forward pass with softmax
    backward.rs        Backprop + SGD
  population/          Evolutionary dynamics
    population.rs      Population management, evolve()
    selection.rs       Tournament selection, culling
    niche.rs           Ecological niches (data distribution speciation)
  data/                MNIST loading, ratio-based data streaming
  fitness/             Rolling window accuracy with half-cosine weighting
notes/                 Research journal and experiment logs
docs/                  GitHub Pages site

Dependencies

• rand 0.9 — random number generation
• rayon 1.10 — parallel training
• byteorder 1.5 — MNIST IDX binary format parsing

Everything else built from scratch. No tensor libraries — arbitrary DAG topologies don't map to regular matrix operations.