MicroGrad

A minimal neural network and autograd engine written in Go.

MicroGrad is a scalar-valued autograd engine, computation graph, backpropagation, and a simple Multi-Layer Perceptron (MLP) built from scratch without any external machine learning libraries.

Architecture

Value
├── data
├── grad
├── source nodes
└── backward function

Neuron
├── weights
├── bias
└── activation

Layer
└── collection of neurons

MLP
└── collection of layers

Training Example

for i := 0; i < 100; i++ {
    output := mlp.Init(input)

    loss := Loss(target, output)

    for _, p := range mlp.Params() {
        p.grad = 0
    }

    loss.Backward()

    for _, p := range mlp.Params() {
        p.data += -0.01 * p.grad
    }

    fmt.Println(loss.data)
}

A simple gradient descent training loop using backpropagation.

References

• Andrej Karpathy's Micrograd
  https://github.com/karpathy/micrograd

• Micrograd Walkthrough Notebook
  https://colab.research.google.com/drive/1FPTx1RXtBfc4MaTkf7viZZD4U2F9gtKN

• Neural Networks: Zero to Hero (Video Series)
  https://www.youtube.com/watch?v=VMj-3S1tku0