Semantic Processing Unit (SPU)

Neurosymbolic Architecture for Invariant-Preserving AI

Large Language Models (LLMs) fail at simple arithmetic because they treat numbers as tokens and use probabilistic approximation. This repository introduces the Semantic Processing Unit (SPU) — a neurosymbolic architecture that decouples semantic parsing (System 1) from algebraic execution (System 2).

Instead of predicting the answer, the neural network predicts a differentiable matrix operator. The calculation itself is performed by deterministic hardware-accelerated matrix algebra, ensuring 100% accuracy and conservation of invariants even on Out-of-Distribution (OOD) data.

---

🔬 Core Experiments

The project is divided into three fundamental proofs of concept:

01. Mass Conservation: MLP vs SPU

File: 01_mass_conservation.py

Tests the ability of the network to redistribute resources without losing or creating "matter."

• Baseline (MLP): Fails catastrophically when input values scale by x1000 (OOD).
• SPU: Maintains zero-error mass conservation regardless of scale, thanks to architectural constraints (Softmax-based gating).

02. Algebraic Composition ( Group)

File: 02_s3_composition.py

Demonstrates Zero-Shot logic. The network learns only two generators of the group (SwapAB and SwapBC).

• Result: The model successfully derives a new operator (SwapAC) through matrix multiplication with machine precision, without ever seeing during training.

03. Locality and Disentanglement ( Group)

File: 03_s4_locality.py

Proves that the SPU can isolate variables in complex systems.

• Locality: Operations on elements do not interfere with .
• Commutativity: The model learns that (Disjoint Commutativity).
• Deep Chain: Successfully derives long-distance swaps (e.g., ) through a chain of 5+ matrix multiplications.

---

🏗 Architecture Overview

The SPU splits the computation graph into four stages:

1. Data Path (Nouns): Raw state vector . Never passes through trainable weights.
2. Neural Controller (System 1): An MLP that parses the command and generates a raw operator matrix.
3. Physics Gate: A Softmax layer that enforces column-stochastic constraints (sum=1), ensuring conservation laws.
4. Algebraic Processor (System 2): Hardware MatMul (). Pure math, zero hallucinations.

---

🔗 Related Resources

Full Article on Habr/Medium https://habr.com/ru/articles/1001400/