Three $4 chips. One can hear you, one can think, one can render worlds. Together, they're one computer.
What happens when you split a computer's brain into three physical pieces, and ask them to cooperate?
Nexus-4D is an attempt to answer that question using three ESP32-S3 microcontrollers — no single board does everything, but together they form a distributed system with voice perception, central routing, and dedicated rendering.
This project has no precedent to follow. The architecture is original, the road is unpaved, and the success rate is honestly near zero.
We're building it anyway.
┌─────────────────────────┐
│ Cloud Brain (LLM) │ ← DeepSeek API (OpenAI-compatible)
│ ✅ Connected │ Deep reasoning, tool calling
└───────────┬─────────────┘
│ Wi-Fi / HTTPS
▼
┌─────────────────────────┐
│ Command Layer (Voice) │ ← Voice in, intent out
│ Top Board ESP32-S3 │ Xiaozhi AI · MQTT → Cloud LLM
│ ✅ Working │
└───────────┬─────────────┘
│ UART @ 115200bps
▼
┌─────────────────────────┐
│ Hub Layer (Brain) │ ← ReAct Agent · 8 LLM Tools
│ Logic Board ESP32-S3 │ MimiClaw framework · DeepSeek
│ ✅ Thinking │ Gamepad · SD routing
└─────┬─────────────┬─────┘
│ │
UART @ 115200bps [Future Nodes]
│
┌─────▼───────────────────┐
│ Render Node (GPU) │ ← Pure execution, zero AI
│ Render Board ESP32-S3 │ ILI9341 · OLED · Audio
│ ✅ 109 FPS │ Retro-Go engine
└─────────────────────────┘
The idea is simple: Don't cram everything into one chip. Let each board do one thing, and do it well. Connect them. Let them talk.
| Layer | Role | What It Actually Does |
|---|---|---|
| Command | Ears & Voice | Listens, speaks, connects to cloud LLM via MQTT |
| Hub | Agent & Router | ReAct Agent loop (think → act → observe), routes commands via UART, reads gamepad, 8 registered tools |
| Render | GPU & Media | Runs Retro-Go engine: games, images, audio, dual-display. Receives bridge commands (display/notify/return) |
| Milestone | Result | How |
|---|---|---|
| Render board display | 109 FPS on ILI9341 320×240 | Retro-Go, MADCTL=0xE8 |
| OLED secondary display | SSD1306 "Sharingan" eye, 250ms refresh | I2C, custom animation |
| SD card storage | FAT32 @ 40MHz SPI, stable | Soldered connection, no re-trace drops |
| Gamepad → Hub → Render | 7-byte packet, <5ms latency | UART @ 115200bps, soldered |
| Hub ↔ Top AI | Bidirectional heartbeat, boot messages | UART @ 115200bps, jumper wires |
| Tri-node full loop | Top ↔ Hub ↔ Render end-to-end | Three boards, two UART links |
| PSRAM 8MB Octal | Enabled, strategically reserved | All hot paths run from SRAM |
| Xiaozhi AI voice | Wake, listen, cloud LLM response | ESP-IDF v5.5.4, MQTT session |
| Hub AI (NEW) | DeepSeek API online, ReAct Agent loop | MimiClaw framework, FreeRTOS dual-core |
| LLM Tool Calling (NEW) | 8 registered tools, function calling works | read_gamepad, send_key, display_text, etc. |
| Render bridge commands (NEW) | DISPLAY / NOTIFY / RETURN implemented | 0xBB protocol, Retro-Go launcher handles |
| Hub WiFi (NEW) | Connected, stable IP assigned | WiFi STA mode, 192.168.110.118 |
Current status: Researching. No winner yet.Current status: ✅ Implemented. The Hub board can think.
The Hub board runs MimiClaw — an open-source ReAct Agent framework for ESP32-S3. Here's what that means:
User speaks: "Show me a message on the screen"
│
┌───────────────▼───────────────┐
│ Hub Board (MimiClaw) │
│ │
│ 1. Receive text from Top AI │
│ 2. Send to DeepSeek API │
│ 3. LLM returns tool call: │
│ display_text("Hello!") │
│ 4. Execute tool → UART → │
│ Render board shows text │
│ 5. Observe result, reply │
└───────────────────────────────┘
| Tool | What It Does |
|---|---|
read_gamepad |
Read current joystick + button state |
send_key |
Simulate a button press to the render board |
send_to_render |
Send raw text to the render board |
send_to_top |
Send a message back to the top (voice) board |
nexus_status |
Query full system status (all boards) |
display_text |
Display text on the render board's LCD |
notify |
Pop up a NEXUS alert on the render board |
return_to_launcher |
Make the render board exit to menu |
- Framework: MimiClaw (MIT, pure C, ESP-IDF native)
- LLM: DeepSeek API (OpenAI-compatible, function calling)
- Memory: Persistent —
SOUL.md(personality) +USER.md(user info) +MEMORY.md(context) - Dual-core: Core 0 = Agent loop + network I/O, Core 1 = gamepad polling + UART routing
- Flash: 16MB (factory=8MB + spiffs=7MB + coredump=64KB)
User: "Play some music"
→ Top AI → Hub Agent decides → tool call: send_to_render(PLAY_MP3)
→ Render board plays MP3 from SD card
The bus carries commands, not media. Files live on the render board's SD card.
Voice → movement. Hub enforces safety boundaries. Anti-collision validation before any motor command reaches a physical actuator.
Distributed clock sync. Hot-plug node discovery. OTA with atomic rollback. SPI HD high-speed bus. The works.
Each function board gets its own AI shell:
- Mobile platform (wheels + camera + ESP32-S3)
- Local image recognition + path planning via cloud LLM
- Video stream back to Hub over Wi-Fi
- Hot-swappable, self-registering nodes
Will we get there? Probably not. Will we learn trying? Absolutely.
We've all been there: one ESP32 running WiFi + I2S audio + SPI display + RTOS + AI = audio pops, heap fragmentation, mysterious crashes, and nights of debugging.
| The Single-Chip Nightmare | Nexus-4D Answer |
|---|---|
| I2S + SPI + WiFi fight for bus time | Each subsystem has its own chip |
| Heap fragments, system dies | Memory is physically isolated |
| One crash kills everything | Faulty board resets independently |
| Adding features = rewriting everything | Add a new board, update Hub routing |
| AI inference starves real-time tasks | Agent loop runs on dedicated core |
Render Board (The Muscle)
SoC: ESP32-S3 N16R8 (16MB Flash + 8MB Octal PSRAM)
LCD: ILI9341 320×240 landscape (MADCTL=0xE8, 109 FPS)
OLED: SSD1306 128×32 performance monitor
SD: MicroSD FAT32 @ 40MHz (SPI3 DMA)
Audio: I2S DAC
Engine: Retro-Go (launcher + retro-core + prboom-go)
IDF: v4.4.8
Logic Board (The Brain)
SoC: ESP32-S3 N16R8 (16MB Flash + 8MB Octal PSRAM)
FW: MimiClaw (ReAct Agent framework)
Serial1: → Render Board @ 115200bps (GPIO1 TX / GPIO2 RX, soldered)
Serial2: → Top Board @ 115200bps (GPIO15 TX / GPIO16 RX, jumper)
Gamepad: Joystick(GPIO4/5) + A(6) B(7) C(9) D(11) E(12) F(13)
ADC: Joystick center=1900, deadzone=500 (12-bit)
WiFi: STA mode, DeepSeek API connected
LLM: DeepSeek (deepseek-chat, OpenAI-compatible)
Console: UART0 GPIO43/44 @ 115200bps
IDF: v5.5.4
Top Board (The Voice)
SoC: ESP32-S3-WROOM-1 N8R2
AI: Xiaozhi AI firmware (cloud LLM via MQTT)
Audio: ES8311 codec (I2S)
Display: SPI LCD
LED: WS2812 RGB (status indicator)
UART1: → Logic Board @ 115200bps (GPIO2 TX / GPIO1 RX)
IDF: v5.5.4
Top Board Logic Board Render Board
GPIO2 (TX) ───────► GPIO16 (RX)
GPIO1 (RX) ◄─────── GPIO15 (TX)
GPIO1 (TX) ────────► GPIO2 (RX)
GPIO2 (RX) ◄──────── GPIO8 (TX)
GND ◄─────────────► GND ◄──────────────► GND
Gamepad (Hub → Render): 7 bytes, 115200bps
[0xAA][0x55][btn0][btn1][btn2][btn3][xor ^ 0x55]
Bridge Commands (Hub → Render): Variable length
Header: [0xBB][CMD][DLEN][DATA...][XOR]
Commands: 0x03=DISPLAY, 0x04=NOTIFY, 0x05=RETURN_TO_LAUNCHER
Hub ↔ Top: JSON text lines, \n terminated, 5-second heartbeat
This is not a polished product. It's a lab experiment on a desk.
Here's what's real and what's not:
What's real:
- Three boards talk to each other through two UART links. Press a button on the gamepad → it travels through the Hub board → arrives at the Render board → a game moves on screen. Latency: under 5ms.
- The Hub board runs a genuine ReAct Agent loop. It sends your words to DeepSeek, gets back tool calls, executes them, and reports the result. This is not
if (command == "UP") send(UP);anymore. - The Render board runs Retro-Go at 109 FPS with PSRAM-backed frame buffers.
- 8 LLM tools are registered and callable.
display_textandnotifyactually put pixels on the screen.
What's not there yet:
- The Top↔Hub link still uses jumper wires that fall off if you breathe wrong.
- There's no OTA. No hot-plug. No fault recovery. No watchdog.
- The Agent loop has no streaming — it waits for the full LLM response before acting.
- There's no audio bridge yet — the Hub can't tell the Render board to play sounds.
- The whole thing runs on a desk. No motors. No sensors. No autonomy.
But here's the thing: We went from "three boards that can pass bytes" to "a distributed system where you speak, an AI agent thinks, and a game console responds." In a week. With $12 worth of chips.
That's not nothing.
cd retro-go
python rg_tool.py --target esp32-s3-devkit --no-networking build-img launcher retro-core prboom-go
# Flash: esptool.py -p COM6 write_flash 0x0 retro-go_unknown_esp32-s3-devkit.imgcd mimiclaw
idf.py set-target esp32s3
idf.py -p COM4 flash monitorSee Xiaozhi AI documentation for build instructions.
This project is built by one person and an AI assistant (Aily), with occasional "help" from Doubao and DeepSeek (who are great at formatting and terrible at technical accuracy).
We need:
- ESP32 tinkerers who've tried multi-board projects
- Embedded engineers who know FreeRTOS and RTOS internals
- Edge AI researchers who've deployed models on microcontrollers
- Anyone who thinks the single-chip approach is wrong and wants to prove it
You don't need to be an expert. We certainly aren't. You just need to be willing to hit walls with us.
git clone https://github.com/NextGod2016/Nexus-4D.gitOpen an issue. Break something. Suggest something. Build something.
"I'm doing this project with the certainty that it will probably never succeed. But I can't predict what I'll encounter along the way. Only when I hit a wall can I figure out how to go through it, over it, or around it."
- Retro-Go — The software soul of the render board. Display, audio, emulators, PSRAM management.
- MimiClaw — The Agent brain of the Hub board. ReAct loop, LLM proxy, tool registry, persistent memory.
- Xiaozhi AI — Voice interaction kernel for the command layer.
- ESP-IDF — Espressif's IoT development framework.
- Aily Blockly — Visual embedded development environment.
- DeepSeek — Affordable, capable LLM API that powers the Hub's brain.
- Every ESP32 maker who ever stared at a heap fragmentation error at 3 AM.
MIT — Take it, break it, make it better.
Last updated: July 6, 2026 · Maintainer: @NextGod2016
Topics: esp32-s3 · distributed-computing · edge-computing · heterogeneous-computing · embedded-cluster · voice-assistant · react-agent · llm-tools · retro-gaming · uart · maker-project