manipsim

TL;DR Small, research-oriented MuJoCo 3.3.6 layer for manipulation robotics.

Features

• Shared assets — one MJCF tree for both languages (assets/robots/, assets/scenes/)
• Robot presets — KUKA IIWA, Franka Panda, UR5
• Scene builder — attach multiple robots, add box/sphere/cylinder primitives, compile into one model
• Per-robot binding — state(), control(), dynamics() operate on each robot's DOFs only
• Passive viewer (C++) — MuJoCo simulate viewer with wall-clock sync (1.0 = 100% real-time)
• Python viewer — MuJoCo passive viewer via render=True

Repository layout

manipsim/
├── README.md
├── CMakeLists.txt              # optional; prefer configuring from cpp/
├── environment.yml
├── assets/
│   ├── scenes/default_scene.xml    # checker floor, lighting, skybox
│   └── robots/{iiwa,panda,ur5}/    # robot.xml + mesh assets/
├── python/
│   ├── pyproject.toml
│   ├── manipsim/                   # Python package
│   │   ├── core/                   # Scene, BaseRobot, binding
│   │   └── robots/                 # IIWA, Panda, UR5
│   ├── examples/
│   └── tests/
├── cpp/
│   ├── CMakeLists.txt              # main C++ build entry (cmake -B build here)
│   ├── include/manipsim/
│   ├── src/
│   ├── apps/                       # sources for in-tree demos (scene, iiwa, …)
│   ├── examples/                   # standalone CMake projects (installed lib)
│   │   ├── scene/
│   │   ├── iiwa/
│   │   ├── panda/
│   │   └── ur5/
│   └── tests/
└── third_party/mujoco/             # MuJoCo 3.3.6 headers + simulate sources

Path	Role
python/examples/	Python demos (scene.py, iiwa.py, …)
cpp/apps/	Same demos, built with the main CMake project
cpp/examples/<name>/	Same demos as downstream projects via find_package(manipsim)

Requirements

Component	Python	C++
MuJoCo	3.3.6 (pip/conda)	3.3.6 (bundled under third_party/mujoco/lib/ or pip/conda)
Python	≥ 3.10	—
NumPy	≥ 1.23	—
Compiler	—	C++17 (GCC/Clang)
Graphics	OpenGL display (for viewer)	OpenGL + GLFW (fetched by CMake)
Linear algebra	—	Eigen3 (system or fetched)

---

Python

Install

conda env create -f environment.yml
conda activate manipsim
pip install -e "python/.[dev]"

Or, if the environment already exists:

conda activate manipsim
pip install -e "python/.[dev]"

Single robot

import numpy as np
from manipsim import IIWA

robot = IIWA(render=True)

try:
    while robot.is_running():
        q, dq = robot.state()
        M, h = robot.dynamics()

        tau = np.zeros(robot.nu)   # your controller here
        robot.control(tau)
        robot.step()
finally:
    robot.close()

Multi-robot scene

import numpy as np
from manipsim import IIWA, Panda, Scene

iiwa = IIWA()
panda = Panda()

scene = Scene(model_name="dual_arm_scene", render=True)
scene.add_ground()
scene.add_box("target_box", pos=[0.55, 0.0, 0.05], half_size=[0.04, 0.04, 0.04])
scene.add_robot(iiwa, pos=[0.0, -0.4, 0.0])
scene.add_robot(panda, pos=[0.0, 0.4, 0.0])

try:
    while scene.is_running():
        iiwa.control(np.zeros(iiwa.nu))
        panda.control(np.zeros(panda.nu))
        scene.update()
finally:
    scene.close()

Examples

python python/examples/iiwa.py
python python/examples/panda.py
python python/examples/ur5.py
python python/examples/scene.py      # dual-arm scene + PD control

Tests

cd python && PYTHONPATH="" python -m pytest

If a ROS/Humble shell sets PYTHONPATH, use PYTHONPATH="" to avoid plugin import errors.

Python API summary

Class / method	Description
IIWA, Panda, UR5	Robot presets pointing at assets/robots/*/robot.xml
Scene	Build arena, attach robots and primitives, shared update()
robot.nq, nv, nu, dt	DOF counts and timestep
robot.state()	(q, dq) for this robot only
robot.dynamics()	(M, h) mass matrix and bias forces
robot.control(tau)	Set actuator torques (length nu)
robot.step()	Advance one timestep (standalone)
scene.update()	Advance shared scene one timestep
scene.get_object_position(name)	World-frame body position

Asset paths resolve from the repo root (assets/). Override with:

export MANIPSIM_ASSETS_DIR=/path/to/assets

---

C++

Build

From cpp/ (recommended):

cd cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j16

You can still configure from the repo root (cmake -B build); that forwards to cpp/, but binaries land under build/cpp/ instead of cpp/build/.

CMake locates MuJoCo in this order:

1. -DMUJOCO_DIR=... (manual override)
2. third_party/mujoco/lib/ (bundled)
3. $HOME/mujoco-3.3.6
4. Python pip/conda install (mujoco==3.3.6)

If auto-detection fails:

cmake -B build -DMUJOCO_DIR=$(python -c "import mujoco, pathlib; print(pathlib.Path(mujoco.__path__[0]))")
cmake --build build -j16

Run (in-tree apps)

Sources live in cpp/apps/; targets are defined in cpp/CMakeLists.txt and mirror python/examples/:

App	Command
Dual-arm scene + primitives	./build/scene
Single IIWA	./build/iiwa
Single Panda	./build/panda
Single UR5	./build/ur5
IIWA + Robotiq gripper	./build/iiwa_with_gripper
Panda + Hand	./build/panda_with_gripper
Headless smoke test	./build/test_binding

(Paths assume you built from cpp/ as above.)

Install

User prefix (no sudo):

cd cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/.local
cmake --build build -j16
cmake --install build

System prefix (default /usr/local, usually needs sudo):

cd cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j16
sudo cmake --install build

Installed artifacts:

• libmanipsim_core.so, libmanipsim_viewer.so → $PREFIX/lib/
• Headers → $PREFIX/include/manipsim/
• MJCF assets → $PREFIX/share/manipsim/assets/
• CMake package → $PREFIX/lib/cmake/manipsim/

If you change CMAKE_INSTALL_PREFIX, remove build/ first so CMake does not reuse a cached prefix.

Consumer examples (cpp/examples/)

Each example is its own CMake project (same logic as cpp/apps/, but links the installed library):

cpp/examples/
├── scene/    # dual IIWA + Panda + primitives
├── iiwa/
├── panda/
└── ur5/

After cmake --install, build and run (set CMAKE_PREFIX_PATH to your install prefix):

cd cpp/examples/scene
cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=$HOME/.local
cmake --build build
./build/scene

Repeat for iiwa, panda, or ur5 (executable name matches the folder). For a system install, use -DCMAKE_PREFIX_PATH=/usr/local or omit it if CMake finds the package automatically.

Use in your own project:

find_package(manipsim 0.1 REQUIRED COMPONENTS viewer)
target_link_libraries(my_app PRIVATE manipsim::viewer)

More detail: cpp/examples/README.md.

Real-time simulation

The C++ PassiveViewer syncs simulation time to wall clock using MuJoCo's standard CPU/sim alignment (same approach as the official simulate app):

manipsim::PassiveViewer viewer(scene.model(), scene.data(), "dual_arm_scene");

viewer.run(
    [&]() {
        // Called before each mj_step — set torques only.
        iiwa.control(tau_iiwa);
        panda.control(tau_panda);
    },
    /*realtime_factor=*/1.0);   // 1.0 = 100% real-time (1 sim sec = 1 wall sec)

realtime_factor	Speed
1.0	Real-time (100%)
0.5	Half speed
2.0	2× faster

The viewer UI also has Run/Pause and a Real-time % slider (set to 100% for true 1:1).

C++ API summary

Class / method	Description
manipsim::IIWA, Panda, UR5	Robot presets
manipsim::Scene	addGround(), addBox/Sphere/Cylinder(), addRobot(), compile()
robot.state()	Eigen::VectorXd pairs (q, dq)
robot.dynamics()	Eigen (M, h)
robot.control(tau)	Set torques
PassiveViewer	Real-time passive viewer wrapping mujoco::Simulate

---

Troubleshooting

C++ app hangs or is killed at startup

The MuJoCo viewer requires Load() and RenderLoop() on separate threads. Use PassiveViewer::run() as shown above — do not call Simulate::Load() before the render loop is running.

Wrong libmujoco loaded at runtime

If LD_LIBRARY_PATH points to an old MuJoCo install, it overrides the bundled library:

unset LD_LIBRARY_PATH
./cpp/build/scene                              # in-tree app (built from cpp/)
./cpp/examples/scene/build/scene               # consumer example (after install)

Check which library is loaded:

ldd ./cpp/build/scene | grep mujoco
ldd cpp/examples/scene/build/scene | grep mujoco

MuJoCo header/library version mismatch

Python and C++ must both use 3.3.6. The C++ app prints an error and exits if versions disagree.

Python tests fail with ROS plugin errors

Use PYTHONPATH="" when running pytest (see Tests above).

---

License

MIT