Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ AI adaptation guides:
- English: [`docs/AI_ASSISTANT_DEPLOYMENT.md`](./docs/AI_ASSISTANT_DEPLOYMENT.md)
- 中文:[`docs/AI_ASSISTANT_DEPLOYMENT.zh-CN.md`](./docs/AI_ASSISTANT_DEPLOYMENT.zh-CN.md)
- AI debug runbook: [`docs/AI_DEBUG_RUNBOOK.md`](./docs/AI_DEBUG_RUNBOOK.md)
- IPC integration: [`docs/IPC.md`](./docs/IPC.md)
- IPC 集成说明: [`docs/IPC.zh-CN.md`](./docs/IPC.zh-CN.md)

## What This Project Does

Expand Down
2 changes: 2 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ AI 适配指南:
- English: [`docs/AI_ASSISTANT_DEPLOYMENT.md`](./docs/AI_ASSISTANT_DEPLOYMENT.md)
- 中文:[`docs/AI_ASSISTANT_DEPLOYMENT.zh-CN.md`](./docs/AI_ASSISTANT_DEPLOYMENT.zh-CN.md)
- AI 调试 Runbook:[`docs/AI_DEBUG_RUNBOOK.md`](./docs/AI_DEBUG_RUNBOOK.md)
- IPC integration: [`docs/IPC.md`](./docs/IPC.md)
- IPC 集成说明:[`docs/IPC.zh-CN.md`](./docs/IPC.zh-CN.md)

## 这个项目解决什么问题

Expand Down
129 changes: 129 additions & 0 deletions docs/IPC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# IPC Integration

VibeMouse exposes two IPC paths:

- Built-in `agent <-> listener` transport: `stdio + LPJSON`
- External control transport: stable local endpoint for command-only clients

The message schema is defined in [`shared/schema/ipc.schema.json`](../shared/schema/ipc.schema.json).

## External Command Endpoint

External programs should connect to the stable local endpoint exposed by the agent:

- Windows: `\\.\pipe\vibemouse`
- Linux/macOS: `${XDG_RUNTIME_DIR:-temp}/vibemouse.sock`

The current endpoint is also written to `status.json` as `ipc_socket`.

Notes:

- External clients should send `command` messages only.
- External clients should not send `event` messages.
- The built-in listener path still uses `stdio`; it does not use the external endpoint.

## Framing

All IPC messages use LPJSON:

1. 4-byte little-endian unsigned length prefix
2. UTF-8 JSON payload

Example payload:

```json
{"type":"command","command":"toggle_recording"}
```

## Message Types

Two message shapes are supported:

### Command Message

```json
{"type":"command","command":"shutdown"}
```

### Event Message

```json
{"type":"event","event":"mouse.side_front.press"}
```

`event` messages are intended for the listener-to-agent path. External integrations should generally use `command`.

## Supported Commands

| Command | Description |
|---|---|
| `noop` | No operation; event is ignored |
| `toggle_recording` | Start or stop voice recording |
| `trigger_secondary_action` | In idle: send Enter. In recording: stop and send transcript to OpenClaw |
| `submit_recording` | Stop recording and send transcript to OpenClaw |
| `send_enter` | Send Enter key to the focused input |
| `workspace_left` | Switch workspace left |
| `workspace_right` | Switch workspace right |
| `reload_config` | Reload `config.json` |
| `shutdown` | Gracefully shut down the agent |

Canonical source: [`shared/protocol/COMMANDS.md`](../shared/protocol/COMMANDS.md)

## Supported Events

| Event | Description |
|---|---|
| `mouse.side_front.press` | Front side button press |
| `mouse.side_rear.press` | Rear side button press |
| `hotkey.record_toggle` | Recording toggle hotkey |
| `hotkey.recording_submit` | Recording submit hotkey |
| `gesture.up` | Upward gesture |
| `gesture.down` | Downward gesture |
| `gesture.left` | Leftward gesture |
| `gesture.right` | Rightward gesture |

Canonical source: [`shared/protocol/EVENTS.md`](../shared/protocol/EVENTS.md)

## Client Examples

### Python: Unix Socket

```python
import json
import socket
import struct

endpoint = "/tmp/vibemouse.sock"
payload = {"type": "command", "command": "toggle_recording"}
body = json.dumps(payload).encode("utf-8")
frame = struct.pack("<I", len(body)) + body

with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as conn:
conn.connect(endpoint)
conn.sendall(frame)
```

### Python: Windows Named Pipe

```python
import json
import struct

endpoint = r"\\.\pipe\vibemouse"
payload = {"type": "command", "command": "toggle_recording"}
body = json.dumps(payload).encode("utf-8")
frame = struct.pack("<I", len(body)) + body

with open(endpoint, "r+b", buffering=0) as pipe:
pipe.write(frame)
```

## Runtime Discovery

If you do not want to hardcode the endpoint, read `status.json` and inspect:

- `ipc_socket`: stable local endpoint
- `listener_mode`: `inline`, `child`, or `off`
- `state`: `idle`, `recording`, or `processing`

Default status file path is platform/runtime dependent and configured by `runtime.status_file`.
129 changes: 129 additions & 0 deletions docs/IPC.zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# IPC 集成说明

VibeMouse 当前有两条 IPC 通路:

- 内建 `agent <-> listener`:`stdio + LPJSON`
- 外部控制通道:面向第三方程序的稳定本地 endpoint,仅接收命令

消息 schema 定义见 [`shared/schema/ipc.schema.json`](../shared/schema/ipc.schema.json)。

## 外部命令通道地址

外部程序集成时,应该连接 agent 暴露出来的稳定本地地址:

- Windows: `\\.\pipe\vibemouse`
- Linux/macOS: `${XDG_RUNTIME_DIR:-temp}/vibemouse.sock`

agent 也会把当前地址写入 `status.json` 的 `ipc_socket` 字段。

注意:

- 外部客户端应只发送 `command` 消息
- 外部客户端不应发送 `event` 消息
- 内建 listener 链路仍然走 `stdio`,不会走这个外部 endpoint

## 帧格式

所有 IPC 消息都使用 LPJSON:

1. 前 4 字节:小端无符号整数,表示后续 JSON 长度
2. 后续内容:UTF-8 JSON payload

示例:

```json
{"type":"command","command":"toggle_recording"}
```

## 消息类型

支持两种消息结构:

### Command Message

```json
{"type":"command","command":"shutdown"}
```

### Event Message

```json
{"type":"event","event":"mouse.side_front.press"}
```

`event` 主要用于 listener -> agent。外部集成一般只需要发送 `command`。

## 当前支持的命令

| Command | 说明 |
|---|---|
| `noop` | 空操作 |
| `toggle_recording` | 开始或停止录音 |
| `trigger_secondary_action` | 空闲时发送 Enter,录音时停止并提交到 OpenClaw |
| `submit_recording` | 停止录音并提交到 OpenClaw |
| `send_enter` | 向当前焦点输入框发送 Enter |
| `workspace_left` | 切换到左侧工作区 |
| `workspace_right` | 切换到右侧工作区 |
| `reload_config` | 重新加载 `config.json` |
| `shutdown` | 优雅关闭 agent |

规范来源:[`shared/protocol/COMMANDS.md`](../shared/protocol/COMMANDS.md)

## 当前支持的事件

| Event | 说明 |
|---|---|
| `mouse.side_front.press` | 前侧键按下 |
| `mouse.side_rear.press` | 后侧键按下 |
| `hotkey.record_toggle` | 录音切换热键 |
| `hotkey.recording_submit` | 录音提交热键 |
| `gesture.up` | 上手势 |
| `gesture.down` | 下手势 |
| `gesture.left` | 左手势 |
| `gesture.right` | 右手势 |

规范来源:[`shared/protocol/EVENTS.md`](../shared/protocol/EVENTS.md)

## 客户端示例

### Python: Unix Socket

```python
import json
import socket
import struct

endpoint = "/tmp/vibemouse.sock"
payload = {"type": "command", "command": "toggle_recording"}
body = json.dumps(payload).encode("utf-8")
frame = struct.pack("<I", len(body)) + body

with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as conn:
conn.connect(endpoint)
conn.sendall(frame)
```

### Python: Windows Named Pipe

```python
import json
import struct

endpoint = r"\\.\pipe\vibemouse"
payload = {"type": "command", "command": "toggle_recording"}
body = json.dumps(payload).encode("utf-8")
frame = struct.pack("<I", len(body)) + body

with open(endpoint, "r+b", buffering=0) as pipe:
pipe.write(frame)
```

## 运行时发现方式

如果不想在客户端里硬编码地址,可以读取 `status.json`,关注这些字段:

- `ipc_socket`:稳定本地 IPC 地址
- `listener_mode`:`inline` / `child` / `off`
- `state`:`idle` / `recording` / `processing`

`status.json` 的默认路径由 `runtime.status_file` 决定。
6 changes: 3 additions & 3 deletions tests/core/test_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,12 @@ def test_set_recording_status_writes_idle_payload(self) -> None:
{"recording": False, "state": "idle", "listener_mode": "inline"},
)

def test_set_recording_status_includes_ipc_port_when_command_server_is_running(self) -> None:
def test_set_recording_status_includes_ipc_socket_when_command_server_is_running(self) -> None:
subject = self._make_subject()
with tempfile.TemporaryDirectory(prefix="vibemouse-status-") as tmp:
status_file = Path(tmp) / "status.json"
setattr(subject, "_config", SimpleNamespace(status_file=status_file))
setattr(subject, "_command_server", SimpleNamespace(port=43125))
setattr(subject, "_command_server", SimpleNamespace(endpoint="test://vibemouse"))

set_status = cast(
Callable[[bool], None],
Expand All @@ -131,7 +131,7 @@ def test_set_recording_status_includes_ipc_port_when_command_server_is_running(s
self.assertEqual(
payload,
{
"ipc_port": 43125,
"ipc_socket": "test://vibemouse",
"listener_mode": "inline",
"recording": False,
"state": "idle",
Expand Down
49 changes: 29 additions & 20 deletions tests/core/test_output.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,10 @@ def fake_run(*args: object, **kwargs: object) -> SimpleNamespace:
captured_timeouts.append(timeout)
return SimpleNamespace(returncode=0, stdout="1\n")

with patch("vibemouse.system_integration.subprocess.run", side_effect=fake_run):
with (
patch("vibemouse.system_integration.sys.platform", "linux"),
patch("vibemouse.system_integration.subprocess.run", side_effect=fake_run),
):
subject = self._make_subject()
probe = cast(Callable[[], bool], getattr(subject, "_is_text_input_focused"))
call_probe: Callable[[], bool] = probe
Expand All @@ -57,25 +60,31 @@ def fake_run(*args: object, **kwargs: object) -> SimpleNamespace:
self.assertTrue(result)
self.assertEqual(captured_timeouts, [1.5])

@patch(
"vibemouse.system_integration.subprocess.run",
side_effect=subprocess.TimeoutExpired(
cmd=["python3", "-c", "..."], timeout=1.5
),
)
def test_focus_probe_timeout_returns_false(self, _mock_run: object) -> None:
subject = self._make_subject()
probe = cast(Callable[[], bool], getattr(subject, "_is_text_input_focused"))
self.assertFalse(probe())

@patch(
"vibemouse.system_integration.subprocess.run",
side_effect=OSError("spawn failed"),
)
def test_focus_probe_oserror_returns_false(self, _mock_run: object) -> None:
subject = self._make_subject()
probe = cast(Callable[[], bool], getattr(subject, "_is_text_input_focused"))
self.assertFalse(probe())
def test_focus_probe_timeout_returns_false(self) -> None:
with (
patch("vibemouse.system_integration.sys.platform", "linux"),
patch(
"vibemouse.system_integration.subprocess.run",
side_effect=subprocess.TimeoutExpired(
cmd=["python3", "-c", "..."], timeout=1.5
),
),
):
subject = self._make_subject()
probe = cast(Callable[[], bool], getattr(subject, "_is_text_input_focused"))
self.assertFalse(probe())

def test_focus_probe_oserror_returns_false(self) -> None:
with (
patch("vibemouse.system_integration.sys.platform", "linux"),
patch(
"vibemouse.system_integration.subprocess.run",
side_effect=OSError("spawn failed"),
),
):
subject = self._make_subject()
probe = cast(Callable[[], bool], getattr(subject, "_is_text_input_focused"))
self.assertFalse(probe())

def test_focus_probe_prefers_system_integration_result(self) -> None:
subject = self._make_subject()
Expand Down
Loading
Loading