feat(dictation): mouse middle/side buttons with Hold refcount (split from #724) by HKLHaoBin · Pull Request #727 · Open-Less/openless

HKLHaoBin · 2026-06-21T09:42:22Z

User description

摘要

Fixes #718（部分：鼠标触发）。

由已关闭的 #724 拆分而来（第 2/3 部分）：独立 prefs 开关；Win/macOS 全局鼠标 hook；Hold 模式 HoldSourceTracker 多源 refcount。Linux 输入走 PR3 evdev。可与 #726 并行审查/合并。

关联：#718、#724。

修复 / 新增 / 改进

鼠标中键 / 侧键听写开关与 i18n
mouse_dictation + hold_source_tracker
设置变更时 refresh_mouse_dictation

兼容

不包含：side_aware_combo、侧向录制 UI、Linux evdev
Linux 鼠标/侧向组合需 PR3

测试计划

本地实机验证（回应 @jiangmuran CHANGES_REQUESTED）：

Win/macOS：开启中键/侧键 Toggle → Hold 与 Toggle 听写均可触发
Hold refcount：键盘 Hold + 鼠标 Hold 同时按下，分别释放 → 仅最后一个源释放后才结束
Android：cargo metadata --filter-platform aarch64-linux-android（stub 编译）
npm run build

请维护者：在真机验证鼠标触发；Linux 需 PR3 合并后复测。

PR Type

Enhancement, Tests

Description

Add mouse middle/side button dictation triggers
Track multiple Hold sources with refcount
Integrate into coordinator and settings
Add UI toggles and i18n for mouse buttons

Diagram Walkthrough

flowchart LR
  MouseInput["Mouse Events\n(Win/macOS low-level hook)"]
  MouseDict["mouse_dictation\n(Middle/Side Button Detection)"]
  HoldTracker["hold_source_tracker\n(Refcount Multiple Hold Sources)"]
  Coordinator["Coordinator\n(Start/Stop Dictation)"]
  MouseInput --> MouseDict
  MouseDict --> HoldTracker
  HoldTracker --> Coordinator

File Walkthrough

Relevant files

Enhancement

13 files

hold_source_tracker.rs `Implement HoldSourceTracker for multi-source hold`	+125/-0
mouse_dictation.rs `Add global mouse button dictation monitor`	+356/-0
coordinator.rs `Integrate mouse dictation and hold sources`	+182/-2
dictation.rs `Handle pressed/released edges with trigger source`	+44/-16
hotkey_loops.rs `Add mouse dictation bridge and supervisor loops`	+194/-6
hotkey.rs `Hook macOS mouse events for dictation`	+13/-1
lib.rs `Register mouse_dictation module and start listener`	+7/-0
mouse_dictation.rs `Add mobile stub for mouse dictation`	+38/-0
types.rs `Add mouse dictation preference fields`	+16/-0
settings.rs `Add refresh_mouse_dictation to SettingsWriter`	+15/-0
settings.rs `Persist mouse dictation changes and call refresh`	+15/-0
types.ts `Add mouse dictation preference types`	+5/-1
RecordingInputSection.tsx `Add mouse middle/side button toggle UI`	+23/-0

Tests

4 files

mod.rs `Add mouse_dictation_refreshes to test mock`	+5/-0
coordinator.rs `Add hold mode integration tests`	+182/-2
mock-data.ts `Add mouse dictation defaults to mock settings`
stylePrefs.test.ts `Add mouse dictation fields to test prefs`	+2/-0

Documentation

5 files

en.ts `Add English i18n for mouse dictation toggles`	+4/-0
ja.ts `Add Japanese i18n for mouse dictation toggles`	+4/-0
ko.ts `Add Korean i18n for mouse dictation toggles`	+4/-0
zh-CN.ts `Add Chinese i18n for mouse dictation toggles`	+4/-0
zh-TW.ts `Add Traditional Chinese i18n for mouse dictation toggles`	+4/-0

Additional files

1 files

mock-data.ts	+2/-0

Split from Open-Less#724: Win/macOS global mouse hooks, prefs toggles, HoldSourceTracker for multi-source hold release. Linux evdev path deferred to PR3. Fixes Open-Less#718 (partial: mouse triggers). Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T09:44:40Z

PR Reviewer Guide 🔍

(Review updated until commit `8027eca`)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 718 - Partially compliant Compliant requirements: 鼠标中键（滚轮按下）作为独立的语音识别唤起方式开启 / 关闭。增加“鼠标中键唤起识别”开关，独立于键盘快捷键配置。鼠标中键和键盘快捷键可以共存，也可以分别关闭。增加了鼠标侧键（前进/后退）听写开关，同样独立于键盘快捷键。 Non-compliant requirements: (None) Requires further human verification: (None)
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T09:48:32Z

Persistent review updated to latest commit 71787ba

HKLHaoBin · 2026-06-21T09:55:04Z

稍等，我处理一下ci问题。

Felix201209

方向对、拆得也清楚，但有一个会影响使用的 HIGH bug 必须先修，所以暂时 request changes 🙏

🔴 HIGH：按住鼠标键时禁用鼠标听写 → 听写卡死直到重启

mouse_dictation.rs + coordinator/hotkey_loops.rs（update_mouse_dictation_binding_now）

复现（Hold 模式）：

按住鼠标中键 → Pressed → hold_sources.press(MouseMiddle) → count=1 → 听写开始；
此时在设置里把"中键听写"关掉；
update_mouse_dictation_binding_now drop 掉 monitor（ACTIVE_MOUSE=None），但从不重置 hold_sources；
物理松开按键时 handle_button 已经收不到事件了 → handle_released_edge 永不触发 → active_count 永远停在 1；
听写会话一直跑到 app 重启。

修法（二选一）：update_mouse_dictation_binding_now 在禁用分支返回前调用 inner.hold_sources.reset()；或 update_config 清 middle_held/side_held 之前，对当前仍持握的源补发合成 HotkeyEvent::Released。

🟠 与 #726 必然冲突（已批准，建议本 PR rebase 去重）

本 PR 在 types.ts 给 HotkeyTrigger 加了 leftCommand/leftShift/rightShift，并在 5 个语言文件加了对应文案——#726 在同样位置加了同样的行。#726 已经 approve、排在前面合并，所以本 PR 需要 rebase 到最新 beta 并删掉这几处重复新增，否则二合一时会冲突。

其它（不阻断，建议一并处理）

并发 press 的 TOCTOU（coordinator/dictation.rs）：press() 后又单独读 active_count()，键鼠几乎同时按下时两边都读到 2、都提前 return，begin_session 漏调。建议直接用 fetch_add 的返回旧值判断"是否首次按下"。
Linux 上鼠标听写 UI 可设但无效：mouse_dictation.rs 没有 Linux 平台实现，但 RecordingInputSection.tsx 在 Linux 仍显示开关。建议 toggle 加 os !== 'linux' 或 start() 在 Linux 返回错误（Linux 的鼠标触发本就归 #728 的 evdev）。
dead i18n：mouseMiddleDesc/mouseSideDesc 五语言都翻译了，但 SettingRow 没传 description，要么接上要么删掉。

测试缺口

hold_source_tracker / coordinator 的 happy path 测得不错，但缺：①上面"持握中禁用→卡死"的回归测试；②并发同时按下的 TOCTOU。建议至少给 HIGH 那条补个测试。

- Fix hold-mode tests to pass TriggerSource::KeyboardDictation - Hide mouse toggles on Linux until PR3 evdev - Remove PR1 side-modifier type/i18n/mock remnants - Complete mouse pref wire/deserialize/default in types.rs Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T10:11:17Z

Persistent review updated to latest commit 60672bf

github-actions · 2026-06-21T10:15:01Z

Persistent review updated to latest commit df02187

…ents Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T10:17:29Z

Persistent review updated to latest commit 645b35b

…d source On update_config disable or Drop, swap held flags and send Released events so HoldSourceTracker does not retain stale mouse sources after prefs refresh. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T10:22:47Z

Persistent review updated to latest commit 2135949

…andling

github-actions · 2026-06-21T10:34:53Z

Persistent review updated to latest commit 1bf48fe

HKLHaoBin · 2026-06-21T10:35:06Z

@Felix201209 已按 review 处理，请 re-review 🙏

HIGH（持握中禁用鼠标听写）

mouse_dictation 层：Drop / update_config 禁用路径补发 Released（21359494）
coordinator 层：新增 sync_release_mouse_hold_sources，在 update_mouse_dictation_binding_now 与 supervisor 禁用分支、take() 前同步释放 mouse hold 源并在最后一个源释放时 end_session（1bf48fec）

TOCTOU + debounce

HoldSourceTracker::press() 返回递增前的 count；handle_pressed_edge 用 prev_count == 0 判断是否首个源
debounce 丢弃时 rollback hold_sources / hotkey_trigger_held

其它

RecordingInputSection 接上 mouseMiddleDesc / mouseSideDesc
新增回归测试：hold_mode_mouse_disable_while_holding_ends_session、hold_mode_concurrent_press_starts_once
与 feat(hotkey): side-specific modifier combos for dictation (split from #724) #726 重复的 types.ts/i18n 已在先前 commit 去掉；Linux 鼠标开关仍隐藏至 feat(linux): evdev input for side combos and mouse dictation (split from #724) #728

CI 请在 GitHub 上确认。

Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T10:51:36Z

Persistent review updated to latest commit a841adc

Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-06-21T11:06:24Z

Persistent review updated to latest commit e866c41

github-actions · 2026-06-21T11:27:40Z

Persistent review updated to latest commit f718636

…use_hold_sources

github-actions · 2026-06-21T11:42:40Z

Persistent review updated to latest commit 02a3b6f

…nc to avoid nested runtime in tests

github-actions · 2026-06-21T11:54:16Z

Persistent review updated to latest commit c85b192

…ock_on to avoid nested runtime

github-actions · 2026-06-21T12:08:12Z

Persistent review updated to latest commit 0abc2c0

…tokio runtime

github-actions · 2026-06-21T12:14:27Z

Persistent review updated to latest commit ed339a1

…okio worker thread

github-actions · 2026-06-21T13:01:46Z

Persistent review updated to latest commit 8027eca

HKLHaoBin · 2026-06-21T13:31:04Z

@Felix201209 感谢 review 🙏 针对 review #4539504044 里的各点，最新 head 8027ecae 已全部处理，CI 绿。

🔴 HIGH：持握中禁用鼠标听写 → 卡死

采用你建议的「补发合成 Released」方案，并加了 coordinator 侧兜底：

mouse_dictation.rs：Drop / update_config 禁用路径在清 middle_held/side_held 前补发 HotkeyEvent::Released（21359494）
hotkey_loops.rs：新增 sync_release_mouse_hold_sources，在 update_mouse_dictation_binding_now、supervisor 禁用分支、以及 take() 前同步释放仍持握的 mouse 源；最后一个源释放时走 end_session（1bf48fec）

🟠 与 #726 重复

types.ts / 五语言 i18n 里 PR1 的 side-modifier 字段与文案已去掉，本 PR 只保留 mouse 相关（71787bad、60672bf7）。

其它建议

点	处理
TOCTOU	`press()` 返回递增前 count；`handle_pressed_edge` 用 `prev_count == 0` 判首个源；debounce 丢弃时 rollback `hold_sources` / `hotkey_trigger_held`
Linux UI	`showMouseDictation = showDesktopHotkey && os !== 'linux'`，等 #728 evdev
dead i18n	`SettingRow` 已接 `mouseMiddleDesc` / `mouseSideDesc`

测试

hold_mode_mouse_disable_while_holding_ends_session — HIGH 回归
hold_mode_concurrent_press_starts_once — TOCTOU
hold_mode_sync_release_mouse_only_keeps_keyboard_hold — 只释 mouse 时 keyboard hold 不受影响（a841adce）
hold_mode_hotkey_rebind_while_holding_clears_sources_and_ends_session — 持握中改快捷键/模式也会清源（e866c414）

CI 侧 nested-runtime 问题也已修完（f7186364 → 8027eca），Windows / macOS / Linux / Android check 均 pass。

方便的话请再扫一眼，谢谢！

Felix201209

响应得又快又到位 👍 重新逐条核对后，上一轮提的问题全部解决，改判 approve ✅

🔴 HIGH 卡死 bug — 已解决：monitor drop（mouse_dictation.rs 的 Drop → release_held_sources）、disable（sync_release_mouse_hold_sources 先清零再 .take()）、rebind mid-hold（update_hotkey_binding 首步调 clear_active_hold_sources_on_hotkey_rebind）三条路径都会补发 Released / 重置 refcount，且有状态机 + 幂等保护，不会再卡。
🟠 与 #726 的冲突 — 已解决：types.ts 和五个语言文件里重复的 leftCommand/leftShift/rightShift 已移除，本 PR 现在只加 mouseMiddle*/mouseSide*，#726 先合后不会冲突。
并发 press TOCTOU — 已解决：改用 swap(true) 闩 + fetch_add 返回旧值（prev_count==0 才是首次按下），不再有单独 active_count() 读的竞态。
测试 — 已补齐：hold_mode_*（含 disable-mid-hold、rebind-mid-hold、键鼠重叠）四个集成测试 + update_config/Drop 两个单测，正好覆盖之前的缺口。CI 四平台全绿。

一个 LOW nit，留作后续即可、不拦合并：coordinator/hotkey_loops.rs 的 mouse_dictation_supervisor_loop 里那对互补 #[cfg(not(linux))] / #[cfg(linux)] 块体一样、等价于无 cfg，是重构残留，可直接去掉 cfg。

合并顺序提醒：本 PR 依赖 #726 的类型定义，请先合 #726 再合本 PR。另：beta 的 ruleset 只有 @jiangmuran 能 bypass，最终 merge 仍需他来点。

This was referenced Jun 21, 2026

feat(linux): evdev input for side combos and mouse dictation (split from #724) #728

Open

feat(hotkey): 支持左右修饰键与鼠标触发听写 #724

Closed

fix(dictation): keep PR2 types mouse-only without side trigger variants

71787ba

Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions Bot added the Review effort 3/5 label Jun 21, 2026

Felix201209 mentioned this pull request Jun 21, 2026

feat(hotkey): side-specific modifier combos for dictation (split from #724) #726

Open

6 tasks

Felix201209 requested changes Jun 21, 2026

View reviewed changes

Trigger CI

df02187

github-actions Bot added Review effort 4/5 and removed Review effort 3/5 labels Jun 21, 2026

docs(types): drop PR1 side-modifier wording from ShortcutBinding comm…

645b35b

…ents Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions Bot added Review effort 2/5 and removed Review effort 4/5 labels Jun 21, 2026

fix(dictation): emit mouse release when monitor drops or disables hel…

2135949

…d source On update_config disable or Drop, swap held flags and send Released events so HoldSourceTracker does not retain stale mouse sources after prefs refresh. Co-authored-by: Cursor <cursoragent@cursor.com>

feat(dictation): add mouse hold mode tests and improve mouse source h…

1bf48fe

…andling

github-actions Bot added Review effort 3/5 and removed Review effort 2/5 labels Jun 21, 2026

test(dictation): sync_release_mouse preserves keyboard hold source

a841adc

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(dictation): clear hold sources when hotkey binding changes mid-hold

e866c41

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(ci): import HotkeyMode in hotkey_loops.rs

f718636

github-actions Bot added Review effort 4/5 and removed Review effort 3/5 labels Jun 21, 2026

fix(ci): avoid nested runtime in tests by extracting async release_mo…

02a3b6f

…use_hold_sources

fix(ci): extract async clear_active_hold_sources_on_hotkey_rebind_asy…

c85b192

…nc to avoid nested runtime in tests

github-actions Bot added Review effort 3/5 and removed Review effort 4/5 labels Jun 21, 2026

fix(ci): use futures::executor::block_on instead of async_runtime::bl…

0abc2c0

…ock_on to avoid nested runtime

github-actions Bot added Review effort 4/5 and removed Review effort 3/5 labels Jun 21, 2026

fix(ci): use tokio::spawn + mpsc channel instead of block_on when in …

ed339a1

…tokio runtime

github-actions Bot added Review effort 3/5 and removed Review effort 4/5 labels Jun 21, 2026

fix(ci): use std::thread::spawn + handle.block_on to avoid blocking t…

8027eca

…okio worker thread

github-actions Bot added Review effort 4/5 and removed Review effort 3/5 labels Jun 21, 2026

Felix201209 approved these changes Jun 21, 2026

View reviewed changes

Conversation

HKLHaoBin commented Jun 21, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

摘要

修复 / 新增 / 改进

兼容

测试计划

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit 8027eca)

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

HKLHaoBin commented Jun 21, 2026

Uh oh!

Felix201209 left a comment

Choose a reason for hiding this comment

🔴 HIGH：按住鼠标键时禁用鼠标听写 → 听写卡死直到重启

🟠 与 #726 必然冲突（已批准，建议本 PR rebase 去重）

其它（不阻断，建议一并处理）

测试缺口

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

HKLHaoBin commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

HKLHaoBin commented Jun 21, 2026

🔴 HIGH：持握中禁用鼠标听写 → 卡死

🟠 与 #726 重复

其它建议

测试

Uh oh!

Felix201209 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

HKLHaoBin commented Jun 21, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

(Review updated until commit `8027eca`)