Skip to content

feat(hotkey): 支持左右修饰键与鼠标触发听写#724

Closed
HKLHaoBin wants to merge 4 commits into
Open-Less:betafrom
HKLHaoBin:feat/input-trigger-enhancement
Closed

feat(hotkey): 支持左右修饰键与鼠标触发听写#724
HKLHaoBin wants to merge 4 commits into
Open-Less:betafrom
HKLHaoBin:feat/input-trigger-enhancement

Conversation

@HKLHaoBin

@HKLHaoBin HKLHaoBin commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

User description

摘要

Fixes #718

桌面端听写触发增强:听写 start/stop 支持左右侧修饰键组合(含 Fn 等单键预设)、鼠标中键/侧键独立开关、Linux evdev 输入层与 Hold 多触发源 refcount;非听写全局快捷键仍走原有 global-hotkey 路径,并拒绝侧向修饰键绑定。

修复 / 新增 / 改进

  • 侧向修饰键(仅听写 start/stop)SideAwareComboMonitor + side_aware_combo 状态机;Win/macOS 物理键监听,Linux 走 evdev;ShortcutRecorder 默认录制 generic 修饰键,听写入口启用 sideSpecificModifiers
  • 鼠标触发:中键/侧键独立 pref;Win/macOS hook + Linux evdev;HoldSourceTracker 多源 Hold refcount
  • Linux evdev 热更新:prefs 变更时 refresh_linux_evdev_monitor,避免 side_combo / 鼠标开关切换后配置残留
  • 冲突与校验bindings_overlap 按物理修饰键类比较(Windows 上 cmd-leftsuper 冲突、与 cmd/Ctrl 不冲突);QA/翻译/切风格/打开应用/Less Computer 拒绝侧向组合
  • 移动端:stub 模块保持 Android 编译兼容,不启用桌面热键能力
  • 测试:前端 hotkeySideModifiers.test.ts;Rust 单测覆盖 overlap、macOS keycode、evdev 配置、Windows Shift 侧向组合 dispatch

兼容

  • 不包含:Android/iOS 真实侧向热键或鼠标听写(仅 stub)
  • 对现有用户 / 本地环境 / 构建流程的影响:
    • 已有听写快捷键不变;新能力为增量 pref/录制选项
    • 若本地 QA 等快捷键曾保存无效侧向组合,重新保存会被新校验拦截
    • Linux 鼠标/侧向组合需 /dev/input 读权限(input 组)

测试计划

  • 命令:npm run check:hotkey-side-modifiersnpm run buildcargo metadata(desktop + aarch64-linux-android
  • 结果:fork CI 全绿 — https://github.com/HKLHaoBin/openless/actions/runs/27880599990(Linux / macOS / Windows / Android cargo check + Rust 单测)
  • 请仓库维护者下载构建产物进行验证(如有问题或有其他需求请自行修改本 PR):
    • 桌面:checkout 本 PR 分支后本地 npm run build + cargo tauri build(或合并后在 release workflow 产物上验证)
    • Android 编译:可 dispatch Android APK (debug) workflow 于本分支,确认 stub 链路仍通过
    • 手工清单(建议)
      • 听写:cmd-left+D / Left Command 单键 / 鼠标中键 Hold-Toggle
      • 设置切换:鼠标-only → 侧向组合 → 泛化组合,Linux 上 evdev 应热更新
      • Windows:shift-left+D 侧向听写;QA 快捷键录制应为 super+D 而非误存 cmd+D
      • 冲突:cmd-left+Dsuper+D 应提示冲突;与 ctrl+D 不应冲突

PR Type

Enhancement, Tests


Description

  • Support left/right modifier keys in dictation hotkey (Cmd, Shift, Ctrl, Alt, Fn)

  • Add independent mouse middle/side button dictation triggers

  • Implement Linux evdev input layer for side/combo and mouse events

  • Refcount multiple hold sources for Hold mode dictation


Diagram Walkthrough

flowchart LR
  Input["Input Events (Keyboard/Mouse)"]
  Platform["Platform Hooks\nWindows/macOS low-level\nLinux evdev"]
  SideCombo["side_aware_combo\n(Left/Right Modifier Matching)"]
  MouseDict["mouse_dictation\n(Middle/Side Button Press)"]
  HoldTracker["hold_source_tracker\n(Refcount Multiple Sources)"]
  Coordinator["Coordinator\n(Start/Stop Dictation)"]
  Input --> Platform
  Platform --> SideCombo
  Platform --> MouseDict
  SideCombo --> HoldTracker
  MouseDict --> HoldTracker
  HoldTracker --> Coordinator
Loading

File Walkthrough

Relevant files
Enhancement
14 files
side_aware_combo.rs
Side-specific modifier matching for dictation hotkeys       
+568/-0 
linux_evdev_input.rs
Linux evdev input for side modifiers and mouse buttons     
+481/-0 
hotkey_loops.rs
Integrate side-aware combo, mouse dictation, Linux evdev loops
+208/-10
coordinator.rs
Add new monitors and hold source tracker to Coordinator   
+116/-6 
shortcut_binding.rs
Add side modifier validation, overlap detection, normalization
+228/-1 
hotkey.rs
Extend macOS/Windows hooks to dispatch side and mouse events
+66/-3   
mouse_dictation.rs
Global mouse middle/side button dictation monitor               
+244/-0 
hotkeys.rs
Reject side-specific modifiers for non-dictation shortcuts
+81/-18 
types.rs
Add new preferences and HotkeyTrigger variants                     
+36/-1   
dictation.rs
Update pressed/released edge handling for multi-source hold
+39/-16 
hold_source_tracker.rs
Refcount multiple dictation trigger sources for Hold mode
+125/-0 
settings.rs
Add refresh_mouse_dictation to settings persistence           
+15/-0   
linux_fcitx.rs
Map new HotkeyTriggers to Linux key symbols                           
+6/-0     
lib.rs
Add new modules and start mouse dictation listener             
+17/-0   
Other
4 files
shortcut_binding.rs
Stub new functions for mobile builds                                         
+26/-0   
side_aware_combo.rs
Stub side-aware combo for mobile                                                 
+45/-0   
mouse_dictation.rs
Stub mouse dictation for mobile                                                   
+38/-0   
linux_evdev_input.rs
Stub Linux evdev input for mobile                                               
+32/-0   
Tests
3 files
backend_rust.rs
Include side_aware_combo module in test harness                   
+6/-0     
hotkeySideModifiers.test.ts
Frontend tests for side modifier hotkey recording               
+75/-0   
stylePrefs.test.ts
Add new mouse dictation prefs to test data                             
+2/-0     
Additional files
16 files
package.json +1/-0     
Cargo.toml +1/-0     
combo_hotkey.rs +1/-1     
mod.rs +5/-0     
qa.rs +1/-0     
ShortcutRecorder.tsx +64/-23 
en.ts +8/-0     
ja.ts +8/-0     
ko.ts +8/-0     
zh-CN.ts +8/-0     
zh-TW.ts +8/-0     
hotkey.ts +108/-2 
mock-data.ts +5/-0     
types.ts +9/-2     
RecordingInputSection.tsx +18/-0   
ShortcutsSection.tsx +2/-0     

HKLHaoBin and others added 4 commits June 21, 2026 15:14
• 引入了由鼠标中键和侧键触发听写的新用户偏好设置。
• 更新了 UserPreferences 和 UserPreferencesWire 结构体,以包含鼠标按键听写设置。
• 增强了 ShortcutRecorder 组件,支持区分左右侧的修饰键,并增加了单键快捷键预设。
• 更新了英文、日文、韩文、简体中文和繁体中文的新功能翻译。
• 改进了热键处理逻辑,增加了对 Command、Shift 和 Control 键左右变体的支持。
• 为新的热键功能和侧边修饰键添加了单元测试。
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

718 - Partially compliant

Compliant requirements:

  • 支持配置左/右侧修饰键(cmd-left, cmd-right, Fn等)
  • 增加鼠标中键唤起识别开关(mouse_middle_button_dictation)
  • 增加鼠标侧键唤起识别开关(mouse_side_button_dictation)
  • 键盘快捷键与鼠标触发可独立启用/禁用
  • 快捷键录制支持sideSpecificModifiers

Non-compliant requirements:

Requires further human verification:

  • 无(所有要求均可通过代码审查验证)
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

@jiangmuran jiangmuran left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请在提交AI Slop前至少先在本地进行完整实机测试。

@Felix201209

Copy link
Copy Markdown
Collaborator

感谢这块工作,方向是对的(修 #718,CI 全绿、也带了测试)。不过这个 PR 先关闭一下 🙏

原因:体量过大 —— 2788 行改动、38 个文件,单个 PR 很难做到安全、彻底的审查与回归;而且目前还挂着一条未解决的 CHANGES_REQUESTED。

建议拆成几个互相独立、各自可审查可回滚的小 PR,例如:

  1. 侧向修饰键(side_aware_combo / SideAwareComboMonitor + ShortcutRecordersideSpecificModifiers
  2. 鼠标中键 / 侧键触发(mouse_dictation + HoldSourceTracker refcount)
  3. Linux evdev 输入层与热更新(linux_evdev_input + refresh_linux_evdev_monitor

每块单独提交后会好审很多,欢迎拆分后重新发 PR。关闭不代表否定这个方向,只是当前体量和就绪度的问题。

@HKLHaoBin

HKLHaoBin commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

按审查意见拆成 3 个可独立审查、可回滚的小 PR:

  1. 侧向修饰键(Win/macOS)https://github.com/Open-Less/openless/pull/726(\ eat/718-side-modifiers\)
  2. 鼠标中/侧键 + Hold 多源 refcounthttps://github.com/Open-Less/openless/pull/727(\ eat/718-mouse-dictation\)
  3. Linux evdev + 热更新https://github.com/Open-Less/openless/pull/728(\ eat/718-linux-evdev\,含 1+2,补齐 Linux 完整 [feature] 支持左/右修饰键快捷键与鼠标中键唤起语音识别 #718

各 PR 的 Test plan 已列出实机验证项,回应 @jiangmuran 的 CHANGES_REQUESTED。原单体分支仍保留在 fork 的 \origin/feat/input-trigger-enhancement\。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature] 支持左/右修饰键快捷键与鼠标中键唤起语音识别

3 participants