Skip to content

fix(insertion): IME-safe accept so Japanese/CJK suggestions land on Tab#668

Merged
FuJacob merged 2 commits into
mainfrom
investigate/japanese-ime-suggestions
Jun 11, 2026
Merged

fix(insertion): IME-safe accept so Japanese/CJK suggestions land on Tab#668
FuJacob merged 2 commits into
mainfrom
investigate/japanese-ime-suggestions

Conversation

@FuJacob

@FuJacob FuJacob commented Jun 11, 2026

Copy link
Copy Markdown
Owner

Summary

With a composing input method active (Japanese kana, Chinese pinyin, Korean hangul), the synthetic Unicode keystroke that commits an accepted suggestion is re-absorbed into the IME's composition instead of landing as literal text, so Tab consumed the suggestion word by word while inserting nothing and the session reset on the next AX poll. This detects composing input sources via Text Input Sources and commits accepts through the pasteboard instead, triggered by an AXPress on the host's real Edit > Paste menu item (located language-independently by its Cmd-V key equivalent, cached per process), so no key event exists for the input method to swallow.

Validation

  • On-device, dev build, macOS Japanese IME and ATOK in a Chrome contenteditable and a native field: ghost suggestions on committed Japanese now insert on Tab, word-by-word accepts advance correctly, and switching back to a Latin layout keeps using the unchanged keystroke path. Verified in the structured logs that the committed text grows by each accepted chunk (it previously stayed frozen while the session advanced).
  • Two earlier approaches were verified broken on-device and are documented in code comments: an AX kAXSelectedTextAttribute write (Chromium reports .success but silently no-ops) and synthetic Cmd-V at the HID tap (the Command flag is merged away against the physically held Tab; 12/12 pastes traversed the tap chain with nothing landing).
  • xcodebuild build-for-testing ... CODE_SIGNING_ALLOWED=NO then test-without-building for CompositionInputModeClassifierTests + SuggestionSessionReconcilerTests: ** TEST SUCCEEDED **, 80 tests, 0 failures.
  • swiftlint lint --quiet on all changed files: exit 0, no findings.

Linked issues

None filed; reported by a Japanese user on Reddit against v0.5.1-beta (suggestions intermittent, Tab accepts vanish without inserting). Happy to link a tracking issue if we open one.

Risk / rollout notes

  • Detection is deliberately conservative: any non-keyboard-layout input source other than the shared Roman/eisu mode (com.apple.inputmethod.Roman) is treated as composing, so unknown third-party IMEs route through the paste path rather than the swallowable keystroke path. Latin keyboard users see zero behavior change.
  • The IME accept path touches the clipboard: it is snapshotted (all representations) and restored 300ms later, with overlapping accepts coalescing onto one restore and a changeCount guard so a user copy during the window is never clobbered. The menu press can also briefly highlight the host's Edit menu title.
  • CI build heads-up, pre-existing and not introduced here: main already requires a cotabbyinference revision with setComputeLogprob (Cut inference and AX hot-path waste: P-core threads, gated logprob, halved KV, geometry cache #667) that is not yet pushed to the package's GitHub main, so the fresh -resolvePackageDependencies in CI fails on main and on this PR alike until the package commit is pushed.

Greptile Summary

Introduces an IME-safe suggestion acceptance path for Japanese, Chinese, Korean, and other composing input methods, fixing a bug where Tab-accepted completions were silently re-absorbed by the input method instead of landing as text.

  • Detection: KeyboardInputSourceMonitor subscribes to kTISNotifySelectedKeyboardInputSourceChanged and caches the result of CompositionInputModeClassifier, which conservatively treats any non-keyboard-layout, non-Roman-eisu source as composing — so unknown third-party IMEs (ATOK, Sogou) default to the safe path rather than the broken keystroke path.
  • Insertion: SuggestionInserter.insert now checks the provider and routes composing sessions through insertViaPaste, which prefers pressing the host's real Edit > Paste menu item via AXPress (language-independently located by Cmd-V key equivalent, cached per PID) over a synthetic Cmd-V, since no key event exists for the IME to intercept.
  • Suppression fix: The observer tap's keyDown handler now also checks isSynthetic(event) as a backstop, preventing the synthetic Cmd-V from being misclassified as a shortcut mutation when a real keystroke races to consume the countdown token.

Confidence Score: 5/5

Safe to merge. Latin keyboard users follow the unchanged code path; the composing-IME path falls back gracefully through menu press → synthetic Cmd-V → keystroke, restoring the clipboard on every failure branch.

The classification rule is pure and fully unit-tested. The AX menu walk is bounded (depth-1 only, early exit on the first Cmd-V item) and every element handle receives a 50 ms messaging timeout before being queried, capping per-accept latency. Clipboard snapshot/restore is guarded against overlapping accepts and user copies. The suppression backstop in the observer tap correctly prevents the synthetic Cmd-V from being misidentified as a user shortcut. No correctness issues were found across all changed files.

No files require special attention.

Important Files Changed

Filename Overview
Cotabby/Support/CompositionInputModeClassifier.swift New pure classification enum; intentionally free of Carbon/TIS for testability. Conservative 'composing' default for unknown non-layout sources is correct and well-documented.
CotabbyTests/CompositionInputModeClassifierTests.swift Thorough test suite covering layout, hiragana, katakana, Chinese, Korean, Roman direct mode, unknown IME, and nil-mode cases. Behavioral contract is fully specified.
Cotabby/Services/Input/KeyboardInputSourceMonitor.swift New @mainactor monitor subscribes to kTISNotifySelectedKeyboardInputSourceChanged, caches result, and uses takeRetainedValue/takeUnretainedValue correctly per TIS ownership rules.
Cotabby/Services/Suggestion/SuggestionInserter.swift IME-safe path gates on isComposingIMEActiveProvider, routes to pressPasteMenuItem then synthetic Cmd-V fallback; clipboard snapshot/restore logic is well-guarded against overlapping pastes.
Cotabby/Support/AXHelper.swift pasteMenuItem walks menu bar to find Cmd-V item language-independently; every element handle gets pollMessagingTimeout before being messaged, preventing main-thread stalls on slow Chromium pages.
Cotabby/Services/Input/InputMonitor.swift Adds isSynthetic identity check as backstop in the observer tap keyDown case; prevents synthetic Cmd-V from being classified as a shortcut mutation when a real key races to consume the suppression countdown token.
Cotabby/App/Core/CotabbyAppEnvironment.swift Wires KeyboardInputSourceMonitor into SuggestionInserter via a weak-captured closure; monitor is stored on env to extend its lifetime for the session.
Cotabby.xcodeproj/project.pbxproj Adds CompositionInputModeClassifier.swift, KeyboardInputSourceMonitor.swift, and CompositionInputModeClassifierTests.swift to all three relevant build targets (Cotabby, Cotabby Dev, CotabbyTests).

Sequence Diagram

sequenceDiagram
    participant User as User (Tab key)
    participant IM as InputMonitor (accept tap)
    participant SI as SuggestionInserter
    participant KIM as KeyboardInputSourceMonitor
    participant AX as AXHelper
    participant App as Host App

    User->>IM: Tab keyDown
    IM->>SI: insert(suggestion)
    SI->>KIM: isComposingIMEActiveProvider()
    KIM-->>SI: true (Japanese IME active)

    SI->>SI: insertViaPaste(text)
    SI->>SI: snapshotPasteboard()
    SI->>SI: pasteboard.setString(text)

    SI->>AX: pasteMenuItem(forApplicationPID:)
    Note over AX: Walk menuBar→topLevelItem→menu→item
    Note over AX: Match Cmd-V by key char+modifiers
    AX-->>SI: "AXUIElement (Edit > Paste)"

    SI->>App: AXUIElementPerformAction(kAXPressAction)
    App->>App: Execute paste command (no key event)
    App-->>SI: .success

    Note over SI: Cache menu item per pid
    SI->>SI: Schedule clipboard restore (300ms)
    SI-->>IM: true (inserted)

    Note over SI,App: 300ms later
    SI->>SI: restorePasteboard (if changeCount unchanged)
Loading

Comments Outside Diff (1)

  1. Cotabby/Services/Suggestion/SuggestionInserter.swift, line 252-261 (link)

    P2 DispatchWorkItem restore closure weak-self bookkeeping

    The closure calls self?.clearPendingPasteboardRestore() via a weak capture. If self is released before the 300 ms fires, pendingPasteboardRestore and savedClipboardForRestore are never nilled, leaving stale in-flight bookkeeping. The clipboard itself is still restored correctly (saved is captured by value). In practice SuggestionInserter lives for the app's lifetime so this is currently harmless, but worth noting for maintenance.

    Fix in Codex Fix in Claude Code

Fix All in Codex Fix All in Claude Code

Reviews (2): Last reviewed commit: "Address Greptile review on #668" | Re-trigger Greptile

Comment thread Cotabby/Services/Input/KeyboardInputSourceMonitor.swift Outdated
Comment thread Cotabby/Support/AXHelper.swift
Comment thread Cotabby/Support/AXHelper.swift
FuJacob added 2 commits June 11, 2026 00:11
…d on Tab

With a composing input method active (Japanese kana, Chinese pinyin, Korean
hangul), the synthetic Unicode keystroke that commits an accepted suggestion
is re-absorbed into composition by the IME instead of landing as literal
text. The session then desyncs against the live field and the suggestion
clears and regenerates, which Japanese users reported as Tab consuming the
suggestion word by word while inserting nothing.

Detect composing input sources via Text Input Sources (TIS), cached and
refreshed on the input-source-changed distributed notification, with the
classification kept in a pure helper. While one is active, route the commit
through the pasteboard and trigger the paste by AXPress on the host's real
Edit > Paste menu item, located language-independently by its Cmd-V key
equivalent and cached per process. No key event exists on that path, so the
IME can neither swallow nor re-interpret the commit. Synthetic Cmd-V remains
as a fallback for hosts without a Paste menu item, posted with a session
event source at the annotated session tap so its Command flag survives (the
HID tap merges flags with the live hardware state, and during a Tab accept
only Tab is physically down; observed on-device as 12/12 pastes traversing
the tap chain with nothing landing). The user's clipboard is snapshotted and
restored around the paste, and the observer tap now also recognizes synthetic
events by identity so a raced suppression token cannot let our own fallback
Cmd-V classify as a shortcut mutation and tear down the session it is
committing.
Make KeyboardInputSourceMonitor.refresh() private (the notification path is
the only caller), apply the short poll messaging timeout to every element the
paste-menu walk messages so a stalled host cannot hold the main actor for the
multi-second AX default, and make the missing-modifiers rejection in
isCommandVMenuItem an explicit guard instead of relying on nil == 0.
@FuJacob FuJacob force-pushed the investigate/japanese-ime-suggestions branch from 4fec121 to 2ec2fd9 Compare June 11, 2026 14:14
@FuJacob

FuJacob commented Jun 11, 2026

Copy link
Copy Markdown
Owner Author

Addressed all three review findings in 2ec2fd9: refresh() is now private, the paste-menu walk applies the 50ms poll timeout to every element it messages (menu bar, top-level items, menus, and items), and the missing-modifiers case is an explicit guard. Local validation on the rebased branch: xcodegen no drift, swiftlint clean, build-for-testing succeeded, full unit bundle 982 tests / 0 failures.

@FuJacob FuJacob merged commit 4305253 into main Jun 11, 2026
4 checks passed
@FuJacob FuJacob deleted the investigate/japanese-ime-suggestions branch June 11, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant