External Key Controls

[gl4diator]

When I test this keyboard with my movie projector, it seems I can’t type using the remote control — I'm forced to use the mouse to click on the keys, which makes it feel more like a touch-only keyboard. Is there a way to enable typing directly with the remote control buttons or through another method? Also there's no mouse hover event on the keys while I'm crossing it with the pointer.

[Helium314]

From what I found in the code, hover events should occur when the AccessibilityManager is enabled. No idea where to set it on your device.

How does the remote control interact with the keyboard?

[gl4diator]

From what I can tell, the interaction between the remote and the keyboard likely depends on how the device processes input events. If the remote only sends directional or click events (like Up/Down/Left/Right and Select), then the keyboard might not receive hover or focus events unless accessibility features are enabled – just as you mentioned.

It's possible that the remote is being treated more like a mouse or gamepad rather than a physical keyboard. That’s why you can’t type directly using the remote — it’s virtually "clicking" on the screen instead of sending key character events.

One solution might be to manually enable "Accessibility Services" (if available on your device), or you may need to update your app to handle DPAD/KEYCODE events specifically for remote controls. It depends on whether the keyboard you're testing was designed for remote input or only for touch interaction. I can't find yet this setting on my device but..

I tested with Gboard keyboard as well and it works fine and probably works with AOSP Android keyboard system.. Check my uploaded video: https://we.tl/t-RbBvu8ZD0K

[Helium314]

or you may need to update your app to handle DPAD/KEYCODE events

So remote control sends key events? Then you should better tell which version of Heliboard you are using, because there were changes in 3.3 and 3.6-beta1.

[gl4diator]

or you may need to update your app to handle DPAD/KEYCODE events

So remote control sends key events? Then you should better tell which version of Heliboard you are using, because there were changes in 3.3 and 3.6-beta1.

I'm using 3.5 just updated

[gl4diator]

I assume the relevant code is mainly in:

latin/LatinIME.java (handling KeyEvents and IME lifecycle)

keyboard/KeyboardView.java (drawing the keyboard and handling touch/hover)

keyboard/internal/KeyboardState.java and KeyboardActionListenerImpl.kt (keyboard state and key actions).

Other keyboards like Gboard and the AOSP system keyboard already work with my remote, so the remote seems to send standard DPAD / hover events. HeliBoard, however, appears to rely mostly on touch events.

To support remote control navigation, it seems the app would need:

In LatinIME.onKeyDown() / onKeyUp(): explicit handling for KEYCODE_DPAD_UP/DOWN/LEFT/RIGHT/CENTER, mapping them to “move focus between keys” and “press focused key” instead of ignoring them.

In KeyboardView: support for a focused key (drawn with a highlight) and the ability to move that focus programmatically (e.g. from KeyboardState) rather than only reacting to touch coordinates.

Optionally, better onHoverEvent() / onGenericMotionEvent() handling, so pointer events from the remote (mouse / touchpad-like input) can also be used without relying on Accessibility being enabled.

If remote support for the emoji panel is desired as well, EmojiPalettesView would probably need similar focus / DPAD handling.

Does that sound like the right area of the code to look into, or is there another place where input from remotes is handled?

[Helium314]

If your remote inputs result in key press events, it's in LatinIME (onKeyDown / onKeyUp)

[gl4diator]

If your remote inputs result in key press events, it's in LatinIME (onKeyDown / onKeyUp)

Yes, an example of code for the events would be like: @OverRide
public boolean onKeyDown(int keyCode, KeyEvent event) {
switch (keyCode) {
case KeyEvent.KEYCODE_DPAD_UP:
moveFocusUp();
return true;
case KeyEvent.KEYCODE_DPAD_DOWN:
moveFocusDown();
return true;
case KeyEvent.KEYCODE_DPAD_LEFT:
moveFocusLeft();
return true;
case KeyEvent.KEYCODE_DPAD_RIGHT:
moveFocusRight();
return true;
case KeyEvent.KEYCODE_DPAD_CENTER:
case KeyEvent.KEYCODE_ENTER:
pressFocusedKey();
return true;
default:
return super.onKeyDown(keyCode, event);
}
}

private void moveFocusUp() {
// Logic navigation between keys or layout
keyboardView.moveFocus(Direction.UP);
}

private void moveFocusDown() {
keyboardView.moveFocus(Direction.DOWN);
}

private void moveFocusLeft() {
keyboardView.moveFocus(Direction.LEFT);
}

private void moveFocusRight() {
keyboardView.moveFocus(Direction.RIGHT);
}

private void pressFocusedKey() {
keyboardView.pressFocusedKey(); // Simulation of the key pressed
}

[Helium314]

Are you experimenting with this code, or is this just an idea?

[gl4diator]

Are you experimenting with this code, or is this just an idea?

Just an idea 😄. Wish I could test it, but I'm not sure how yet. I haven't thought about it.

[gl4diator]

Alright.. I'll get things ready with Android Studio and this project. I'll test the code on my own and let you know if it works I guess.

[Helium314]

Oh, since you say it works with AOSP keyboard: did you test whether it works in OpenBoard? If yes, we could try to pin the change that broke it for HeliBoard

[gl4diator]

Oh, since you say it works with AOSP keyboard: did you test whether it works in OpenBoard? If yes, we could try to pin the change that broke it for HeliBoard

I've tried OpenBoard version 1.4.5 but it doesn't work.. When I try to select any keys it's not focusing but just disappears. Check my videos, I've already did it by modifying the code in HeliBoard: https://we.tl/t-UR65AC74cs
2 prolems left with my tested version of Heli:

1. It's not focusing on the suggested words.
2. It's not focusing on the emoji keyboard.

[gl4diator]

I've tested my version of HeliBoard debug on my phone and works pretty well there's no bugs with the new code implemented. 😄

[Helium314]

Hey this is really great!
You could do a PR, so maybe I can help with the remaining issues (and maybe find a way to fake a remote control so I can test this).