Add Home Assistant community ability

[harmsolo13]

🆕 New Community Ability

Ability Name: community/home-assistant-voice-control

What does it do?

Voice-controlled interface for Home Assistant via REST API. Control lights, switches, covers, sensors, sirens, media players, and shopping lists — all hands-free. Uses LLM-based intent classification with fuzzy entity name matching so users can say natural phrases like "turn off the kitchen light" or "add milk to my shopping list" without knowing exact entity IDs.

Key features:

• Connects to any Home Assistant instance via Long-Lived Access Token
• Auto-discovers all actionable entities (lights, switches, covers, sensors, sirens, media players, todo lists)
• LLM classifies user commands and fuzzy-matches entity names (e.g. "the floodlight" → light.camera_1_floodlight)
• Safety confirmation prompts for dangerous actions (covers, sirens)
• State checking without API calls ("is the front door locked?")
• Shopping list management via Home Assistant's todo integration
• Conversation loop with natural exit words

Demo

🎬 Watch the demo video on Loom

Suggested Trigger Words

• "home assistant"
• "smart home"
• "control my home"
• "home control"
• "turn on the lights"
• "turn off the lights"
• "check my home"
• "house control"

Example conversation

User: "Smart home"
OpenHome: "Home Assistant connected. I found 24 devices across 5 categories. What would you like to do?"
User: "Turn on the floodlight"
OpenHome: "Turning on Camera 1 Floodlight."
User: "Is there motion at the front door?"
OpenHome: "No motion detected at the front door."
User: "Open the gate"
OpenHome: "Are you sure you want to open the gate?"
User: "Yes"
OpenHome: "Opening the driveway gate."
User: "Add cat food to the shopping list"
OpenHome: "Added cat food to your shopping list."
User: "Done"
OpenHome: "Home Assistant control ended. Have a good one."

APIs Required

• Home Assistant REST API — Long-Lived Access Token required. Generate at http://YOUR_HA_IP:8123/profile → Long-Lived Access Tokens. Free, no rate limits for local instances. Set via HA_TOKEN in .env file alongside main.py.

Type

• New community Ability
• Improvement to existing Ability
• Bug fix
• Documentation update

Checklist

• main.py extends MatchingCapability with register_capability
• README.md exists with description + setup instructions
• No hardcoded API keys or secrets
• Tested on OpenHome device/simulator
• Files only in community/home-assistant-voice-control/
• resume_normal_flow() called on every exit path (in finally block)
• No print() — using editor_logging_handler
• No blocked imports (redis, connection_manager, user_config)
• No asyncio.sleep() or asyncio.create_task() — using session_tasks
• Error handling on all external calls with timeouts
• Demo video recorded and linked above

Anything else?

LLM Intent Classification: Uses the platform's built-in text_to_text_response with a structured system prompt. Returns JSON with action, entity_id, service_data, and a natural spoken response. Handles markdown code fences in LLM output gracefully.

Safety confirmations: Actions like open_cover, close_cover, activate_siren, and deactivate_siren require voice confirmation via run_confirmation_loop() before execution.

Entity registry: Builds a compact entity list from HA's /api/states endpoint, filtered to actionable domains only (light, switch, cover, media_player, siren, binary_sensor, sensor, todo). This keeps LLM context focused and fast.

[github-actions]

✅ Community PR Path Check — Passed

All changed files are inside the community/ folder. Looks good!

[github-actions]

🔀 Branch Merge Check

PR direction: add-home-assistant-voice-control → dev

✅ Passed — add-home-assistant-voice-control → dev is a valid merge direction

[github-actions]

🔍 Lint Results

✅ __init__.py — Empty as expected

Files linted: community/home-assistant-voice-control/main.py

✅ Flake8 — Passed

✅ All checks passed!

[github-actions]

✅ Ability Validation Passed

📋 Validating: community/home-assistant-voice-control
  ✅ All checks passed!

[abubakar4360]

Please share your demo if you can.

[harmsolo13]

Demo link added to the readme

[abubakar4360]

Hey @harmsolo13, the work is great. A few things to fix before this can be approved:

1. EXIT_WORDS is missing too many natural spoken exits to be reliable. ["done", "stop", "exit", "quit", "nevermind", "never mind", "goodbye", "bye", "that's all"] misses: "forget it", "leave it", "I'm good", "that's it", "all done", "no thanks", "close", "end", "actually". Long-term fix: replace the any(word in user_input.lower() for word in EXIT_WORDS) check with a single LLM classifier call — "Is the user trying to stop or exit? Return YES or NO."

2. spoken_response comes directly from LLM output and goes straight into speak() with no guardrails. INTENT_SYSTEM_PROMPT has no instruction on how that string should sound — the model could return "The living room lights have been successfully turned on as requested." or a formatted list and TTS will read every word. Add to the system prompt: "spoken_response must be plain spoken English, no markdown, no bullet points, one sentence under 10 words. Example: 'Living room lights are on.'"

3. The greeting speaks the raw entity count and category count: f"Home Assistant connected. I found {entity_count} devices across {len(registry)} categories. What would you like to do?" — on a real setup with 40+ entities this becomes "I found 47 devices across 8 categories" which is noise the user doesn't need. Change to: "Connected. What would you like to do?"

4. "I couldn't connect to Home Assistant. Please check the connection and try again." — "Home Assistant" is implementation language the user didn't ask to hear. Change to: "Couldn't reach your smart home. Check your connection and try again."

5. "Sorry, I couldn't complete that action. Please check Home Assistant." — same issue, "check Home Assistant" is meaningless to a non-technical user and sounds like an error log. Change to: "Couldn't do that — the device might be unavailable."

6. "I didn't catch that. Could you repeat?" — "Could you repeat?" is overly formal and reads stiffly on TTS. Change to: "Didn't catch that — go ahead."

7. "I had trouble understanding that. Could you try rephrasing?" — same, 10 words where 5 will do. Change to: "Not sure what you mean — try saying it differently."

8. HA_TOKEN = os.environ.get("HA_TOKEN", "YOUR_HOME_ASSISTANT_TOKEN_HERE") — if the env var isn't set, the ability silently sends "YOUR_HOME_ASSISTANT_TOKEN_HERE" as a Bearer token and the API call returns a 401 with no user-facing explanation. Add a startup check: if either HA_TOKEN or HA_URL still contains the placeholder string, speak "This ability isn't configured yet." and return early rather than attempting a doomed API call.

[uzair401]

Hi @harmsolo13, Please address the issues highlighted by @abubakar4360, and then we will proceed further.