Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(knx): Enable KNX Sensor Setup with a Schema-Driven Approach #136293

Draft
wants to merge 19 commits into
base: dev
Choose a base branch
from

Conversation

philippwaller
Copy link

@philippwaller philippwaller commented Jan 22, 2025

Breaking change

Proposed change

Overview

This PR is a follow-up to PR #133979 and focuses on enabling the creation of a KNX sensor directly from the Home Assistant UI. It leverages and demonstrates a new, schema-driven approach to manage KNX device configurations more flexibly. Instead of maintaining extensive, hard-coded metadata in both the integration and the frontend, the goal is to define the data schema in the backend and expose it to the UI via voluptuous-serialize. This eliminates the need for frequent frontend adaptations and accelerates the transition from YAML to a modern UI-driven KNX integration.

Problem Statement

To deliver great user experience for managing KNX UI entities, we need a significant amount of metadata—such as required fields or pre-populated dropdown values. Currently, adding new KNX devices necessitates changes in both the integration and the frontend. This creates unnecessary complexity and slows down the shift toward a fully UI-based configuration model.

While additional endpoints could technically provide this metadata, they add considerable integration overhead. We need a more elegant, scalable way to define and share schema details between the integration and the UI.

Proposed Solution

  1. Backend-Centric Schema Definition
    Define the data schema (and relevant metadata) in the KNX integration itself.

  2. Serialization for the Frontend
    Use voluptuous-serialize to publish the schema to the KNX frontend. This allows dynamic UI rendering—such as automatic form generation, required-field indicators, dropdown options, and field grouping. (see get_entity_schemas_response.json)

  3. UI Rendering from Schema
    Let the frontend derive its structure from the provided schema, eliminating the need for repeated manual updates whenever new device types are introduced (as long as the UI supports the schema definitions).

  4. Device Configuration Objects
    Introduce a config object for each device type (e.g., SensorConfig) that can serialize and deserialize data from YAML, UI input, and a standardized internal format. By abstracting these data sources behind a single object, cosmetic changes to the UI schema—like moving a field from “basic” to “advanced”—won’t affect the core data structure.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

Open Tasks:

  • Clarify the structure for custom translations in strings.json (the config_panel node is too restrictive).
  • Check which (options) translations are unnecessary (some may be covered by built-in selectors like SensorStateClass, DeviceClass, and EntityCategory). If a native solution exists, use a marker NEW description to exclude these translations.
  • Create a WebSocket endpoint for editing schema-driven platforms.
  • Add test cases for all newly introduced elements.
  • Improve docstrings to make the intention of the new classes clearer.
  • Perform final code cleanup.

@home-assistant
Copy link

Hey there @Julius2342, @farmio, @marvin-w, mind taking a look at this pull request as it has been labeled with an integration (knx) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of knx can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign knx Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

Copy link
Contributor

@farmio farmio Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This class was moved.
Only difference is self.entry.async_on_unload(self.entry.add_update_listener(async_update_entry)) which is now done in async_setup_entry().

Please avoid mixing big refactoring / code moving changes and new feature in PRs in the future. Having multiple smaller PRs makes it easier to review - especially if a reviewer is not familiar with the integration code.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you. The change was necessary due to a cyclic dependency. Otherwise, it wouldn’t have been possible to access the config object in all required locations.

homeassistant/components/knx/entity.py Outdated Show resolved Hide resolved
homeassistant/components/knx/entity.py Outdated Show resolved Hide resolved
homeassistant/components/knx/entity.py Outdated Show resolved Hide resolved
Comment on lines 70 to 71
prop: GroupAddressConfig.from_dict(data[prop])
if isinstance(data[prop], dict)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A ConfigGroup can consist of more than just GroupAddressConfig instances, can't it?


return {
"type": "group_address_config",
"properties": convert(value.build_schema()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the resulting schema looks quite complex

{
  "type": "group_address_config",
  "properties": [
    {
      "type": "group_address",
      "allow_none": false,
      "allow_internal_address": true,
      "name": "state",
      "required": true
    },
    {
      "type": "group_address_list",
      "items": {
        "type": "group_address",
        "allow_none": false,
        "allow_internal_address": true
      },
      "name": "passive",
      "optional": true,
      "default": []
    },
    {
      "type": "dpt",
      "properties": [
        {
          "name": "main",
          "required": true
        },
        {
          "allow_none": true,
          "name": "sub",
          "optional": true,
          "default": null
        }
      ],
      "options": [
        ...
      ],
      "name": "dpt",
      "required": true
    }
  ],
  "name": "ga_sensor",
  "required": true
},

for a selector that is already working in frontend that expect something like (written out of the head - have not checked)

{
    "type": "knx_ga_schema",
    "write": false,
    "write_required": false,
    "state": true,
    "state_required": true,
    "passive": true,
    "dpt": [
        ...
    ],
    "name": "ga_sensor",
    "required": true,
}

Since frontend GA selectors are custom anyway, I don't think we have to overcomplicate these.
As resulting config we need not more than eg.

"ga_switch": {
  "write": "1/1/45",
  "state": "1/0/45",
  "passive": []
  "dpt": "5.001"
},

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was to design all objects in a uniform way, so that additional functions—like generating translation keys based on the schema's structure—can work without needing special handling for different types.

homeassistant/components/knx/schema.py Outdated Show resolved Hide resolved
homeassistant/components/knx/sensor.py Show resolved Hide resolved
homeassistant/components/knx/websocket.py Show resolved Hide resolved
homeassistant/components/knx/websocket.py Outdated Show resolved Hide resolved
@home-assistant home-assistant bot marked this pull request as draft January 23, 2025 20:28
@home-assistant
Copy link

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@philippwaller
Copy link
Author

philippwaller commented Jan 30, 2025

Here's the latest update:

To ensure we provide all necessary translations according to the schema, I’ve added unit tests that confirm:

All required translations exist, and no unnecessary translations are left dangling in the strings file.
In addition, these tests generate a ready-to-use JSON skeleton (shown in the test logs). This way, if we introduce new platforms, fields, or changes to the schema, we can simply copy and paste the relevant parts rather than replicating the entire schema in the strings file by hand.

However, there’s a caveat:
We need a place to store KNX-specific frontend translations. I’ve been using the config_panel node in the strings file, which initially seemed like the right spot for custom translations. Unfortunately, the translation schema is quite restrictive—hassfest doesn’t allow the nested dictionaries our approach requires (see strings.json). Ideally, the translation schema would mirror our data schema, much like native config flows. But to achieve that, we need more flexibility/nested elements.

So, my question is:
Is there a possibility to relax the current schema constrains or introduce a new node specifically for integration-related frontend translations.

PS: I added a to-do list to the PR description to transparently track progress.

@farmio
Copy link
Contributor

farmio commented Jan 30, 2025

I sent you some messages on Discord regarding translations. Imho the whole translation topic would best be handled separately in a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants