DTDL-Generalize messages with message types and related schemas

[jonmikeli]

Hi,

Is there any plan to generalize message types and replace the telemetry type by a generic message structure with, for instance:

• messageType (ex: telemetry, error, system, etc)
• messageSchema
• messageSchemaVersion
  and allowing a device (or module) to send different types of messages.

Unless I have misunderstood something, this is not possible today.
I have been confronted with this need in different projects and it is actually quite useful.

[jonmikeli]

I had missed the object type in the schema property of the telemetry type.

schema property values:

object type:

An example:

Source:
https://github.com/Azure/IoTPlugandPlay/tree/master/DTDL

I guess this should allow sending "messages" of different types through telemetry.

Any plan to add JSON schema references to the object type (ideally with the schema version)?

[briancr-ms]

Yes, the intention of DTDL's telemetry is to enable messages of nearly any type to be defined. These could be telemetry messages that contain measurements sent every minute, like temperature and humidity. These could be telemetry messages that are sent only when an event occurs, like when a door opens or an error occurs.

We hadn't planned on adding JSON Schema support, but can you elaborate more on what you're thinking, perhaps with an example?

[jonmikeli]

Thanks @briancr-ms.
Sure, no problem to elaborate.

The idea behind my question was to control the structure of incoming messages and only accept in the system the valid ones (in terms of schema and allowed values, if specified in the schema).
It adds consistency to the whole solution and minimizes the data being stored and going through the processes in the solution.

I have coded that feature with Azure Functions in many projects and it surprised me how useful it has been. If you need code examples I can send you a few (I am also cleaning a private repository to publish it and it will contain the mentioned feature with many others).

[briancr-ms]

I understand the value of validating incoming messages based on the DTDL telemetry definitions. The intention of DTDL is to capture enough information so that validators like you suggest can be built. In general, it should be possible to generate a JSON Schema from a DTDL definition.

[ahernandez-allegion]

@jonmikeli @briancr-ms

I completely agree with this initiative. I believe Telemetry should be a semantic variation on the fundamental capability of a Message. Furthermore, Event should be another semantic type of a Message capability.

There are huge fundamental differences between Telemetry, Events, and Messages. Telemetry can be dropped. Events cannot. An Event can be Telemetry. Telemetry is not necessarily an Event.

There are also significant temporal differences around timestamping, which is also what differentiates a basic Message from the other two (Messages don't necessarily have a time component).

Replacing Telemetry with a more fundamental Message (or Event) type and adding Telemetry and Event would also be more inline with other DSLs:

https://www.w3.org/TR/wot-thing-description/#thing
https://www.eclipse.org/ditto/basic-feature.html
https://webthings.io/schemas/
https://docs.aws.amazon.com/thingsgraph/latest/ug/iot-tg-models-tdm-core.html

See also:

#20
#10

[jrdouceur]

I realize that this is a rather old discussion, but in case it is still relevant....

The idea behind my question was to control the structure of incoming messages and only accept in the system the valid ones (in terms of schema and allowed values, if specified in the schema).

The latest DTDL parser provides support for this type of validation. There is a tutorial that illustrates the feature.

[jonmikeli]

Hi @jrdouceur ,

Thanks for the update. I read about the Parsing/Validation feature some time ago, which is great. At that time, DTDL v2 had some limitations in terms of what could be defined (and how). I need to assess again all this with DTDL v3.