Add KaNN format support

[zylfried]

Hi 👋,

We've added support for .kann files to visualize KaNN graphs, a format developed by Kalray Inc. for accelerating ML computations with our MPPA processors.

Compatibility with Netron is essential for both our employees and customers but adapting an existing format do not address the specific requirements for handling KaNN's images and layers, among other things.

kann.js, kann-metadata.json, kann-schema.js have been added in source/ as well as kann.fbs and the script to process it in tools/. Examples are also available in test/models.json.

Could you please review our pull request?

Let us know if you have any comments or need further information.

Thanks!

[lutzroeder]

but adapting an existing format do not address the specific requirements for handling KaNN's images and layers, among other things.

Can you elaborate on what these requirements are? For example, would it be possible to expose these as operator extensions or metadata in ONNX instead of introducing yet another graph format? What process is used to add this information to KaNN files?

[lutzroeder]

This file should not be checked in to this repo. Instead download from and official repo which always contains the latest version and documentation for this format. Follow tflite implementation as an example, e.g. Kalray should formally take ownership of .fbs content and publish and maintain it under an open source license.

[zylfried]

done

[lutzroeder]

Remove from ./tools and move the source file to an officially maintained repository for this format. For example, "netron-test-models" is not the official main location to learn about this format, signals Netron as the owner of this format which is not the case.

[zylfried]

In progress, but this will result in the .fbs not being accessible for now as the repo is currently private

[lutzroeder]

Great to hear. We can block merging this pull request until this work is done.

[zylfried]

but adapting an existing format do not address the specific requirements for handling KaNN's images and layers, among other things.

Can you elaborate on what these requirements are? For example, would it be possible to expose these as operator extensions or metadata in ONNX instead of introducing yet another graph format? What process is used to add this information to KaNN files?

First, the current ONNX implementation doesn't meet the modularity needed to fully address KaNN's data type requirements. Additionally, not all KaNN layers have corresponding ONNX equivalents. For instance, custom layers like cluster_split or specific attributes of KaNN’s convolution layers don't exist in ONNX.
But most importantly, KaNN treats tensors as equivalent to layers, which necessitates a shift in how they are categorized. In KaNN, layers and tensors need to exist at the same hierarchical level, with Netron’s typical tensors replaced by arcs representing subviews of KaNN’s tensors.

It's important to clarify that KaNN is an inference engine similar to TensorRT, so the graphs we handle are more low-level than ONNX models. We have our own internal model format, and we'd like to use Netron for visualizing it but the difference in structure and purpose makes adapting existing formats insufficient to meet KaNN’s specific needs.

[lutzroeder]

the modularity needed to fully address KaNN's data type requirements

Can you provide an example or such a type or modularity need?

not all KaNN layers have corresponding ONNX equivalents

ONNX can act as a container with an entirely custom set of operators provided via a "domain".

KaNN treats tensors as equivalent to layers

Can you elaborate in this. What is an "Arc" and is there some context why this is needed?

[zylfried]

Indeed, we could extend ONNX to support KaNN’s data types and layer types, but the core difference lies elsewhere. KaNN’s purpose is not to introduce yet another framework for neural network design but rather to translate models from popular frameworks into a lower-level, processor-executable format. The .kann graph we aim to visualize in Netron represents an intermediate stage of model optimization and refactorization. This stage is then compiled into processor-level kernels, which is the final usable format but could not be visualized in Netron, the intermediate format being therefore used for visualization purposes only.

KaNN treats tensors as equivalent to layers

Can you elaborate in this. What is an "Arc" and is there some context why this is needed?

Unlike ONNX, where layers and tensors are distinct (layers as nodes and tensors as arcs), KaNN manipulates tensors as individual graph nodes, much like layers. The input/output relationships between these nodes are then not whole tensors but subviews (smaller parts of the tensors). These subviews correspond to the arcs in our graph, with an Arc being a connection between nodes, as defined in graph theory. Adopting the ONNX format would not work effectively and information would not be displayed properly (subviews' information being different from standard tensors').

the modularity needed to fully address KaNN's data type requirements

Can you provide an example or such a type or modularity need?

Although ONNX implementation does support many data types and could potentially meet our needs at one point, our KaNN implementation allows us to use any data type (whether as a single value or a list) at ANY point in the code without needing to specify individual compatibility for different parts (e.g. nodes, quantization...). This corresponds to our goal of improving our models in the near future to accept a wider variety of data types without having to modify Netron with each update.

Hope this helps you understand our need for a specific implementation of KaNN.

[zylfried]

Hi 👋,

The official model zoo repo is now public, and we should be in line with your requests for adding KaNN compatibility to Netron.

[lutzroeder]

@zylfried running ./tools/kann sync schema creates an invalid file, missing the namespace? Instead of a third_party folder in the model zoo can the .fbs file be move to a more official folder or repo with some format documentation.

[zylfried]

Indeed the namespace was missing, it should be working now.
The thing about this .fbs file is that the format used for Netron isn't the final official serialized format as explained in previous answers. Since it's an intermediate serialization for now used for Netron visualization, keeping it in third_party seems appropriate. The official format is documented in our main repo.

[lutzroeder]

@zylfried can you elaborate and share a link to the previous answer you are referring to as well as a link to the documentation for the official format? Why is there a different serialization format instead of using the official one? Is this format only used for visualization purposes?

[zylfried]

Yes this is for now only used for visualization, here are some of my previous answers :

It's important to clarify that KaNN is an inference engine similar to TensorRT, so the graphs we handle are more low-level than ONNX models. We have our own internal model format, and we'd like to use Netron for visualizing it but the difference in structure and purpose makes adapting existing formats insufficient to meet KaNN’s specific needs.

KaNN’s purpose is not to introduce yet another framework for neural network design but rather to translate models from popular frameworks into a lower-level, processor-executable format. The .kann graph we aim to visualize in Netron represents an intermediate stage of model optimization and refactorization. This stage is then compiled into processor-level kernels, which is the final usable format but could not be visualized in Netron, the intermediate format being therefore used for visualization purposes only.

More documentation is available on the model zoo repo if you're interested in our work https://github.com/kalray/kann-models-zoo

[lutzroeder]

@zylfried instead of introducing another custom format only for visualization, what about exploring exporting to one of the existing 50+ formats the app supports or working on improving the built-in message format. If the files in the model zoo are not actual .kann models and only temporary intermediate files used for Netron visualization there seems a lot less reason to introduce yet another new format and maintain support for it.

[zylfried]

I thought we'd already answered this topic, better explanation should be in our previous messages if you want, plus .kann or .kgraph are not temporary and may grow in importance over time but for now we'd like to use them for visualization to improve our process.

[lutzroeder]

The current implementation of arcs is not supported in Netron. e.g. using string manipulation to represent information that Netron can't process is not the right approach. The question seems more if there is a common denominator. For example, can the graph be presented in another format. The rendered graph looks like an ONNX graph in the current implementation, there aren't any restrictions which operators ONNX can represent. Can extra information be treaded as metadata. For example, the current use of shape is not an actual shape, should it be annotated as metadata instead as it is not a construct the tool understands. Given the format is only used temporarily and there are many redundant formats already, what options have you explored to avoid introducing and maintaining a new format.

[lutzroeder]

nodes are then not whole tensors but subviews (smaller parts of the tensors). These subviews correspond to the arcs in our graph, with an Arc being a connection between nodes, as defined in graph theory

Is there more documentation available explaining the motivations behind this design. The current implementation seems to reduce this to strings and is misusing tensor shapes, is the implementation actually relevant?