Adding feature in Visualizations

[HackPoint]

Hi, I need to add a feature called: Workspace which will represent a folder with dependent files which are Kemletes.

I've added a select and need to add new workspace button, the question is how to add a data structure and backend capabilities to this?

I would extend the question after I'll understand that this is possible at all.

[lordrip]

Hi @HackPoint, it will depend on what you're trying to do. Could you describe the goal of this feature a bit more? For instance, is this something that you want to see in Kaoto or are you trying to base something else out of Kaoto?

Thanks for opening this discussion

[HackPoint]

First of all thank you for a quick answer, I will refine my question , after a short discussion with my development partner.

[HackPoint]

And another question: If don't use VSCode do you have solutions which will allow me to extend the Kaoto application?

[lordrip]

First of all thank you for a quick answer, I will refine my question , after a short discussion with my development partner.
Absolutely, I'll stay tuned

And another question: If don't use VSCode do you have solutions which will allow me to extend the Kaoto application?
Yes, we build Kaoto in 3 flavors:

1. As a VSCode extension
2. A standalone online playground
3. A npm package

The latter exports Kaoto components with the purpose of adapting Kaoto to different needs, nevertheless, we always welcome contributions, so depending on the change or feature type, we could think of ways of adding them directly to Kaoto upstream.

[HackPoint]

I'm developing an extension to this application, not using it only for usage as it is.

[lordrip]

Sure, then the npm package would help you the most, in any case, feel free to expand on your idea if needed, to check if it makes sense to incorporate it in Kaoto

[mhempleman]

Hi All. Let me provide some context. We are in the process of customizing Kaoto for our domain and use case. This includes:

• Adding our custom Camel components to the catalog (done).
• As we're not using the VSCode plugin, we want to add basic workspace functionality such that the user can point the UI at a directory and get the list of the Kamelets, routes, workflows... etc.
• Adding a workflow development tab - similar to the routes development tab, but for our custom Camel workflows.
• Blacklisting/removing unwanted core components and Kamelets.
• Packaging the Java code generator with the UI so we can update the catalog dynamically.

[mhempleman]

Hi lordip. Thanks for the quick reply. I can't really get into specifics at this point, but we are not working on an open-source project. However, we would absolutely be interested in contributing upstream.

[mhempleman]

Hi @mhempleman, let me tackle each item individually:

Adding our custom Camel components to the catalog (done).

Do you mean adding custom user components and perhaps removing existing ones from Camel?

We have a set of custom components we built for our domain. Our components build publishes the JSON definition files, which are then consumed by the Kaoto code generator and added to the UI catalog. If you're interested, we'd be happy to generalize this feature and push it upstream.

As we're not using the VSCode plugin, we want to add basic workspace functionality such that the user can point the UI at a directory and get the list of the Kamelets, routes, workflows... etc.

Yup, that can be done with the aforementioned library that bootstraps a git repository as a filesystem in the browser, I've seen it used in other tools, but I'm not 100% familiar with its API. In a nutshell, you provide your Git credentials and then you gain access to the files in it.

That's great! Thanks. We'll check it out.

Adding a workflow development tab - similar to the routes development tab, but for our custom Camel workflows.

I think you can identify those custom Camel workflows and then filter them out from the Design tab, and only show them in a different one, that's what we're doing for the Beans entities, for instance.

Blacklisting/removing unwanted core components and Kamelets.

I think this can be done while adding support for specifying a Kamelets folder or location

I actually have a PR related to this point to review this afternoon. It's an update to the code generator to support an exclusion list command line argument. Ex.... "aws,twitter". Do you think it makes more sense to filter out unwanted entities at the UI level.

Packaging the Java code generator with the UI so we can update the catalog dynamically.

This will need a more in-depth discussion 😄

😄 I bet. We haven't put much thought into it at this point. We're still in the POC phase. Maybe packaging the code generator is not the best option.

[lordrip]

Sounds great @mhempleman, I'd like to propose tackling each item individually, or at least the ones that are closer to each other.

I think that having a mechanism to add custom components to the catalog would be a great addition, please feel free to submit a PR at any time, we're open to contributions.

For Kamelets, I think we can follow the same approach as custom components, once that's done, perhaps we could move to generate the catalog on the fly, it will depend on where are you in terms of your POC. I think both topics are compatible with Kaoto's goal, so they can be added upstream 🤔

That being said, let me know if it makes sense or if you have another idea.

[mhempleman]

@lordrip I created a "work in progress" PR with a few questions. Please take a look when you have a few minutes.. Thanks!

[lordrip]

Looking at it

[HackPoint]

@lordrip Hi, I have a question, I need to add to DSL another type of YAML template. Let's call it - "Workflows".

FLOW: After I click the DSLSelectorToggle: it's changing the template to a "Workflows" instead of the existing ones:Camel Route, Kamelet, Pipe.

I need to extend Kamelet to a Workflows and create co-depended YAML.

I would like to understand which flows needed to be added to a current implementation to create a new View for Workflows?

[lordrip]

Hi @HackPoint, I'm not sure if I'm following correctly, could you please share your code to see what you mean?

[HackPoint]

I still haven't wrote any code, cause I still don't understand where do I need to add the code.

See screenshot:

[lordrip]

I need to extend Kamelet to a Workflows and create co-depended YAML.

I don't understand what this mean, I'm sorry 😞

I still haven't wrote any code, cause I still don't understand where do I need to add the code.

mmm, maybe sharing some mocks about what you want to achieve?, how the user journey would look like?

As you mentioned, the DSLSelectorToggle component is what renders the list, you can inspect the list and add or remove what you need from the list, let's say, you don't want to have Camel Routes, you can certainly remove them from the list.

Here's how the code looks like:

  const dslEntriesRef = useRef<Partial<Record<SourceSchemaType, ISourceSchema>>>({
    [SourceSchemaType.Route]: sourceSchemaConfig.config[SourceSchemaType.Route],
    [SourceSchemaType.Kamelet]: sourceSchemaConfig.config[SourceSchemaType.Kamelet],
    [SourceSchemaType.Pipe]: sourceSchemaConfig.config[SourceSchemaType.Pipe],
  });

[HackPoint]

Sorry for me being so abstract, let explain.

I need to add a flow to Visualization:

Expected behavior:

1. Select "Workflows" from the DSLSelectToggle (simple click on it)
2. I need the YAML to generate exactly the same YAML as "kamelet.ts" which is inside: visualization/flows/templates/kamelet.ts

apiVersion: camel.apache.org/v1
kind: Workflow
metadata:
  name: ${workflowId}
  annotations:
    camel.apache.org/kamelet.support.level: "Stable"
    camel.apache.org/catalog.version: "main-SNAPSHOT"
    camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iaXNvLTg4NTktMSI/Pgo8IS0tIEdlbmVyYXRvcjogQWRvYmUgSWxsdXN0cmF0b3IgMTkuMC4wLCBTVkcgRXhwb3J0IFBsdWctSW4gLiBTVkcgVmVyc2lvbjogNi4wMCBCdWlsZCAwKSAgLS0+CjxzdmcgdmVyc2lvbj0iMS4xIiBpZD0iQ2FwYV8xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIiB4PSIwcHgiIHk9IjBweCIKCSB2aWV3Qm94PSIwIDAgNjAgNjAiIHN0eWxlPSJlbmFibGUtYmFja2dyb3VuZDpuZXcgMCAwIDYwIDYwOyIgeG1sOnNwYWNlPSJwcmVzZXJ2ZSI+CjxwYXRoIGQ9Ik00OC4wMTQsNDIuODg5bC05LjU1My00Ljc3NkMzNy41NiwzNy42NjIsMzcsMzYuNzU2LDM3LDM1Ljc0OHYtMy4zODFjMC4yMjktMC4yOCwwLjQ3LTAuNTk5LDAuNzE5LTAuOTUxCgljMS4yMzktMS43NSwyLjIzMi0zLjY5OCwyLjk1NC01Ljc5OUM0Mi4wODQsMjQuOTcsNDMsMjMuNTc1LDQzLDIydi00YzAtMC45NjMtMC4zNi0xLjg5Ni0xLTIuNjI1di01LjMxOQoJYzAuMDU2LTAuNTUsMC4yNzYtMy44MjQtMi4wOTItNi41MjVDMzcuODU0LDEuMTg4LDM0LjUyMSwwLDMwLDBzLTcuODU0LDEuMTg4LTkuOTA4LDMuNTNDMTcuNzI0LDYuMjMxLDE3Ljk0NCw5LjUwNiwxOCwxMC4wNTYKCXY1LjMxOWMtMC42NCwwLjcyOS0xLDEuNjYyLTEsMi42MjV2NGMwLDEuMjE3LDAuNTUzLDIuMzUyLDEuNDk3LDMuMTA5YzAuOTE2LDMuNjI3LDIuODMzLDYuMzYsMy41MDMsNy4yMzd2My4zMDkKCWMwLDAuOTY4LTAuNTI4LDEuODU2LTEuMzc3LDIuMzJsLTguOTIxLDQuODY2QzguODAxLDQ0LjQyNCw3LDQ3LjQ1OCw3LDUwLjc2MlY1NGMwLDQuNzQ2LDE1LjA0NSw2LDIzLDZzMjMtMS4yNTQsMjMtNnYtMy4wNDMKCUM1Myw0Ny41MTksNTEuMDg5LDQ0LjQyNyw0OC4wMTQsNDIuODg5eiIvPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8Zz4KPC9nPgo8L3N2Zz4K"
    camel.apache.org/provider: "Apache Software Foundation"
    camel.apache.org/kamelet.group: "Users"
  labels:
    camel.apache.org/kamelet.type: "source"
spec:
  definition:
    title: ${workflowId}
    description: "Produces periodic events about random users!"
    type: object
    properties:
      period:
        title: Period
        description: The time interval between two events
        type: integer
        default: 5000
  types:
    out:
      mediaType: application/json
  dependencies:
    - "camel:timer"
    - "camel:http"
    - "camel:kamelet"
  template:
    from:
      id: ${fromId}
      uri: "timer:user"
      parameters:
        period: "{{period}}"
      steps:
        - to: "https://random-data-api.com/api/v2/users"
        - to: "kamelet:sink"

3. After I've added this enum to DSLSelectorToggle:

const dslEntriesRef = useRef<Partial<Record<SourceSchemaType, ISourceSchema>>>({
    [SourceSchemaType.Workflows]: sourceSchemaConfig.config[SourceSchemaType.Workflows],
    [SourceSchemaType.Route]: sourceSchemaConfig.config[SourceSchemaType.Route],
    [SourceSchemaType.Kamelet]: sourceSchemaConfig.config[SourceSchemaType.Kamelet],
    [SourceSchemaType.Pipe]: sourceSchemaConfig.config[SourceSchemaType.Pipe],
  });

After that I need the select to be able to choose the: SourceSchemaType.Workflows

Question: How to continue from here to add this screen? How to add more fields to yaml and how to change the schema so it would allow me to edit this yaml as I need?

There's a lot of questions, the code you have now, not that extensible and highly coupled.

[lordrip]

My suggestion would be to follow the code path to determine where it makes the most sense to add your additional screen, as you pointed out, the existing code serves our main purpose of providing an Apache Camel UI, so maybe some customizations need to be done outside of the Kaoto project scope.

I'm sorry for not being able to help you more at the moment.

[HackPoint]

@lordrip Hi, I need some explanation, how do I add new schema in java? I need to generate "Workflow".json schema, it's pretty much similar to kamelet schema. But I'm not able to see where and how you generating the typescript for it.

Can you tell me please how to add new schema: workflow so I would be able to add new features to it. I saw your code in java, hope at least you would be able to assist me here.

package io.kaoto.camelcatalog.generator;

import com.fasterxml.jackson.databind.node.ObjectNode;
import java.util.List;

public class WorkflowProcessor {
    private static final List<String> TO_STRING_TYPES = List.of("binary");

    public static void process(ObjectNode workflow) {
        var schema = workflow.withObject("/propertiesSchema");
        var workflowDef = workflow.withObject("/spec")
                .withObject("/definition");
        schema.put("$schema", "http://json-schema.org/draft-07/schema#");
        schema.put("type", "object");
        if (workflowDef.has("title")) schema.set("title", workflowDef.get("title"));
        if (workflowDef.has("description")) schema.set("description", workflowDef.get("description"));
        if (workflowDef.has("required")) schema.set("required", workflowDef.get("required"));
        if (workflowDef.has("properties") && !workflowDef.withObject("/properties").isEmpty()) {
            var workflowProperties = workflowDef.withObject("/properties");
            var schemaProperties = schema.withObject("/properties");
            for (var entry : workflowProperties.properties()) {
                var name = entry.getKey();
                var property = entry.getValue();
                var schemaProperty = schemaProperties.withObject("/" + name);
                if (property.has("type")) schemaProperty.set("type", property.get("type"));
                if (TO_STRING_TYPES.contains(property.get("type").asText())) {
                    schemaProperty.put("$comment", "type:" + property.get("type").asText());
                    schemaProperty.put("type", "string");
                }
                if (property.has("title")) schemaProperty.set("title", property.get("title"));
                if (property.has("description")) schemaProperty.set("description", property.get("description"));
                if (property.has("enum")) schemaProperty.set("enum", property.get("enum"));
                if (property.has("default")) schemaProperty.set("default", property.get("default"));
                if (property.has("format")) schemaProperty.set("format", property.get("format"));
            }
        }
    }
}

I've added new class and I would hope that you would be able to help me move forward from here.