Skip to content

How to implement custom plan state visualization

Jump to bottom
Jan Dolejší edited this page Jan 28, 2021 · 7 revisions

When debugging the JavaScript custom visualization logic, open Help > Toggle Developer Tools in your VS Code to see the output printed by console.log statements.

To see your code changes reflected in the view, re-open the view pane, or hide it (by switching to a different tab) and re-focus the view tab.

Connecting visualization to PDDL domain file

The name of the JavaScript file must be configured in the blocksworld.planviz.json file next to the domain file. That is assuming the domain file is called blocksworld.pddl.

{
    "customVisualization": "blocksWorldViz.js"
}

See full example.

The JavaScript script should implement one of the 6 visualization methods. The visualization function shall be registered in the module export (matching the naming):

module.exports = {
   visualizePlanHtml: visualizePlanHtml, 
};

Understanding the function arguments

There are 6 possible ways how to render plan or state. Let's take the following function that you can implement to inject your visualization to the planVizDiv HTML div Element:

/**
 * Populates the `planVizDiv` element with the plan visualization of the `finalState`.
 * @param {HTMLDivElement} planVizDiv host element on the page
 * @param {Plan} plan plan to be visualized
 * @param {{variableValue: string, value: number | boolean}[]} finalState final state of the `plan`
 * @param {number} displayWidth desired width in pixels
 */
function visualizeStateInDiv(planVizDiv, plan, finalState, displayWidth) {
   for (const v of finalState) {
      console.log(`${v.variableName}: ${v.value}`);
   }

    //...
}

Final state values

As the type hints in the JSDoc above indicate, the finalState is a list of name-value tuples, where the value may be a number or a boolean.

It may be simpler to work with it, if you summarize that vector to a map:

   const valueMap = new Map(finalState.map(i => [i.variableName, i.value]));

Which may be easily read using valueMap.get('predicate1 obj1 obj2');.

Accessing domain and problem info through the plan

To get the total list of PDDL constants and objects from the domain and problem file, you first need to merge the constants and the objects:

   // get all domain constants plus problem objects
   const allObjects = plan.problem && plan.problem.getObjectsTypeMap()
      && plan.domain && plan.domain.getConstants()
      && plan.domain.getConstants().merge(plan.problem.getObjectsTypeMap());

Observe that the plan.problem and plan.domain may not be defined (in case the state/plan is being rendered without the association with the domain and problem. The statement above anticipates this and checks that they exist. If either of them was not set on the plan instance, this returns undefined.

If allObjects is populated, it provides access to the objects for each type. For example, you can request all block objects this way:

allObjects.getTypeCaseInsensitive("block").getObjects()

The above returns array of strings.

However, since the allObjects may be undefined (i.e. the domain and problem were not available), the object names may be extracted from the plan

   /** @type {string[]} block names */
   const blocks = allObjects ? 
      allObjects.getTypeCaseInsensitive("block").getObjects() :
      [...new Set(plan.steps.map(s => s.getObjects()).reduce((prev, cur) => prev.concat(cur)))]

   const sortedBlocks = blocks.sort();

Fine-tuning the experience

The implementer is given the displayWidth (in pixels) and can decide on the height of the content. For the visual stability of the display (expecially in the visual search debugger), it is recommended to set the height of the hosting element:

   const height = displayWidth / 2;
   planVizDiv.style.height = `${height}px`;

It is also recommended to eliminate any randomness, so the same state always displays the same way. It is recommended to order all lists/sets lexicographically.

Rendering images from other servers

All the 3 webview pages, where the custom plan/state visualization may be inserted are controlled by a Content Security Policy. The policy prevents the HTML from requesting resources from unknown locations. This prevents you from using images stored on local disk or remote server. You can however store the images (if small) as Base64 encoded or as SVG right inside the custom Javascript file. Example of that approach is the gripper image in the blocks world visualization.

There are two supported external resources that you can use in your custom visualization, because they are already used by those pages:

  • Google charts
  • Vis.js Network diagrams

Flow charts, trees, graphs and network diagrams

A lot of different visualizations can be achieved with the Vis.js Networks. Here is an example (static mocked diagram for simplicity):

/**
 * Populates the `planVizDiv` element with the plan visualization of the `finalState`.
 * @param {HTMLDivElement} planVizDiv host element on the page
 * @param {Plan} plan plan to be visualized
 * @param {{variableName: string, value: number | boolean}[]} finalState final state of the `plan`
 * @param {number} displayWidth desired width in pixels
 */
function visualizeStateInDiv(planVizDiv, plan, finalState, displayWidth) {
    planVizDiv.style.height = `${displayWidth}px`;
    for (const v of finalState) {
        console.log(`${v.variableName}: ${v.value}`);
    }

    // values are accessible via a map:
    const valueMap = new Map(finalState.map(i => [i.variableName, i.value]));

    // mock example: create an array with nodes
    const nodes = new vis.DataSet([
        { id: 1, label: "node\none", shape: "box", color: "#97C2FC" },
        { id: 2, label: "node\ntwo", shape: "circle", color: "#FFFF00" },
        { id: 3, label: "node\nthree", shape: "diamond", color: "#FB7E81" },
        {
            id: 4,
            label: "node\nfour",
            shape: "dot",
            size: 10,
            color: "#7BE141",
        },
        { id: 5, label: "node\nfive", shape: "ellipse", color: "#6E6EFD" },
        { id: 6, label: "node\nsix", shape: "star", color: "#C2FABC" },
        { id: 7, label: "node\nseven", shape: "triangle", color: "#FFA807" },
        {
            id: 8,
            label: "node\neight",
            shape: "triangleDown",
            color: "#6E6EFD",
        },
    ]);

    // mock example: create an array with edges
    const edges = new vis.DataSet([
        { from: 1, to: 8, color: { color: "red" } },
        { from: 1, to: 3, color: "rgb(20,24,200)" },
        {
            from: 1,
            to: 2,
            color: { color: "rgba(30,30,30,0.2)", highlight: "blue" },
        },
        { from: 2, to: 4, color: { inherit: "to" } },
        { from: 2, to: 5, color: { inherit: "from" } },
        { from: 5, to: 6, color: { inherit: "both" } },
        { from: 6, to: 7, color: { color: "#ff0000", opacity: 0.3 } },
        { from: 6, to: 8, color: { opacity: 0.3 } },
    ]);

    const data = {
        nodes: nodes,
        edges: edges,
    };
    const options = {
        nodes: {
            shape: "circle",
          },
    };
    const network = new vis.Network(planVizDiv, data, options);

    // optionally handle events on the `network` object
}

module.exports = {
    visualizeStateInDiv: visualizeStateInDiv
};

This code will insert a simple diagram to the state/plan visualization.

Examples

Clone this wiki locally