[WIP] DIY NN - examples for loading model

[joeyklee]

An example for loading in a pre-trained model.

There's a number of funky issues we need to resolve:

1. maintaining the input/output order: For some reason it seems like the saved model does not preserve the order of the inputs. Something to to look into.

2. Because we need the input/output min/max values to transform to and from normalized/unnormalized data, we need to specify the normalizationOptions in the constructor. This is not ideal.

let nnOptions = {
    inputs: ['r', 'g', 'b'],
    outputs: ['label'],
    task: 'classification',
    normalizationOptions: {
      inputMax: [255, 255, 255],
      inputMin: [0, 0, 0],
      outputMax: [1, 1, 1, 1, 1, 1, 1, 1, 1],
      outputMin: [0, 0, 0, 0, 0, 0, 0, 0, 0],
    },
    debug: true
  };

3. Lastly since we need to know the correct amount of inputUnits and outputUnits as well as rely on the outputUnits of classification tasks and/or oneHot encoded inputs/outputs to have a legend, we need to add those in somehow:

neuralNetwork = ml5.neuralNetwork(nnOptions);
  neuralNetwork.load('model/model.json', modelReady);

function modelReady() {
  console.log('model loaded!')
  neuralNetwork.data.meta.inputs = {
    r: {
      dtype: 'number'
    },
    g: {
      dtype: 'number'
    },
    b: {
      dtype: 'number'
    }
  }
  neuralNetwork.data.meta.outputUnits = 9;
  neuralNetwork.data.meta.outputs = {
    label: {
      dtype: 'string',
      uniqueValues: ["green-ish", "pink-ish", "orange-ish", "blue-ish", "brown-ish", "red-ish", "yellow-ish", "purple-ish", "grey-ish"],
      legend: {
        "green-ish": [1, 0, 0, 0, 0, 0, 0, 0, 0],
        "pink-ish": [0, 1, 0, 0, 0, 0, 0, 0, 0],
        "orange-ish": [0, 0, 1, 0, 0, 0, 0, 0, 0],
        "blue-ish": [0, 0, 0, 1, 0, 0, 0, 0, 0],
        "brown-ish": [0, 0, 0, 0, 1, 0, 0, 0, 0],
        "red-ish": [0, 0, 0, 0, 0, 1, 0, 0, 0],
        "yellow-ish": [0, 0, 0, 0, 0, 0, 1, 0, 0],
        "purple-ish": [0, 0, 0, 0, 0, 0, 0, 1, 0],
        "grey-ish": [0, 0, 0, 0, 0, 0, 0, 0, 1]
      }
    },

  }

  classify();
};

cc/ @shiffman

[joeyklee]

One way I can imagine handling this is by:

1. When a user saves their model, we also can save an additional json file that saves the following information:

• inputMax:[]
• inputMin:[]
• outputMax:[]
• outputMin:[]
• neuralNetwork.data.meta.inputs
• neuralNetwork.data.meta.outputs

[shiffman]

Yes, this makes sense! Do you think we could try adding it to an ml5 property in model.json or will that cause problems? I suppose if we are already requiring two files then three isn't such a big deal.

[joeyklee]

@shiffman - I will give this a whirl/whorl (i can never remember which one)!

[joeyklee]

This should be ready to merge once ml5js/ml5-library#613 is merged in!