NeuroEvolution

[shiffman]

I'm continuing from where I left off with #23 and moving onto neuro-evolution. One API note, I've changed the noTraining option to neuroEvolution, so now creating the empty model looks like:

let model = ml5.neuralNetwork({
  inputs: 5,
  outputs: ['flap', 'no flap'],
  task: 'classification',
  neuroEvolution: true,
});

I thought about neuroEvolution being a "task", but specifying "classification" or "regression" is still needed.I could adapt task to accept an array of options, but I think that overcomplicates things. This flag is necessary in order for the model to be initialized properly for use with some of the neuroevolution functionality. Thoughts welcome!

Also, I noticed that my VSCode settings made quite a few changes to whitespace, quote style, etc. @ziyuan-linn I followed the CONTRIBUTING.md instructions, did I miss any steps perhaps? I noticed the .prettierrc.json file is empty, do we need to add options to it?

[ziyuan-linn]

Hi @shiffman , the .prettierrc file is empty so it would override any user settings and use Prettier's default settings for this workspace. I looked at your commit, and it seems like Prettier is formatting correctly. I think the problem is that the original files have never been formatted by Prettier. I will make a PR formatting all the existing files to resolve this issue.

[shiffman]

@ziyuan-linn I can't explain it, but for some reason the Pose Detection model caused an error in ml5.neuralNetwork(). I think maybe it may have required updating some dependencies which caused something to break. See this commit de87879 where I temporarily disabled the pose model. After this commit, when I run yarn install and yarn run start the neural network examples work again. If I add pose detection back in, run yarn install they break.

I think it may be due to a tfjs-data update but I haven't had time to diagnose this. Do you have any idea or would you be able to look into this? cc @gohai as another pair of eyes.

[ziyuan-linn]

@shiffman That's strange. I can take a look today!

[ziyuan-linn]

@shiffmanThe error should go away after merging #34 :)

[shiffman]

Thank you @ziyuan-linn, what's weird is that I am explicitly using ml5.tf.setBackend("cpu"); for my neuroEvolution examples which is actually much higher performance for tiny models. And I still got the error! I wrote a short explanation in the nature of code book:

Let me try merging #34, updating this branch, and see where it leaves us!

[shiffman]

Ah, ok, so the issue is resolved but I am not sure why my call to ml5.tf.setBackend("cpu"); was not working previously. Now with the forced tf.setBackend("webgl"); in the init() method, I am not able to properly override the backend. I'm hard-coding it to CPU for now in b460d80 just so I can keep working on my examples. Maybe we can look at a better way to handle this as you suggested in #34!

Is the backend a global setting for any given webpage? If so then I don't think it makes sense to add an option to ml5.neuralNetwork() since it would give the appearance of different backends for different models? We could also just wrap tf.setBackend() and add a utility method:

ml5.setBackend("cpu"):

This would be my preference! What do you think @ziyuan-linn?

[ziyuan-linn]

@shiffman Ohh I think calling ml5.tf.setBackend("cpu") before initializing Bird might have worked previously before the forced called was added to init().

function setup() {
  createCanvas(640, 240);
  ml5.tf.setBackend("cpu");//<------- moved to before Bird constructor call
  for (let i = 0; i < 200; i++) {
    birds[i] = new Bird();
  }
  pipes.push(new Pipe());
  
}

My guess is that if the Bird constructor is called and subsequently calls ml5.nerualNetwork() before setBackend("cpu"), the program would run with the default webgpu backend and crash before the backend could be changed.

I think tf.setBackend is a global setting, and I agree that a ml5 utility method would make more sense. I would also be happy to implement it, which shouldn't be too hard.

[shiffman]

This is just about ready but I'd like to resolve and merge #36 first. Also, workflow-wise, I wonder if we should be using "squash and merge" since our development branches involve a lot of small steps and changes? This could apply to #26 as well? @ziyuan-linn @gohai

[ziyuan-linn]

@shiffman I think squash and merge can be a good idea for the big development branches! I don't really have a strong opinion either way.

[shiffman]

These examples along with a completed draft of Chapter 10: Nature of Code are ready to go! @gohai and @ziyuan-linn any comments before I squash and merge?

The examples could probably use some cleanup / comments, I will continue to refine as I work on the chapter.

[ziyuan-linn]

I briefly read through chapter 10 of The Nature of Code. I was only vaguely familiar with the concept of reinforcement learning, but I think the book explained the concept very well through the step-by-step flappy bird example! The Neuroevolution Ecosystem example is also fascinating and makes me want to try it on my own.

The examples code seems very clear to me and I was able to run them without any problem.

[shiffman]

I'm working on reviewing pull requests today! I think I'll go ahead and merge this one first!