[RL] Review RL brains and update documentation

[sergiopaniego]

RL brains are currently broken, so a review of that workflow is needed.

Update the documentation about that RL brains is also needed.

[sergiopaniego]

This issue comes from the previous one -> #140

[UtkarshMishra04]

Hi

So we will derive the RL brain from one of the DL brain base classes, right? If not, what approach do you plan?
Btw, I have a continuous RL brain here and I guess it is sufficient if this works

[sergiopaniego]

I guess the idea is to review the RL part and make it work again so we shouldn't use any no RL based brain for this task. We don't need to generate a new brain now but just fix the ones already in the project (RL part).

Thanks for the brain, we can review and add it later on but this task has the maximum priority now. Without it fixed, we can't merge your brain.

[UtkarshMishra04]

Hi Sergio

I have some doubts.

1. this line closes gazebo when we try to go through an f1rl brain. Is that so?
2. No brains are assigned when robot type is f1rl. That is not what we want. I am confused. Is the RL pipeline different from the DL pipeline?

[sergiopaniego]

It actually is different. The RL pipeline is based on gym-gazebo, a combination of OpenAI Gym and Gazebo that has everything related to RL inside (if you follow the calls that's where the workflow is heading after that line you comment).
Additionally, you need to install some dependencies for working with RL (search for Reinforcement learning in the installation web) and the .yml is different (search for reinforcement learning in the quick start web)

There is a folder with gym-gazebo in the project root and it comes from this repo and is now under development by our org here. It will be a separated project soon by for now it's integrated in here.

The DL pipeline doesn't need the OpenAI gym part.

[UtkarshMishra04]

Hi

Yes, I followed everything and I verified that whatever is required already exists in the same form as @dcharrezt formulated in his latest commit on the RL setup.
I am not sure but do you believe it is a gym-gazebo issue? Your suggestion on this will narrow down my debugging as I don't see any significant change in the RL brains after I followed all the commits of @dcharrezt, which could have caused this issue.

Thanks

[sergiopaniego]

I can't assure that the problem comes from gym-gazebo but I believe that it's related with Behavior Studio itself. You could go back to the version of Behavior Studio from last @dcharrezt commit (somewhere in August) and verify that that version is also broken

[UtkarshMishra04]

Thank you very much!

[sergiopaniego]

I solved the problem with the RL brains. It was related with some circuits path changes and the motors publisher. Now you should be able to run both DL and RL based brains.

For the DL brains, use default.yml file and for the RL ones I've created default-rl.yml.
The steps to run the default RL training are:

• bash load-env.sh under gym-gazebo folder.
• cd to BehaviorStudio/behavior-studio
• Try with the default configuration to show that it works -> python driver.py -c default-rl.yml -g
• Change parameters/algorithm for your own training 😄

Please try it out and tell me if that worked for you.