In Step 1 you deployed a snake using a pre-trained deep learning model. However, deep learning models can sometimes choose sub-optimal actions which can result in undesirable outcomes for your snake, such as colliding with a wall or facing off with an opponent snake that has a longer body (in both cases, your snake will die).
So in this section, you will write some heuristics that serve as ground rules for your snake. These ground rules will override any decisions made by the deep learning model. For example, you can write a routine that will determine if a given movement decision will result in a collision with a wall, or if your snake can be able to defeat another shorter snake by colliding with it, head to head.
In the next section, you will go "deeper" and customize your deep learning model.
Let's begin!
PRE-REQUISITE: You need to run Step 1 before completing this step.
If you do clever things, your pull request is welcome!
Estimated cost : This environment does not add any cost to Step 1. The free tiers include 250 hours per month of this notebook instance during the first two months. After the free tiers, the charge will be $0.269 per hour for the notebook instance ($6.5 per 24 hour period). Cost savings tip : Once you have finished working, you can stop your notebook instance in order to stop consuming free tiers or incurring charges. You can easily restart them at a later date to continue with your work.
From the Cloudformation stack created during Step 1, go to the 'Outputs' tab and click on the link next to HeuristicsDevEnvironment:
You need to be authenticated for that link to work. Click on SourceEditionInNotebook link if you are denied access
Now that you have the notebook HeuristicDeveloper.ipynb open, ensure that you have a functioning model (if you have altered the model, you may need to configure the inference step in heuristics_utils.get_action(*args)). Press ► at the top of the Jupyter window to run the notebook (see here for a tutorial on how to use Jupyter notebooks).
From the Cloudformation stack created during Step 1, go to the 'Outputs' tab and click on the link next to SourceEditionInNotebook:
Then navigate to battlesnake/RLlibEnv/inference/inference_src/battlesnake_heuristics.py
You can customize the run() method in the class MyBattlesnakeHeuristics with your own rules (see go_to_food_if_close for an example).
Dev tools: Editing code in Jupyter notebooks with no source version control is convenient for testing purposes, but isn't practical for larger projects. If you are interested in implementing source code version control, see: setup source control
- If you want to visualize your AI in action, ensure that you are using Jupyter instead of JupyterLab (this is the default if you use the links from the CloudFormation 'Outputs' tab).
- The notebook loads a pre-trained model and allows your AI to interact with the environment
- After the Playback the simulation section, you should see the step-by-step positions, actions, health etc. of each snake.
- If you want to specify the positions of each snake and food (instead of randomly generating it), you can enter it in
initial_statein Define the openAI gym. initial_state is defined similarly to the battlesnake API.
- If you trained a new model (but did not deploy it yet), run the cell in (Optional) Run if you retrained the model.
- Otherwise, simply run the cell in Deploy your new heuristics and your snake will be deployed.
- Note that there's no need to change the url on the battlesnake engine.
Go to Step 3 to train the model again with your own settings.