InputActuatorComponent to allow the generation of an action space from an InputActionAsset #4881

surfnerd · 2021-01-23T00:35:00Z

Proposed change(s)

Enable developers to generate an action space for agents based off of an input action asset.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

~~- [ ] Bug fix~~

New feature
~~- [ ] Code refactor~~
~~- [ ] Breaking change~~
Documentation update
~~- [ ] Other (please describe)~~

Checklist

Added tests that prove my fix is effective or that my feature works

Updated the changelog (if applicable)
Updated the documentation (if applicable)
~~- [ ] Updated the migration guide (if applicable)~~

Other comments

com.unity.ml-agents.extensions/Runtime/Input/InputActionActuator.cs

com.unity.ml-agents/Runtime/Actuators/ActionSpec.cs

… the original method. The original method will be removed in the future.

Co-authored-by: Vincent-Pierre BERGES <[email protected]>

…uators.

…ent for debugging.

…ion Asset, default to the player input one if we can't find it.

com.unity.ml-agents.extensions/Runtime/Input/InputActuatorComponent.cs

…n-heuristic modes between play/edit mode. Remove agent field.

…tion spec generated from the InputActionAsset.

com.unity.ml-agents.extensions/Documentation~/InputActuatorComponent.md

com.unity.ml-agents.extensions/Editor/Input/InputActuatorComponentEditor.cs

com.unity.ml-agents.extensions/Runtime/Input/IRLActionInputAdaptor.cs

chriselion · 2021-02-18T18:37:41Z

com.unity.ml-agents/Runtime/Actuators/ActuatorManager.cs

@@ -157,9 +158,11 @@ public ActionSpec GetCombinedActionSpec()
        /// actions for the IActuators in this list.</param>
        public void UpdateActions(ActionBuffers actions)
        {
+            Profiler.BeginSample("ActuatorManager.UpdateActions");


If you want profiling on these, you can also use

using (TimerStack.Instance.Scoped("ActuatorManager.UpdateActions")) { ... }

(which adds Profiler samples too)

com.unity.ml-agents/Runtime/Policies/BehaviorParameters.cs

com.unity.ml-agents/Runtime/Actuators/ActionSpec.cs

…m an InputActionAsset (#4881)

…m an InputActionAsset (#4881) (#4974)

* [MLA-1768] retrain Match3 scene (Unity-Technologies#4943) * improved settings and move to default_settings * update models * Release 13 versions. (Unity-Technologies#4946) - updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation. * Update readme for release_13. (Unity-Technologies#4951) * Update docs to pass doc validation. (Unity-Technologies#4953) * update defines, compile out Initialize body on non-desktop (Unity-Technologies#4957) * Masking Discrete Actions typos (Unity-Technologies#4961) (Unity-Technologies#4964) Co-authored-by: Philipp Siedler <[email protected]> * Adding references to the Extensions package to help promote it. (Unity-Technologies#4967) (Unity-Technologies#4968) Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Chris Elion <[email protected]> * Fix release link validations. (Unity-Technologies#4970) * Adding the Variable length observation to the readme and to the overview of ML-Agents * - * forgot a dot * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (Unity-Technologies#4881) (Unity-Technologies#4974) * handle no plugins found (Unity-Technologies#4969) (Unity-Technologies#4973) * Tick extension version. (Unity-Technologies#4975) * adding a comic and readding removed feaures docs * Update 2018 project version to fix burst errors. (Unity-Technologies#4977) (Unity-Technologies#4978) * Add an example project for the InputSystemActuator. (Unity-Technologies#4976) (Unity-Technologies#4980) * Update barracuda, swtich Agents in Sorter use Burst. (Unity-Technologies#4979) (Unity-Technologies#4981) * Set ignore done=False in GAIL (Unity-Technologies#4971) * MultiAgentGroup Interface (Unity-Technologies#4923) * add SimpleMultiAgentGroup * add group reward field to agent and proto * Fix InputActuatorComponent tests asmdef. (Unity-Technologies#4994) (Unity-Technologies#4995) * Fix asmdef? (Unity-Technologies#4994) (Unity-Technologies#4996) * Make TrainingAnalyticsSideChannel internal (Unity-Technologies#4999) * [MLA-1783] built-in actuator type (Unity-Technologies#4950) * Add component menues for some sensors and actuators. (Unity-Technologies#5001) * Add component menues for some sensors and actuators. (Unity-Technologies#5001) (Unity-Technologies#5002) * Fixing the number of layers in the config of PyramidsRND * Merge master -> release_13_branch-to-master * Fix RpcCommunicator merge. * Move the Critic into the Optimizer (Unity-Technologies#4939) Co-authored-by: Ervin Teng <[email protected]> * master -> main. (Unity-Technologies#5010) * Adding links to CV/Robotics/GameSim (Unity-Technologies#5012) * Make GridSensor a non allocating object after initialization. (Unity-Technologies#5014) Co-authored-by: Chris Elion <[email protected]> * Modified the model_serialization to have correct inputs and outputs * switching from CamelCase to snake_case * Fix gpu pytests (Unity-Technologies#5019) * Move tensors to cpu before converting it to numpy * Adding a name field to BufferSensorComponent * Adding a note to the CHANGELOG about var len obs * Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders * Adding a custom editor for BufferSensorComponent * Changing Sorter to use the new Editor and serialization * adding inheritdoc * Update cattrs dependencies to support python3.9 (Unity-Technologies#4821) * Python Dataflow for Group Manager (Unity-Technologies#4926) * Make buffer type-agnostic * Edit types of Apped method * Change comment * Collaborative walljump * Make collab env harder * Add group ID * Add collab obs to trajectory * Fix bug; add critic_obs to buffer * Set group ids for some envs * Pretty broken * Less broken PPO * Update SAC, fix PPO batching * Fix SAC interrupted condition and typing * Fix SAC interrupted again * Remove erroneous file * Fix multiple obs * Update curiosity reward provider * Update GAIL and BC * Multi-input network * Some minor tweaks but still broken * Get next critic observations into value estimate * Temporarily disable exporting * Use Vince's ONNX export code * Cleanup * Add walljump collab YAML * Lower max height * Update prefab * Update prefab * Collaborative Hallway * Set num teammates to 2 * Add config and group ids to HallwayCollab * Fix bug with hallway collab * Edits to HallwayCollab * Update onnx file meta * Make the env easier * Remove prints * Make Collab env harder * Fix group ID * Add cc to ghost trainer * Add comment to ghost trainer * Revert "Add comment to ghost trainer" This reverts commit 292b6ce. * Actually add comment to ghosttrainer * Scale size of CC network * Scale value network based on num agents * Add 3rd symbol to hallway collab * Make comms one-hot * Fix S tag * Additional changes * Some more fixes * Self-attention Centralized Critic * separate entity encoder and RSA * clean up args in mha * more cleanups * fixed tests * entity embeddings work with no max Integrate into CC * remove group id * very rough sketch for TeamManager interface * One layer for entity embed * Use 4 heads * add defaults to linear encoder, initialize ent encoders * add team manager id to proto * team manager for hallway * add manager to hallway * send and process team manager id * remove print * small cleanup * default behavior for baseTeamManager * add back statsrecorder * update * Team manager prototype (Unity-Technologies#4850) * remove group id * very rough sketch for TeamManager interface * add team manager id to proto * team manager for hallway * add manager to hallway * send and process team manager id * remove print * small cleanup Co-authored-by: Chris Elion <[email protected]> * Remove statsrecorder * Fix AgentProcessor for TeamManager Should work for variable decision frequencies (untested) * team manager * New buffer layout, TeamObsUtil, pad dead agents * Use NaNs to get masks for attention * Add team reward to buffer * Try subtract marginalized value * Add Q function with attention * Some more progress - still broken * use singular entity embedding (Unity-Technologies#4873) * I think it's running * Actions added but untested * Fix issue with team_actions * Add next action and next team obs * separate forward into q_net and baseline * might be right * forcing this to work * buffer error * COMAA runs * add lambda return and target network * no target net * remove normalize advantages * add target network back * value estimator * update coma config * add target net * no target, increase lambda * remove prints * cloud config * use v return * use target net * adding zombie to coma2 brnch * add callbacks * cloud run with coma2 of held out zombie test env * target of baseline is returns_v * remove target update * Add team dones * ntegrate teammate dones * add value clipping * try again on cloud * clipping values and updated zombie * update configs * remove value head clipping * update zombie config * Add trust region to COMA updates * Remove Q-net for perf * Weight decay, regularizaton loss * Use same network * add base team manager * Remove reg loss, still stable * Black format * add team reward field to agent and proto * set team reward * add maxstep to teammanager and hook to academy * check agent by agent.enabled * remove manager from academy when dispose * move manager * put team reward in decision steps * use 0 as default manager id * fix setTeamReward Co-authored-by: Vincent-Pierre BERGES <[email protected]> * change method name to GetRegisteredAgents * address comments * Revert C# env changes * Remove a bunch of stuff from envs * Remove a bunch of extra files * Remove changes from base-teammanager * Remove remaining files * Remove some unneeded changes * Make buffer typing neater * AgentProcessor fixes * Back out trainer changes * use delegate to avoid agent-manager cyclic reference * put team reward in decision steps * fix unregister agents * add teamreward to decision step * typo * unregister on disabled * remove OnTeamEpisodeBegin * change name TeamManager to MultiAgentGroup * more team -> group * fix tests * fix tests * Use attention tests from master * Revert "Use attention tests from master" This reverts commit 78e052b. * Use attention from master * Renaming fest * Use NamedTuples instead of attrs classes * Bug fixes * remove GroupMaxStep * add some doc * Fix mock brain * np float32 fixes * more renaming * Test for team obs in agentprocessor * Test for group and add team reward * doc improve Co-authored-by: Ervin T. <[email protected]> * store registered agents in set * remove unused step counts * Global group ids * Fix Trajectory test * Remove duplicated files * Add team methods to AgentAction * Buffer fixes (cherry picked from commit 2c03d2b) * Add test for GroupObs * Change AgentAction back to 0 pad and add tests * Addressed some comments * Address some comments * Add more comments * Rename internal function * Move padding method to AgentBufferField * Fix slicing typing and string printing in AgentBufferField * Fix to-flat and add tests * Rename GroupmateStatus to AgentStatus * Update comments * Added GroupId, GlobalGroupId, GlobalAgentId types * Update comment * Make some agent processor properties internal * Rename add_group_status * Rename store_group_status, fix test * Rename clear_group_obs Co-authored-by: Andrew Cohen <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Vincent-Pierre BERGES <[email protected]> * Removing some scenes (Unity-Technologies#4997) * Removing some scenes, All the Static and all the non variable speed environments. Also removed Bouncer, PushBlock, WallJump and reacher. Removed a bunch of visual environements as well. Removed 3DBallHard and FoodCollector (kept Visual and Grid FoodCollector) * readding 3DBallHard * readding pushblock and walljump * Removing tennis * removing mentions of removed environments * removing unused images * Renaming Crawler demos * renaming some demo files * removing and modifying some config files * new examples image? * removing Bouncer from build list * replacing the Bouncer environment with Match3 for llapi tests * Typo in yamato test * Fix issue with queuing input events that stomp on others. (Unity-Technologies#5034) Co-authored-by: Chris Elion <[email protected]> Co-authored-by: Chris Goy <[email protected]> Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: vincentpierre <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Andrew Cohen <[email protected]>

surfnerd requested a review from chriselion January 23, 2021 00:35

surfnerd self-assigned this Jan 23, 2021

surfnerd force-pushed the develop-input-actuator branch 2 times, most recently from 0a60209 to 8fb4a72 Compare February 2, 2021 19:16

chriselion reviewed Feb 6, 2021

View reviewed changes

com.unity.ml-agents.extensions/Runtime/Input/InputActionActuator.cs Outdated Show resolved Hide resolved

chriselion reviewed Feb 6, 2021

View reviewed changes

com.unity.ml-agents.extensions/Runtime/Input/InputActionActuator.cs Outdated Show resolved Hide resolved

chriselion reviewed Feb 6, 2021

View reviewed changes

com.unity.ml-agents/Runtime/Actuators/ActionSpec.cs Outdated Show resolved Hide resolved

surfnerd marked this pull request as ready for review February 11, 2021 23:17

surfnerd requested review from dongruoping and vincentpierre February 11, 2021 23:18

surfnerd changed the title ~~[Draft] First cut of some InputActuator abstractions~~ InputActuatorComponent to allow the generation of an action space from an InputActionAsset Feb 12, 2021

surfnerd requested a review from Hunter-Unity February 12, 2021 00:11

surfnerd and others added 18 commits February 11, 2021 16:28

Add CreateActuators method to the ActuatorComponent class which wraps…

6ef9ebb

… the original method. The original method will be removed in the future.

Update migrating doc.

9c0dcde

typo

7bcdb0d

Co-authored-by: Vincent-Pierre BERGES <[email protected]>

Add input action asset

a60e9b0

[WIP] Initial implementation of InputActuator with Vector2 values.

2e9da14

Fix rider warning.

2911655

Some changes.

f235c38

Inject bindings into the asset at runtime.

fdf8dc8

remove some cruft.

2a8194f

some small cleanup.

e37cba0

Rebase onto multiple actuator branch. thanks yamato.

372f38f

more cleanup before creating 1:1 relationship bettwen actions and act…

1ded6ec

…uators.

Update .gitignore, remove files i wish not to track.

6d1608f

break up actuators by action definition. Add a more interesting movem…

9dccd8f

…ent for debugging.

comment.

e0187e6

Add jump to actions along with adaptors.

da26c1f

Jumping working.

27139b4

Add jump action. Setup an interface for us to get the users Input Act…

f2326ad

…ion Asset, default to the player input one if we can't find it.

Rene-Damm reviewed Feb 17, 2021

View reviewed changes