Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix gpu pytests #5019

Merged
merged 3 commits into from
Mar 1, 2021
Merged

Fix gpu pytests #5019

merged 3 commits into from
Mar 1, 2021

Conversation

dongruoping
Copy link
Contributor

Proposed change(s)

Move tensors to cpu before converting it to numpy

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

  • Bug fix
  • New feature
  • Code refactor
  • Breaking change
  • Documentation update
  • Other (please describe)

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)
  • Updated the documentation (if applicable)
  • Updated the migration guide (if applicable)

Other comments

@chriselion
Copy link
Contributor

Does this affect release 13? If so, we should probably do a hotfix release...

@chriselion
Copy link
Contributor

Also, would it explain https://forum.unity.com/threads/im-getting-runtimeerror-rnn-hx-is-not-contiguos.1066898/ ?

@dongruoping
Copy link
Contributor Author

@chriselion This is affected by #4939 and I think the release is safe.
The forum post seems to be a different issue and that should have already been resolved in the latest version.

@chriselion
Copy link
Contributor

👍 Thanks for clarifying.

@dongruoping dongruoping merged commit 355bc79 into main Mar 1, 2021
@delete-merged-branch delete-merged-branch bot deleted the develop-fix-gpu branch March 1, 2021 21:37
philippds added a commit to philippds-forks/ml-agents that referenced this pull request Mar 5, 2021
@philippds @surfnerd
@vincentpierre @andrewcoh @dongruoping
Updating repository (#3)
b5773f2 
* [MLA-1768] retrain Match3 scene (Unity-Technologies#4943)

* improved settings and move to default_settings

* update models

* Release 13 versions. (Unity-Technologies#4946)

- updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation.

* Update readme for release_13. (Unity-Technologies#4951)

* Update docs to pass doc validation. (Unity-Technologies#4953)

* update defines, compile out Initialize body on non-desktop (Unity-Technologies#4957)

* Masking Discrete Actions typos (Unity-Technologies#4961) (Unity-Technologies#4964)

Co-authored-by: Philipp Siedler <[email protected]>

* Adding references to the Extensions package to help promote it. (Unity-Technologies#4967) (Unity-Technologies#4968)

Co-authored-by: Marwan Mattar <[email protected]>
Co-authored-by: Chris Elion <[email protected]>

* Fix release link validations. (Unity-Technologies#4970)

* Adding the Variable length observation to the readme and to the overview of ML-Agents

* -

* forgot a dot

* InputActuatorComponent to allow the generation of an action space from an InputActionAsset (Unity-Technologies#4881) (Unity-Technologies#4974)

* handle no plugins found (Unity-Technologies#4969) (Unity-Technologies#4973)

* Tick extension version. (Unity-Technologies#4975)

* adding a comic and readding removed feaures docs

* Update 2018 project version to fix burst errors. (Unity-Technologies#4977) (Unity-Technologies#4978)

* Add an example project for the InputSystemActuator. (Unity-Technologies#4976) (Unity-Technologies#4980)

* Update barracuda, swtich Agents in Sorter use Burst. (Unity-Technologies#4979) (Unity-Technologies#4981)

* Set ignore done=False in GAIL (Unity-Technologies#4971)

* MultiAgentGroup Interface (Unity-Technologies#4923)

* add SimpleMultiAgentGroup

* add group reward field to agent and proto

* Fix InputActuatorComponent tests asmdef. (Unity-Technologies#4994) (Unity-Technologies#4995)

* Fix asmdef? (Unity-Technologies#4994) (Unity-Technologies#4996)

* Make TrainingAnalyticsSideChannel internal (Unity-Technologies#4999)

* [MLA-1783] built-in actuator type (Unity-Technologies#4950)

* Add component menues for some sensors and actuators. (Unity-Technologies#5001)

* Add component menues for some sensors and actuators. (Unity-Technologies#5001) (Unity-Technologies#5002)

* Fixing the number of layers in the config of PyramidsRND

* Merge master -> release_13_branch-to-master

* Fix RpcCommunicator merge.

* Move the Critic into the Optimizer (Unity-Technologies#4939)

Co-authored-by: Ervin Teng <[email protected]>

* master -> main. (Unity-Technologies#5010)

* Adding links to CV/Robotics/GameSim (Unity-Technologies#5012)

* Make GridSensor a non allocating object after initialization. (Unity-Technologies#5014)

Co-authored-by: Chris Elion <[email protected]>

* Modified the model_serialization to have correct inputs and outputs

* switching from CamelCase to snake_case

* Fix gpu pytests (Unity-Technologies#5019)

* Move tensors to cpu before converting it to numpy

* Adding a name field to BufferSensorComponent

* Adding a note to the CHANGELOG about var len obs

* Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders

* Adding a custom editor for BufferSensorComponent

* Changing Sorter to use the new Editor and serialization

* adding inheritdoc

* Update cattrs dependencies to support python3.9 (Unity-Technologies#4821)

* Python Dataflow for Group Manager (Unity-Technologies#4926)

* Make buffer type-agnostic

* Edit types of Apped method

* Change comment

* Collaborative walljump

* Make collab env harder

* Add group ID

* Add collab obs to trajectory

* Fix bug; add critic_obs to buffer

* Set group ids for some envs

* Pretty broken

* Less broken PPO

* Update SAC, fix PPO batching

* Fix SAC interrupted condition and typing

* Fix SAC interrupted again

* Remove erroneous file

* Fix multiple obs

* Update curiosity reward provider

* Update GAIL and BC

* Multi-input network

* Some minor tweaks but still broken

* Get next critic observations into value estimate

* Temporarily disable exporting

* Use Vince's ONNX export code

* Cleanup

* Add walljump collab YAML

* Lower max height

* Update prefab

* Update prefab

* Collaborative Hallway

* Set num teammates to 2

* Add config and group ids to HallwayCollab

* Fix bug with hallway collab

* Edits to HallwayCollab

* Update onnx file meta

* Make the env easier

* Remove prints

* Make Collab env harder

* Fix group ID

* Add cc to ghost trainer

* Add comment to ghost trainer

* Revert "Add comment to ghost trainer"

This reverts commit 292b6ce.

* Actually add comment to ghosttrainer

* Scale size of CC network

* Scale value network based on num agents

* Add 3rd symbol to hallway collab

* Make comms one-hot

* Fix S tag

* Additional changes

* Some more fixes

* Self-attention Centralized Critic

* separate entity encoder and RSA

* clean up args in mha

* more cleanups

* fixed tests

* entity embeddings work with no max
Integrate into CC

* remove group id

* very rough sketch for TeamManager interface

* One layer for entity embed

* Use 4 heads

* add defaults to linear encoder, initialize ent encoders

* add team manager id to proto

* team manager for hallway

* add manager to hallway

* send and process team manager id

* remove print

* small cleanup

* default behavior for baseTeamManager

* add back statsrecorder

* update

* Team manager prototype  (Unity-Technologies#4850)

* remove group id

* very rough sketch for TeamManager interface

* add team manager id to proto

* team manager for hallway

* add manager to hallway

* send and process team manager id

* remove print

* small cleanup

Co-authored-by: Chris Elion <[email protected]>

* Remove statsrecorder

* Fix AgentProcessor for TeamManager
Should work for variable decision frequencies (untested)

* team manager

* New buffer layout, TeamObsUtil, pad dead agents

* Use NaNs to get masks for attention

* Add team reward to buffer

* Try subtract marginalized value

* Add Q function with attention

* Some more progress - still broken

* use singular entity embedding (Unity-Technologies#4873)

* I think it's running

* Actions added but untested

* Fix issue with team_actions

* Add next action and next team obs

* separate forward into q_net and baseline

* might be right

* forcing this to work

* buffer error

* COMAA runs

* add lambda return and target network

* no target net

* remove normalize advantages

* add target network back

* value estimator

* update coma config

* add target net

* no target, increase lambda

* remove prints

* cloud config

* use v return

* use target net

* adding zombie to coma2 brnch

* add callbacks

* cloud run with coma2 of held out zombie test env

* target of baseline is returns_v

* remove target update

* Add team dones

* ntegrate teammate dones

* add value clipping

* try again on cloud

* clipping values and updated zombie

* update configs

* remove value head clipping

* update zombie config

* Add trust region to COMA updates

* Remove Q-net for perf

* Weight decay, regularizaton loss

* Use same network

* add base team manager

* Remove reg loss, still stable

* Black format

* add team reward field to agent and proto

* set team reward

* add maxstep to teammanager and hook to academy

* check agent by agent.enabled

* remove manager from academy when dispose

* move manager

* put team reward in decision steps

* use 0 as default manager id

* fix setTeamReward

Co-authored-by: Vincent-Pierre BERGES <[email protected]>

* change method name to GetRegisteredAgents

* address comments

* Revert C# env changes

* Remove a bunch of stuff from envs

* Remove a bunch of extra files

* Remove changes from base-teammanager

* Remove remaining files

* Remove some unneeded changes

* Make buffer typing neater

* AgentProcessor fixes

* Back out trainer changes

* use delegate to avoid agent-manager cyclic reference

* put team reward in decision steps

* fix unregister agents

* add teamreward to decision step

* typo

* unregister on disabled

* remove OnTeamEpisodeBegin

* change name TeamManager to MultiAgentGroup

* more team -> group

* fix tests

* fix tests

* Use attention tests from master

* Revert "Use attention tests from master"

This reverts commit 78e052b.

* Use attention from master

* Renaming fest

* Use NamedTuples instead of attrs classes

* Bug fixes

* remove GroupMaxStep

* add some doc

* Fix mock brain

* np float32 fixes

* more renaming

* Test for team obs in agentprocessor

* Test for group and add team reward

* doc improve

Co-authored-by: Ervin T. <[email protected]>

* store registered agents in set

* remove unused step counts

* Global group ids

* Fix Trajectory test

* Remove duplicated files

* Add team methods to AgentAction

* Buffer fixes

(cherry picked from commit 2c03d2b)

* Add test for GroupObs

* Change AgentAction back to 0 pad and add tests

* Addressed some comments

* Address some comments

* Add more comments

* Rename internal function

* Move padding method to AgentBufferField

* Fix slicing typing and string printing in AgentBufferField

* Fix to-flat and add tests

* Rename GroupmateStatus to AgentStatus

* Update comments

* Added GroupId, GlobalGroupId, GlobalAgentId types

* Update comment

* Make some agent processor properties internal

* Rename add_group_status

* Rename store_group_status, fix test

* Rename clear_group_obs

Co-authored-by: Andrew Cohen <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>
Co-authored-by: Chris Elion <[email protected]>
Co-authored-by: andrewcoh <[email protected]>
Co-authored-by: Vincent-Pierre BERGES <[email protected]>

* Removing some scenes (Unity-Technologies#4997)

* Removing some scenes, All the Static and all the non variable speed environments. Also removed Bouncer, PushBlock, WallJump and reacher. Removed a bunch of visual environements as well. Removed 3DBallHard and FoodCollector (kept Visual and Grid FoodCollector)

* readding 3DBallHard

* readding pushblock and walljump

* Removing tennis

* removing mentions of removed environments

* removing unused images

* Renaming Crawler demos

* renaming some demo files

* removing and modifying some config files

* new examples image?

* removing Bouncer from build list

* replacing the Bouncer environment with Match3 for llapi tests

* Typo in yamato test

* Fix issue with queuing input events that stomp on others. (Unity-Technologies#5034)

Co-authored-by: Chris Elion <[email protected]>
Co-authored-by: Chris Goy <[email protected]>
Co-authored-by: Marwan Mattar <[email protected]>
Co-authored-by: vincentpierre <[email protected]>
Co-authored-by: andrewcoh <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>
Co-authored-by: Ervin Teng <[email protected]>
Co-authored-by: Andrew Cohen <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@andrewcoh andrewcoh andrewcoh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dongruoping @chriselion @andrewcoh