Update DQN Colab example to support export for Barracuda inference #5643
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…X model that works for inference. -Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case. -Add GridWorldColab.onnx file, trained and generated with the colab.
…fied) GridWorld Scene. -permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW. -update input_shape indexes and remove equivalent permute from the VisualQNetwork -fix reference to global qnet inside generate_trajectories (use the passed in version) -create wrapper network to add required constants and dummy input masks required by the runtime inference (Barracuda) -export the wrapped network to ONNX format compatible with Barracuda
…l sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example.
maryamhonari
approved these changes
Dec 21, 2021
jrupert-unity
changed the title
maryamhonari
added a commit
that referenced
this pull request
Nov 18, 2022
* yamato promotion min editor bumped to 2020.3 (#34) * Fix pypi publish github action (#35) * update logo (#39) * Move SB3 dependencies to colab (#29) * Basic example running with multiple envs in sb3 * Merge & cleanup * trains 3dball * move SB3 helpers into colab * remove sb3/supersuit hard reqs * remove sb3 funcs * remove unused imports * remove sb3 example directory and script * remove vec env tests * clean colab * fix link Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: mahon94 <[email protected]> * bump to barracuda 3.0.0 (#30) * bump to barracuda 3.0.0 & Updated submodule com.unity.ml-agents * Feature/mla 2394 update web docs (#37) * Added mkdocs and refactored docs. * Updated docs. * Added gh action to manually public docs. Updated mkdocs.yml. * fix markdown docs linking issue (#40) * Updated submodule com.unity.ml-agents (#41) * Add sonarqube yamato job (#38) * Pinning markdown link check to 3.8.7 on nightly gha. (#43) * Update submodules * pin black version to 22.3.0 (#46) * fix-sonar-bugs (#44) * Update submodules * bump version to 0.29.0 (#42) * bump version to 0.29.0 * remove dev tag * add checks for pytorch-gpu install * pin black version to 22.3.0 * Update submodules * Update automated trigger references to support develop branch (#36) * Update automated trigger references to support develop branch * Update pull request template links to refer to develop branch. * Fix release branch name for github actions * Update pre-commit pattern * Removing test that no longer applys to this repo * Removed dead links in pull request template. (#48) * Use pinned version of package-ci/mac image (#50) * Add shared critic configurability for PPO (#45) * Update submodules * Updated Unity version to 2020.3.33f1. (#53) * Update submodules (#56) Co-authored-by: GitHub Actions - update submodules <[email protected]> * Add poca reference and update to latex refs (#57) * add poca ref in docs (#58) * Update submodules * Update Training-Configuration-File.md (#59) * Update Training-Configuration-File.md Doc says "behavior_cloning" when it should say "behavioral_cloning". * Update Learning-Environment-Design-Agents.md Minor correction to wording. (https://github.com/Unity-Technologies/ml-agents/issues/5745) * rename ml-agents/trainers/torch to torch_entities (#55) * rename ml-agents/trainers/torch to torch_entities * fix mock patch * Update submodules * Update docs to explain Alternating Ray Order switch (mla-2537) (#61) * Update docs to explain Alternating Ray Order switch (mla-2537) * Update submodules * Update submodules * Updated validate meta files. (#65) * Fix failing gha pytest (#66) * Updated pytest. * Updated SURVEY.md to remove dead survey link. * Updated bug_report.md to update dead link. * Update submodules * Fixed upm ci bug in yamato tests. (#67) * Update submodules * remove torch shadow in trainers tests (#69) * extend stale bot to 90 days (#70) * Update submodules * Dev bump min ver (#74) * Bumped minimum version of Unity. * Bumped minimum version of python and pytorch. fixed issues with deprecation warnings for x.T (tensor transpose). * Fixed np.bool deprecation warning. This is a port of an external PR: https://github.com/Unity-Technologies/ml-agents/pull/5795. (#71) * Updated submodule. * Bumped max ver of python. * Bumped to 2021.3 in yamato tests. * Fixed version issue with torch in test constraints. * update sysroot version * update toolchain version Co-authored-by: zhuo <[email protected]> * Removed old backwards compatibility test since tensorflow/2019.4 are no longer s upported. (#76) * Develop custom trainers (#73) * Make create_policy more generic (#54) * add on/off policy classes and inherit from * trainers as plugins * remove swap files * clean up registration debug * clean up all pre-commit * a2c plugin pass precommit * move gae to trainer utils * move lambda return to trainer util * add validator for num_epoch * add types for settings/type methods * move create policy into highest level api * move update_reward_signal into optimizer * move get_policy into Trainer * remove get settings type * dummy_config settings * move all stats from actor into dict, enables arbitrary actor data * remove shared_critic flag, cleanups * refactor create_policy * remove sample_actions, evaluate_actions, update_norm from policy * remove comments * fix return type get stat * update poca create_policy * clean up policy init * remove conftest * add sharedecritic to settings * fix test_networks * fix test_policy * fix test network * fix some ppo/sac tests * add back conftest.py * improve specification of trainer type * add defaults fpr trainer_type/hyperparam * fix test_saver * fix reward providers * add settings check utility for tests * fix some settings tests * add trainer types to run_experiment * type check for arbitary actor data * cherrypick rename ml-agents/trainers/torch to torch_entities (#55) * make all trainers types and setting visible at module level * remove settings from run_experiment console script * fix test_settings and upgrade config scripts * remove need of trainer_type argument up to trainefactory * fix gohst trainer behavior id in policy Queue * fix torch shadow in tests * update trainers, rl trainers tests * update tests to match the refactors * fixing behavior name in ghost trainer * update ml-agents-envs test configs * separating the plugin package changes * bring get_policy back for sake of ghost trainer * add return types and remove unused returns * remove duplicate methods in poca (_update_policy, add_policy) Co-authored-by: mahon94 <[email protected]> * Online/offline custom trainer examples with plugin system (#52) * add on/off policy classes and inherit from * trainers as plugins * a2c trains * remove swap files * clean up registration debug * clean up all pre-commit * a2c plugin pass precommit * move gae to trainer utils * move lambda return to trainer util * add validator for num_epoch * add types for settings/type methods * move create policy into highest level api * move update_reward_signal into optimizer * move get_policy into Trainer * remove get settings type * dummy_config settings * move all stats from actor into dict, enables arbitrary actor data * remove shared_critic flag, cleanups * refactor create_policy * remove sample_actions, evaluate_actions, update_norm from policy * remove comments * fix return type get stat * update poca create_policy * clean up policy init * remove conftest * add sharedecritic to settings * fix test_networks * fix test_policy * fix test network * fix some ppo/sac tests * add back conftest.py * improve specification of trainer type * add defaults fpr trainer_type/hyperparam * fix test_saver * fix reward providers * add settings check utility for tests * fix some settings tests * add trainer types to run_experiment * type check for arbitary actor data * cherrypick rename ml-agents/trainers/torch to torch_entities (#55) * make all trainers types and setting visible at module level * remove settings from run_experiment console script * fix test_settings and upgrade config scripts * remove need of trainer_type argument up to trainefactory * fix gohst trainer behavior id in policy Queue * fix torch shadow in tests * update trainers, rl trainers tests * update tests to match the refactors * fixing behavior name in ghost trainer * update ml-agents-envs test configs * fix precommit * separating the plugin package changes * bring get_policy back for sake of ghost trainer * add return types and remove unused returns * remove duplicate methods in poca (_update_policy, add_policy) * add a2c trainer back * Add DQN cleaned up trainer/optimizer * nit naming * fix logprob/entropy types in torch_policy.py * clean up DQN/SAC * add docs for custom trainers,TODO: refrence tutorial * add docs for custom trainers,TODO: refrence tutorial * add clipping to loss function * set old importlim-metadata version * bump precomit hook env to 3.8.x * use smooth l1 loss Co-authored-by: mahon94 <[email protected]> * add tutorial for validation * fix formatting errors * clean up * minor changes Co-authored-by: Andrew Cohen <[email protected]> Co-authored-by: zhuo <[email protected]> * Trainer qa fix (#78) * grammer fixes * fix nit comments from QA * add info about on/off policy * add more context to the code block * more context and minor fix * pip3 Co-authored-by: zhuo <[email protected]> * Develop merge submodule (#77) * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Arthur Juliani <[email protected]> * Update changelog * [MLA-1762] reduce memory allocations from DiscreteActionOutputApplier (#4922) * add option for Burst inference (#4925) * surface specific GRPC errors more visibly (#4930) * Add additional logic to avoid load being called on every advance (#4934) * [MLA-1767] Refactor communicator connection exceptions (#4935) * Update changelog for release 13. (#4938) * Update master versions for release 13. (#4945) * Release 13 versions. (#4946) - updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation. * Update docs to pass doc validation. (#4953) * update defines, compile out Initialize body on non-desktop (#4957) * Adding references to the Extensions package to help promote it. (#4967) * Adding references to the Extensions package to help promote it. * Updating header formatting to match the rest of the page * Adding references to the Extensions package to help promote it. (#4967) (#4968) Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Chris Elion <[email protected]> * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) (#4974) * pass sensor dimension flags to analytics (#4954) * Communicator factory (#4965) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) (#4981) * Set ignore done=False in GAIL (#4971) * MultiAgentGroup Interface (#4923) * add SimpleMultiAgentGroup * add group reward field to agent and proto * Make TrainingAnalyticsSideChannel internal (#4999) * [MLA-1783] built-in actuator type (#4950) * Add component menues for some sensors and actuators. (#5001) * Add component menues for some sensors and actuators. (#5001) (#5002) * Merge master -> release_13_branch-to-master * Fix RpcCommunicator merge. * master -> main. (#5010) * Adding a name field to BufferSensorComponent * Adding a note to the CHANGELOG about var len obs * Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders * Adding a custom editor for BufferSensorComponent * adding inheritdoc * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update versions for release 14 hotfix. (#5040) * master -> main. (#5010) (#5044) * Update changelog. (#5045) * [MLA-1809] catch mismatched observation sizes (#5030) * Update changelog. (#5055) * Fix xml docs. (#5057) * pass sensor name through to ObservationSpec (#5036) * Remove unused allocation (#5068) * Automatically generate samples based on placement of mlagents-sample.json files in our examples. (#5077) * Update barracuda to 1.3.2-preview. (#5084) * POCA trainer (#5005) Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: Vincent-Pierre BERGES <[email protected]> * [docs] Documentation for POCA and cooperative behaviors (#5056) Co-authored-by: Ruo-Ping Dong <[email protected]> * [docs] Update changelog (#5095) * Add multiAgentGroup capabilities flag (#5096) * Add multiAgentGroup capabilities flag * Add proto * Fix compiler error * Add warning for multiagent group * Add comment * Fix spelling mistake * Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <[email protected]> * [cherry-pick] Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <[email protected]> * Update changelog for samples. (#5103) * Update changelog for samples. (#5103) (#5106) * Update versions on main (#5102) Increment versions after release 15 branch split * [release_15] Release 15 update versions (#5101) * Update versions * Fix for validate release links * Update release tag and docs * [docs] Add missing package docs (#5108) * Add missing package docs * Fix grammar * Update comment on Dispose * Fix typo in doc * [docs] Update readme and changelog (#5107) * Make analytics module an optional dependency. (#5109) Co-authored-by: Chris Elion <[email protected]> * Redo dotnet format (#5119) * [docs] Add links to example envs for Buffer Sensor and MultiAgentGroup (#5116) * Update changelog with links to example envs * Cooperative push block link update * [cherry-pick] Integrate Group Manager to soccer/retrain with POCA (#5115) (#5121) * Integrate Group Manager to soccer/retrain with POCA (#5115) * Add Soccer env to changelog Co-authored-by: andrewcoh <[email protected]> * [cherry-pick][docs] Add Dungeon Escape Environment (#5133) * Add DungeonEscape POCA Environment (#5128) * Add DungeonEscape assets from working branch * Add Dungeon Escape docs * Create dungeon_escape.png * Add to docs Co-authored-by: Hunter-Unity <[email protected]> * update pre-commit instructions for CONTRIBUTING.md (#5130) * Remove duplicated changelog * Fix another duplicated line in the changelog... * Removing Obsolete methods from the package (#5024) * Removing Obsolete methods from the package * Missing depecration and modified changelog * Readding the obsolete BrainParameter methods, will need a larger discussion on these * Removing Action Masker, readding the warining when using a non-implemented Heuristic, Removing NumAction from Brain Parameters * removing documentation and some calls to deprecated methods in the extensions package * Editing the Changelog to put the unreleased on top * non-IEnumerable interface for action masking (#5060) * V2 staging new model version (#5080) * Make modelCheck have flavors of error messages * ONNX exporter v3 * Using a better CheckType and a switch statement * Removing unused message * More tests * Use an enum for valid versions and use GetVersion on model directly * Maybe the model export version a static constant in Python * Use static constructor for FailedCheck * Use static constructor for FailedCheck * Modifying the docstrings * renaming LegacyDiscreteActionOutputApplier * removing testing code * better warning message * Nest the CheckTypeEnum into the FailedCheck class * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs * Adding a line explaining that legacy tensor checks are for versions 1.X only * Modifying the changelog * Exporting all the branches size instead of omly the sum (#5092) * addressing comments * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs Co-authored-by: Chris Elion <[email protected]> * readding tests * Adding a comment around the new DiscreteOutputSize method * Clearer warning : Model contains unexpected input > Model requires unknown input * Fixing a bug in the case where the discrete action tensor does not exist Co-authored-by: Chris Elion <[email protected]> * Make IActuator implement IHeuristicProvider. (#5110) * Adding more tests for v2 models (#5138) * Adding more tests for v2 models * formatting * fixing formatting * [MLA-1634] Add ObservationSpec and update ISensor interfaces (#5127) * Add custom package settings (#5027) * cleanup: yamato triggers, changelog, docstrings * clean up merge * fix compile error * update c# version to 2.0.0-exp.1 (#5160) Co-authored-by: Chris Elion <[email protected]> * Move physics/sensor tests that fail in edit mode depending on what's in the scene to playmode tests. (#5162) * update package version * remove defines that are now always true * changelog * Remove all old analytics defines. (#5168) * [MLA-1634] Compression spec (#5164) * Low hanging fruit tests for coverage. (#5170) * Update Policies to conditionally compile their analytics method. (#5171) * Turns physics modules into optional dependencies. (#5112) * [MLA-1634] Remove SensorComponent.GetObservationShape() (#5172) * PR Feedback. (#5173) * change default barracuda behavior (#5175) * [MLA-1824] make SensorComponent return ISensor[] (#5181) * Make SensorComponent return an array * split match3 sensors, partial retrain * docstrings, migration, changelog, cleanup * Adding the goal conditioning sensors with the new observation specs (#5159) * Fixing networks.py for the merge * fix compile error * Adding the goal conditioning sensors with the new observation specs * addressing feedback * I forgot to change the m_observationType * Renaming Goal to GoalSignal (#5190) * Renaming GOAL to GOAL_SIGNAL * VectorSensorComponent to use new API * Adding docstrings * verbose pytest on github action Co-authored-by: Chris Elion <[email protected]> * [MLA-1879] culture-invariant sorting for sensors and actuators (#5194) * Goal conditioning grid world : Example of goal conditioning (#5193) * Aded the Goal conditioned GridWorld to replace regular gridworld * adding missing files * Code improvements * Documentation change on gridworld * resolving conflicts * new model * Addressing comments * comments and renames * Update docs/Learning-Environment-Examples.md Co-authored-by: Ervin T. <[email protected]> * adding reference to gridworld in docs about goal signal Co-authored-by: Chris Elion <[email protected]> Co-authored-by: Ervin T. <[email protected]> * [MLA-1584] Match3 variable board size (#5189) * [bug-fix] When agent isn't training, don't clear update buffer (#5205) * Don't clear update buffer, but don't append to it either * Update changelog * Address comments * Make experience replay buffer saving more verbose * Changelog for goal conditioning (#5195) * Changelog for goal conditioning * adding a line about the changes to the gridworld example * Addressing comments * Change capitalization * [bug fix] Fix warning using demo recorder (#5216) * Making VectorActionSize and VectorActionSpaceType internal (#5214) Made sure the editor does not complain; * Turning some logger.info into logger.debug and remove some logging overhead when not using debug (#5211) * turning some logger.info into logger.debug and remove some logging overhead when not using debug * Addressing comments * Adding to changelog * rearrange match3 docs (#5215) * rearrange match3 docs * changelog and migration from previous PR * [bug-fix] Fix POCA LSTM, pad sequences in the back (#5206) * Pad buffer at the end * Fix padding in optimizer value estimate * Fix additional bugs and POCA * Fix groupmate obs, add tests * Update changelog * Improve tests * Address comments * Fix poca test * Fix buffer test * Increase entropy for Hallway * Add EOF newline * Fix Behavior Name * Address comments * [MLA-1880] Raycast sensor interface improvements (#5222) * WIP * remove debug info struct * cleanup + add to test * changelog * fix unit tests * PR feedback * Load individual elements if state dict load fails (#5213) Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: Ervin T. <[email protected]> * [MLA-1159] Add virtual methods to DecisionRequester (#5223) * [Bug Fix] Serialization of vectorActionSpace (#5220) * [Bug Fix] Serialization of vectorActionSpace * adding a test * Clear ActionBuffers before Heuristic calls (#5227) * Add ObservationType to analytics ObservationSpec (#5218) * [MLA-1909] Match3 and Camera/RenderTexture sensor GC improvements (#5233) * avoid empty set iteration, avoid Debug.AssertFormat (#5246) * avoid empty set iteration, avoid Debug.AssertFormat * changelog * Update to barracuda 1.3.3 and changes to the model inputs and outputs for LSTM (#5236) * Initial commit * making it work with hallway. Added a new model version * addressing feedback * Adding a comment * formatting * modifying the changelog * Adding descriptions on the model version descriptions * Update com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs Co-authored-by: Chris Elion <[email protected]> * addressing comments Co-authored-by: Chris Elion <[email protected]> * [Release 16] Release 16 Merge Back to Main (#5255) Update versions and documentation for Release 16. Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Chris Elion <[email protected]> * fix link, run link check in quiet mode (#5257) * [MLA-1912] Move Match3 to com.unity.ml-agents (#5259) * Move GridSensor into main package (#5256) * move OneHotGridSensor into main package * changelog and migration guide * remove old doc * check if physics module presents * Add advanced installation guide for com.unity.ml-agents (#5260) * Fix errors when creating new RayCast sensor (#5261) * check if tag and angle is null * add test * changelog * fix all PVS and doc generation warnings (#5262) * Fix --results-dir (#5269) * Fix GridSensorComponent bug (#5270) * LSTM models from 1.x will be incompatible with MLA 2.x (#5254) * LSTM models from 1.x will be incompatible with MLA 2.x * Adding a test and a new v2 model * Make the Model Runner raise an error if using 1.0 model with LSTM * adding a new model for hallway trained with 2.0 * reword error messages * Only raise if error, not if warning * Addressing comments: The legacy Barrauda memory generator and applier were removed. All code that checked for (memories + v1.X) have been removed since these will no longer be supported * Modifying the changelog and the migrating guide with this change * Fixing the merge issues Co-authored-by: Chris Elion <[email protected]> * [MLA-1939] Match3 Custom Editor (#5263) * Rider suggested cleanup, part 1 (#5265) * Fix the attention module embedding size (#5272) * Fix the attention module embedding size * editing the changelog * [bug-fix] Fix null ref exception in the demo recorder (#5274) * Fix null ref exception in the demo recorder * Update changelog * Clearer logic * Modifying the changelog for release 17 (#5277) * Modifying the changelog for release 17 * Change version to 2.0.0-exp.1 * Removing the [Unreleased] section * check for missing AbstractBoard, display warning (#5276) * Updating the barracuda 1.4.0 (#5291) * [debug] Require all behavior names to have a matching YAML entry (#5210) * Add strict check to settings.py * Remove warning from trainer factory, add test * Add changelog * Fix test * Update changelog * Remove strict CLI options * Remove strict option, rename, make strict default * Remove newline * Update comments * Set default dict to actually default to a default dict * Fix tests * Fix tests again * Default trainer dict to requiring all fields * Fix settings typing * Use logger * Add default_settings to error * Version bump main (#5279) * Version bump for the main branch following branching of release_17 * Messed up the extensions package version * Bring back root reference in grid sensor (#5300) * [MLA-1952] Add optional seed for gym action spaces (#5303) * add optional seed for action spaces * add optional seed for action spaces * changelog * undo packages-lock.json change * Release 17 version bumps and docs version bumps (#5280) * Migration guide - list removed APIs with replacements (#5307) * list removed APIs with replacements * fixes * [docs] Update changelog (#5308) * [MLA-1952] Add optional seed for gym action spaces (#5303) (#5315) * Fix stacked grid sensor (#5335) * Update to Unity Package licenses (#5340) * Update to Unity Package licenses Per Dan * Updating the main repo license dates. * Adding a fully connected visual encoder for super small visual input + tests (#5351) * initial commit for a fully connected visual encoder * adding a test * addressing comments * Fixing error with minimal size of fully connected network * adding documentation and changelog * Make OverlapChecker an interface (#5324) * Add stacking option to VectorSensorComponent (#5376) * [WIP] [Fix] Fixing collect observation called on done (#5375) * [WIP] [Fix] Fixing collect observation called on done * Update com.unity.ml-agents/Runtime/Agent.cs * :warning: Modifying the test of stacking sensor when the agent is done * modifying the documentation for BufferSensor to specify to call AddObservation in the CollectObservations method * Fix NullReferenceException for Behavior Parameters without Agent (#5382) * check agent existence. add warning box * changelog * [WIP] 2.0 verified to main (#5385) * Warn if no joints on Root Body (#5387) * Modifying the changelog for the continuous action SAC target entropy fix (#5394) * Editing the GridSensor documentation for 2D use case (#5396) * Editing the GridSensor documentation for 2D use case * changing chagelog * [MLA-2017] Move colab notebooks to github (#5399) * Fix the reporting of histogram stats and adding a test (#5410) * Fix the reporting of histogram stats and adding a test * Appending to the Changelog * Update changelog before Release 18 branch (#5412) * Update package versions for main branch (#5413) * [Release 18] Update versions and links (#5414) * add changelog entries for PRs that were missing them before. (#5416) * fix cref error found by doc validation (#5421) * fix release versions and changelog section * [bug-fix] Fix when group terminal steps are deleted, robust test (#5441) * Fix when terminal steps are deleted, robust test * Update changelog * Fix test comment * Fix torch device option in argparser (#5467) * [FIX] prioritize cli options over yaml (#5495) * prioritize cli over yaml in checkpointSettings * prioritize resume if both set in one place * fixed test_commandline_args * addressing comments:renaming, commenting * more tests and referenced in change_log * Only update the native library and the c# calls that need to be made to clean up channels. (#5283) * Fix Mac backcompat test (#5519) * Initialize-from custom checkpoints (#5525) * init from any checkpoint including older ones * moving init_path logic ahead to learn.py * fixing pytest to take the full path * doc & changelog * Update gym version to 0.20.0 (#5540) * Fixed failing test due to gym 0.20.0 release. Updated gym-unity package to fix gym version to 0.20.0. * Updated CHANGELOG. * Fix VAIL (#5546) * Custom trainer editor analytics (#5511) * Custom trainer editor analytics * inherit the default_training_analytics_side_channel * Feature/mla 2205 separate schedule lr beta epsilon (#5538) * Added initial (untested) support for separating schedule for beta, LR, and epsilon. * Added support for beta and epsilon schedule in test config conversion * Updated docs. Set epsilon to be linear by default. * Updated configs to add beta and epsilon schedule where appropriate. * Addressed CR change requests. * Minor update. * Updated docs. * Added changes to CHANGELOG. * Updated CHANGELOG. * Fixed typo in Training-Configuration-File.md * Reverted config changes to remove beta and epsilon schedules. * mypy error fixed from pre-commit. * Fixed a bug that was causing a settings test to fail. * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> * Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512) * Hash128 is not a cryptographic hash, replace with HMAC-SHA256. * Extend TrainingAnalytics side channel to expose configuration details * Change member function scopes and hash demo_paths * Extract tbiEvent hashing method and add test coverage * Restart crashed Unity environments (#5553) ### Change(s) Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash. When a worker receives an ENV_EXITED signal it will now: 1. Record all failures coming through the step queue and drop all other messages. 2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted. 3. Restart all failed workers (up to a configurable limit) This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values. ⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times). ```yaml env_settings: # Can restart 10 times over the lifetime of the experiment. max_lifetime_restarts: 10 # Rate limit of 1 failure per 60s restarts_rate_limit_n: 1 restarts_rate_limit_period_s: 60 ``` They can of course be passed via CLI arguments as well ```bash --max-lifetime-restarts --restarts-rate-limit-n --restarts-rate-limit-period-s ``` ### Disabling this feature * Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1` * Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1` * Develop add training area replicator (#5568) * Added training area replicator to com.unity.ml-agents package. * Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas. * Changed training area replicator to size grid automatically from number of areas. * Added tests for the training area replicator. * Added setup for tests for the training area replicator. * Added comments and updated create tutorial for training area replicator. * Updated CHANGELOG. * Fixed some failing tests. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Henry Peteet <[email protected]> * Update docs/Learning-Environment-Create-New.md Co-authored-by: Henry Peteet <[email protected]> * Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs Co-authored-by: Henry Peteet <[email protected]> * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: Henry Peteet <[email protected]> * Fixing the analytics side-channel for curriculum learning. (#5586) * Fixing the analytics side-channel for curriculum learning. * Made a more robust test. * Update the changelog. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Maryam Honari <[email protected]> * Method to return stacked observations (#5547) * Method to return stacked observations * Added testing and returning staked observation flat. * Update the comment lines. * Remove brainstorm commits. * Upgrade Barracuda to 2.3.1-preview (#5591) * Upgrade to 2.3.1-preview * Fix for critic normalization bug (#5595) * Added normalization to critic during training for poca, ppo, and sac. * Fixed critic normalization bug. Added new tests to cover. * Updated CHANGELOG. * Fixed typo in test_trainers.py * Deterministic actions python training (#5619) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <[email protected]> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <[email protected]> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: cmard <[email protected]> * Revert "Deterministic actions python training (#5619)" (#5622) This reverts commit 9ea962027d4c581de7c45a07d669f96453236c5c. * Deterministic actions python training (#5626) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <[email protected]> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <[email protected]> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md * Fix the deterministic showing up all the tiime (#5621) Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: cmard <[email protected]> * Changelog cleanup. (#5627) * Release cleanup for changelog * Update CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> * Fix analytics error when compiling XBOX and PS5 (#5628) * Add ENABLE_CLOUD_SERVICES_ANALYTICS Guards To use analytics safely you must wrap usage with `#if ENABLE_CLOUD_SERVICES_ANALYTICS` See https://docs.unity3d.com/ScriptReference/Analytics.Analytics.html * Update DQN Colab example to support export for Barracuda inference (#5643) * Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference. -Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case. -Add GridWorldColab.onnx file, trained and generated with the colab. * Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene. -permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW. -update input_shape indexes and remove equivalent permute from the VisualQNetwork -fix reference to global qnet inside generate_trajectories (use the passed in version) -create wrapper network to add required constants and dummy input masks required by the runtime inference (Barracuda) -export the wrapped network to ONNX format compatible with Barracuda * Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example. * Minor edit in Colab docs. * Delete unnecessary .lighting file. * Update changelog. * Update colab/Colab_UnityEnvironment_2_Train.ipynb Co-authored-by: Maryam Honari <[email protected]> * Fix error in colab metadata name. * Add suggested comments and fix incorrect parameter ordering. * Update docs for Elo curriculum completion measure. Co-authored-by: Maryam Honari <[email protected]> * Changelog prepped for release (#5629) * Changelog prepped for release * Added missing PRs to the changelog. * Added missing CHANGELOG item. * enable 3D Ball sample import button in package manager for unity sdk (#5665) * enable 3D Ball sample import button in package manager for unity sdk * add Samples folder * add meta file * fix git ignore config * Updated versions for main. Also updated validate versions script to r… (#5668) * Updated versions for main. Also updated validate versions script to remove exp.1 from package version. * Reverted validate versions. * Updated versions. * Bumped package version for patch. * Removed unreleased section. (#5667) * Updated release versions and release links. (#5669) * Updates for package validation errors. (#5671) * Release 19 branch lts deprecation (#5673) * Updated Project and DevProject to 2020.3 LTS. * Updated supported LTS version to 2020.3. * Updated changelog. * Bumped up CI Unity version from 2020.2 to 2020.3. * Update CHANGELOG.md * Removing VS Studio dependency from manifest.json. * Update packages-lock.json * Update CHANGELOG.md * Bumped package version for patch. * Fixed build-docker and updated CHANGELOG to add unreleased section. (#5682) * Fixed build-docker and updated CHANGELOG to add unreleased section. * Updated backward compat mac training yamato test. (#5685) * Develop deprecate python 3.6 (#5) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * Updated failing yamato jobs. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Develop python api ga (#6) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <[email protected]> * Updated failing yamato jobs. * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <[email protected]> * Updated CHANGELOG. * Updated Migration guide. * Doc updates based on CR. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Fixed yamato import error. Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: jmercado1985 <[email protected]> Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: mahon94 <[email protected]> Co-authored-by: Andrew Cohen <[email protected]> * Refactoring for extraction to separate repo. * removing unused github workflows. * Updates for pre-commit and github actions. * Fixed bug with validate_meta_files.py missing a file to exclude. * Updated precommit gh action. * Refactored yamato pack. * Added yamato test yaml. * Updated to CHANGELOG and VectorSensor from mono repo before swap. * Pin dotnet version for GHA formatting. * Refactor for MLA pro support. (#1) * Refactored for MLA pro support. * Fixed bug in yamato test. * Updated the CHANGELOG. * Fixed trigger logic in yamato test. * Fixed failing tests. (#2) * Fixed failing tests. * Adding meta files. * Fix execution order (#4) * Fixed execution order. * Fixed bug with CRT build. * precommit formatting update. * Updated agent Awake docstring. * Set execution order to be negative to make sure scripts run before default time. (#5) * Added #if to exclude communicator registration in the Agent awake script (#6) * Add visibility to pro package (#7) * add coverage publish promotion yamato jobs (#3) * bump barracuda to 3.0.0 (#9) upgrade to barracuda 3.0.0 * Updated README. (#8) * add validation test dep. to publish job (#12) * add validation test dep. to publish job * fix package path * Update automated trigger rules to support oneflow develop branch (#11) * Update automated trigger rules to support oneflow develop branch * Fix release branch name for github actions * Release 2.3.0 exp.1 (#13) * upgrade package to 2.3.0-exp.1 * rebase on develop branch * update changelog * remove unreleased section (#16) * Updated experimental prerelease version and changelog. (#18) * Added submodule update gha. (#19) * fix bugs from sonar-scanner (#21) * add ml-agents-pro as a upstream for submodule update (#22) * Added support for standalone use of Academy with refactored communicator factory. (#24) * clean commandline Arguments (#25) * clean commandline Arguments * fix precommit * update unrelease section in changelog (#27) * MLA-2537 Reorder RayPerceptionSensor output to reflect its spatial structure. (#26) * Changes to support in order rays. GUI for switching to this. Updated tests. * Updated README. (#29) * Fixed samples. (#30) * Added camera sensor component checkbox for runtime enablement. update… (#31) * Added camera sensor component checkbox for runtime enablement. updated camera sensor to only be active when enabled for runtime. * Updated yamato job. * Updated CHANGELOG. * Fixed issue with Academy not building for webgl builds. (#32) * Dev bump min ver (#33) * Bumped min version of Unity to 2021.3. * Updated min version of python in pre-commit. Updated changelog. * Moving files into packages/com.unity.ml-agents directoy. * Remove com.unity.ml-agents submodule. * Added back in ml-agents package. * Removed packages temp folder. * Updated yamato protobuf gen. * Updated yamato to remove submodule update. * Updated linux tool chain. * Updated Project package dependencies. * Updated flake8 repo link in precommit. Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: Arthur Juliani <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Chris Goy <[email protected]> Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Hunter-Unity <[email protected]> Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Jason Bowman <[email protected]> Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: cmard <[email protected]> Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: Jason Rupert <[email protected]> Co-authored-by: Xun Cao <[email protected]> Co-authored-by: jmercado1985 <[email protected]> Co-authored-by: mahon94 <[email protected]> Co-authored-by: Andrew Cohen <[email protected]> Co-authored-by: Xun Cao <[email protected]> Co-authored-by: Jason Rupert <[email protected]> * Release 20 changelog (#80) * clean changelog * match extention package dependency * fix typos Co-authored-by: Xun Cao <[email protected]> Co-authored-by: Brandon Henry <[email protected]> Co-authored-by: Andrew Cohen <[email protected]> Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: GitHub Actions - update submodules <[email protected]> Co-authored-by: Jason Bowman <[email protected]> Co-authored-by: Florence Rolland <[email protected]> Co-authored-by: Jason Rupert <[email protected]> Co-authored-by: zhuo <[email protected]> Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: Arthur Juliani <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Chris Goy <[email protected]> Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Hunter-Unity <[email protected]> Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: cmard <[email protected]> Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: Jason Rupert <[email protected]> Co-authored-by: Xun Cao <[email protected]> Co-authored-by: jmercado1985 <[email protected]>
miguelalonsojr
added a commit
that referenced
this pull request
Dec 2, 2022
* Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Arthur Juliani <[email protected]> * Update changelog * [MLA-1762] reduce memory allocations from DiscreteActionOutputApplier (#4922) * add option for Burst inference (#4925) * surface specific GRPC errors more visibly (#4930) * Add additional logic to avoid load being called on every advance (#4934) * [MLA-1767] Refactor communicator connection exceptions (#4935) * Update changelog for release 13. (#4938) * Update master versions for release 13. (#4945) * Release 13 versions. (#4946) - updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation. * Update docs to pass doc validation. (#4953) * update defines, compile out Initialize body on non-desktop (#4957) * Adding references to the Extensions package to help promote it. (#4967) * Adding references to the Extensions package to help promote it. * Updating header formatting to match the rest of the page * Adding references to the Extensions package to help promote it. (#4967) (#4968) Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Chris Elion <[email protected]> * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) (#4974) * pass sensor dimension flags to analytics (#4954) * Communicator factory (#4965) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) (#4981) * Set ignore done=False in GAIL (#4971) * MultiAgentGroup Interface (#4923) * add SimpleMultiAgentGroup * add group reward field to agent and proto * Make TrainingAnalyticsSideChannel internal (#4999) * [MLA-1783] built-in actuator type (#4950) * Add component menues for some sensors and actuators. (#5001) * Add component menues for some sensors and actuators. (#5001) (#5002) * Merge master -> release_13_branch-to-master * Fix RpcCommunicator merge. * master -> main. (#5010) * Adding a name field to BufferSensorComponent * Adding a note to the CHANGELOG about var len obs * Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders * Adding a custom editor for BufferSensorComponent * adding inheritdoc * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update versions for release 14 hotfix. (#5040) * master -> main. (#5010) (#5044) * Update changelog. (#5045) * [MLA-1809] catch mismatched observation sizes (#5030) * Update changelog. (#5055) * Fix xml docs. (#5057) * pass sensor name through to ObservationSpec (#5036) * Remove unused allocation (#5068) * Automatically generate samples based on placement of mlagents-sample.json files in our examples. (#5077) * Update barracuda to 1.3.2-preview. (#5084) * POCA trainer (#5005) Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: Vincent-Pierre BERGES <[email protected]> * [docs] Documentation for POCA and cooperative behaviors (#5056) Co-authored-by: Ruo-Ping Dong <[email protected]> * [docs] Update changelog (#5095) * Add multiAgentGroup capabilities flag (#5096) * Add multiAgentGroup capabilities flag * Add proto * Fix compiler error * Add warning for multiagent group * Add comment * Fix spelling mistake * Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <[email protected]> * [cherry-pick] Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <[email protected]> * Update changelog for samples. (#5103) * Update changelog for samples. (#5103) (#5106) * Update versions on main (#5102) Increment versions after release 15 branch split * [release_15] Release 15 update versions (#5101) * Update versions * Fix for validate release links * Update release tag and docs * [docs] Add missing package docs (#5108) * Add missing package docs * Fix grammar * Update comment on Dispose * Fix typo in doc * [docs] Update readme and changelog (#5107) * Make analytics module an optional dependency. (#5109) Co-authored-by: Chris Elion <[email protected]> * Redo dotnet format (#5119) * [docs] Add links to example envs for Buffer Sensor and MultiAgentGroup (#5116) * Update changelog with links to example envs * Cooperative push block link update * [cherry-pick] Integrate Group Manager to soccer/retrain with POCA (#5115) (#5121) * Integrate Group Manager to soccer/retrain with POCA (#5115) * Add Soccer env to changelog Co-authored-by: andrewcoh <[email protected]> * [cherry-pick][docs] Add Dungeon Escape Environment (#5133) * Add DungeonEscape POCA Environment (#5128) * Add DungeonEscape assets from working branch * Add Dungeon Escape docs * Create dungeon_escape.png * Add to docs Co-authored-by: Hunter-Unity <[email protected]> * update pre-commit instructions for CONTRIBUTING.md (#5130) * Remove duplicated changelog * Fix another duplicated line in the changelog... * Removing Obsolete methods from the package (#5024) * Removing Obsolete methods from the package * Missing depecration and modified changelog * Readding the obsolete BrainParameter methods, will need a larger discussion on these * Removing Action Masker, readding the warining when using a non-implemented Heuristic, Removing NumAction from Brain Parameters * removing documentation and some calls to deprecated methods in the extensions package * Editing the Changelog to put the unreleased on top * non-IEnumerable interface for action masking (#5060) * V2 staging new model version (#5080) * Make modelCheck have flavors of error messages * ONNX exporter v3 * Using a better CheckType and a switch statement * Removing unused message * More tests * Use an enum for valid versions and use GetVersion on model directly * Maybe the model export version a static constant in Python * Use static constructor for FailedCheck * Use static constructor for FailedCheck * Modifying the docstrings * renaming LegacyDiscreteActionOutputApplier * removing testing code * better warning message * Nest the CheckTypeEnum into the FailedCheck class * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs * Adding a line explaining that legacy tensor checks are for versions 1.X only * Modifying the changelog * Exporting all the branches size instead of omly the sum (#5092) * addressing comments * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs Co-authored-by: Chris Elion <[email protected]> * readding tests * Adding a comment around the new DiscreteOutputSize method * Clearer warning : Model contains unexpected input > Model requires unknown input * Fixing a bug in the case where the discrete action tensor does not exist Co-authored-by: Chris Elion <[email protected]> * Make IActuator implement IHeuristicProvider. (#5110) * Adding more tests for v2 models (#5138) * Adding more tests for v2 models * formatting * fixing formatting * [MLA-1634] Add ObservationSpec and update ISensor interfaces (#5127) * Add custom package settings (#5027) * cleanup: yamato triggers, changelog, docstrings * clean up merge * fix compile error * update c# version to 2.0.0-exp.1 (#5160) Co-authored-by: Chris Elion <[email protected]> * Move physics/sensor tests that fail in edit mode depending on what's in the scene to playmode tests. (#5162) * update package version * remove defines that are now always true * changelog * Remove all old analytics defines. (#5168) * [MLA-1634] Compression spec (#5164) * Low hanging fruit tests for coverage. (#5170) * Update Policies to conditionally compile their analytics method. (#5171) * Turns physics modules into optional dependencies. (#5112) * [MLA-1634] Remove SensorComponent.GetObservationShape() (#5172) * PR Feedback. (#5173) * change default barracuda behavior (#5175) * [MLA-1824] make SensorComponent return ISensor[] (#5181) * Make SensorComponent return an array * split match3 sensors, partial retrain * docstrings, migration, changelog, cleanup * Adding the goal conditioning sensors with the new observation specs (#5159) * Fixing networks.py for the merge * fix compile error * Adding the goal conditioning sensors with the new observation specs * addressing feedback * I forgot to change the m_observationType * Renaming Goal to GoalSignal (#5190) * Renaming GOAL to GOAL_SIGNAL * VectorSensorComponent to use new API * Adding docstrings * verbose pytest on github action Co-authored-by: Chris Elion <[email protected]> * [MLA-1879] culture-invariant sorting for sensors and actuators (#5194) * Goal conditioning grid world : Example of goal conditioning (#5193) * Aded the Goal conditioned GridWorld to replace regular gridworld * adding missing files * Code improvements * Documentation change on gridworld * resolving conflicts * new model * Addressing comments * comments and renames * Update docs/Learning-Environment-Examples.md Co-authored-by: Ervin T. <[email protected]> * adding reference to gridworld in docs about goal signal Co-authored-by: Chris Elion <[email protected]> Co-authored-by: Ervin T. <[email protected]> * [MLA-1584] Match3 variable board size (#5189) * [bug-fix] When agent isn't training, don't clear update buffer (#5205) * Don't clear update buffer, but don't append to it either * Update changelog * Address comments * Make experience replay buffer saving more verbose * Changelog for goal conditioning (#5195) * Changelog for goal conditioning * adding a line about the changes to the gridworld example * Addressing comments * Change capitalization * [bug fix] Fix warning using demo recorder (#5216) * Making VectorActionSize and VectorActionSpaceType internal (#5214) Made sure the editor does not complain; * Turning some logger.info into logger.debug and remove some logging overhead when not using debug (#5211) * turning some logger.info into logger.debug and remove some logging overhead when not using debug * Addressing comments * Adding to changelog * rearrange match3 docs (#5215) * rearrange match3 docs * changelog and migration from previous PR * [bug-fix] Fix POCA LSTM, pad sequences in the back (#5206) * Pad buffer at the end * Fix padding in optimizer value estimate * Fix additional bugs and POCA * Fix groupmate obs, add tests * Update changelog * Improve tests * Address comments * Fix poca test * Fix buffer test * Increase entropy for Hallway * Add EOF newline * Fix Behavior Name * Address comments * [MLA-1880] Raycast sensor interface improvements (#5222) * WIP * remove debug info struct * cleanup + add to test * changelog * fix unit tests * PR feedback * Load individual elements if state dict load fails (#5213) Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: Ervin T. <[email protected]> * [MLA-1159] Add virtual methods to DecisionRequester (#5223) * [Bug Fix] Serialization of vectorActionSpace (#5220) * [Bug Fix] Serialization of vectorActionSpace * adding a test * Clear ActionBuffers before Heuristic calls (#5227) * Add ObservationType to analytics ObservationSpec (#5218) * [MLA-1909] Match3 and Camera/RenderTexture sensor GC improvements (#5233) * avoid empty set iteration, avoid Debug.AssertFormat (#5246) * avoid empty set iteration, avoid Debug.AssertFormat * changelog * Update to barracuda 1.3.3 and changes to the model inputs and outputs for LSTM (#5236) * Initial commit * making it work with hallway. Added a new model version * addressing feedback * Adding a comment * formatting * modifying the changelog * Adding descriptions on the model version descriptions * Update com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs Co-authored-by: Chris Elion <[email protected]> * addressing comments Co-authored-by: Chris Elion <[email protected]> * [Release 16] Release 16 Merge Back to Main (#5255) Update versions and documentation for Release 16. Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Chris Elion <[email protected]> * fix link, run link check in quiet mode (#5257) * [MLA-1912] Move Match3 to com.unity.ml-agents (#5259) * Move GridSensor into main package (#5256) * move OneHotGridSensor into main package * changelog and migration guide * remove old doc * check if physics module presents * Add advanced installation guide for com.unity.ml-agents (#5260) * Fix errors when creating new RayCast sensor (#5261) * check if tag and angle is null * add test * changelog * fix all PVS and doc generation warnings (#5262) * Fix --results-dir (#5269) * Fix GridSensorComponent bug (#5270) * LSTM models from 1.x will be incompatible with MLA 2.x (#5254) * LSTM models from 1.x will be incompatible with MLA 2.x * Adding a test and a new v2 model * Make the Model Runner raise an error if using 1.0 model with LSTM * adding a new model for hallway trained with 2.0 * reword error messages * Only raise if error, not if warning * Addressing comments: The legacy Barrauda memory generator and applier were removed. All code that checked for (memories + v1.X) have been removed since these will no longer be supported * Modifying the changelog and the migrating guide with this change * Fixing the merge issues Co-authored-by: Chris Elion <[email protected]> * [MLA-1939] Match3 Custom Editor (#5263) * Rider suggested cleanup, part 1 (#5265) * Fix the attention module embedding size (#5272) * Fix the attention module embedding size * editing the changelog * [bug-fix] Fix null ref exception in the demo recorder (#5274) * Fix null ref exception in the demo recorder * Update changelog * Clearer logic * Modifying the changelog for release 17 (#5277) * Modifying the changelog for release 17 * Change version to 2.0.0-exp.1 * Removing the [Unreleased] section * check for missing AbstractBoard, display warning (#5276) * Updating the barracuda 1.4.0 (#5291) * [debug] Require all behavior names to have a matching YAML entry (#5210) * Add strict check to settings.py * Remove warning from trainer factory, add test * Add changelog * Fix test * Update changelog * Remove strict CLI options * Remove strict option, rename, make strict default * Remove newline * Update comments * Set default dict to actually default to a default dict * Fix tests * Fix tests again * Default trainer dict to requiring all fields * Fix settings typing * Use logger * Add default_settings to error * Version bump main (#5279) * Version bump for the main branch following branching of release_17 * Messed up the extensions package version * Bring back root reference in grid sensor (#5300) * [MLA-1952] Add optional seed for gym action spaces (#5303) * add optional seed for action spaces * add optional seed for action spaces * changelog * undo packages-lock.json change * Release 17 version bumps and docs version bumps (#5280) * Migration guide - list removed APIs with replacements (#5307) * list removed APIs with replacements * fixes * [docs] Update changelog (#5308) * [MLA-1952] Add optional seed for gym action spaces (#5303) (#5315) * Fix stacked grid sensor (#5335) * Update to Unity Package licenses (#5340) * Update to Unity Package licenses Per Dan * Updating the main repo license dates. * Adding a fully connected visual encoder for super small visual input + tests (#5351) * initial commit for a fully connected visual encoder * adding a test * addressing comments * Fixing error with minimal size of fully connected network * adding documentation and changelog * Make OverlapChecker an interface (#5324) * Add stacking option to VectorSensorComponent (#5376) * [WIP] [Fix] Fixing collect observation called on done (#5375) * [WIP] [Fix] Fixing collect observation called on done * Update com.unity.ml-agents/Runtime/Agent.cs *⚠️ Modifying the test of stacking sensor when the agent is done * modifying the documentation for BufferSensor to specify to call AddObservation in the CollectObservations method * Fix NullReferenceException for Behavior Parameters without Agent (#5382) * check agent existence. add warning box * changelog * [WIP] 2.0 verified to main (#5385) * Warn if no joints on Root Body (#5387) * Modifying the changelog for the continuous action SAC target entropy fix (#5394) * Editing the GridSensor documentation for 2D use case (#5396) * Editing the GridSensor documentation for 2D use case * changing chagelog * [MLA-2017] Move colab notebooks to github (#5399) * Fix the reporting of histogram stats and adding a test (#5410) * Fix the reporting of histogram stats and adding a test * Appending to the Changelog * Update changelog before Release 18 branch (#5412) * Update package versions for main branch (#5413) * [Release 18] Update versions and links (#5414) * add changelog entries for PRs that were missing them before. (#5416) * fix cref error found by doc validation (#5421) * fix release versions and changelog section * [bug-fix] Fix when group terminal steps are deleted, robust test (#5441) * Fix when terminal steps are deleted, robust test * Update changelog * Fix test comment * Fix torch device option in argparser (#5467) * [FIX] prioritize cli options over yaml (#5495) * prioritize cli over yaml in checkpointSettings * prioritize resume if both set in one place * fixed test_commandline_args * addressing comments:renaming, commenting * more tests and referenced in change_log * Only update the native library and the c# calls that need to be made to clean up channels. (#5283) * Fix Mac backcompat test (#5519) * Initialize-from custom checkpoints (#5525) * init from any checkpoint including older ones * moving init_path logic ahead to learn.py * fixing pytest to take the full path * doc & changelog * Update gym version to 0.20.0 (#5540) * Fixed failing test due to gym 0.20.0 release. Updated gym-unity package to fix gym version to 0.20.0. * Updated CHANGELOG. * Fix VAIL (#5546) * Custom trainer editor analytics (#5511) * Custom trainer editor analytics * inherit the default_training_analytics_side_channel * Feature/mla 2205 separate schedule lr beta epsilon (#5538) * Added initial (untested) support for separating schedule for beta, LR, and epsilon. * Added support for beta and epsilon schedule in test config conversion * Updated docs. Set epsilon to be linear by default. * Updated configs to add beta and epsilon schedule where appropriate. * Addressed CR change requests. * Minor update. * Updated docs. * Added changes to CHANGELOG. * Updated CHANGELOG. * Fixed typo in Training-Configuration-File.md * Reverted config changes to remove beta and epsilon schedules. * mypy error fixed from pre-commit. * Fixed a bug that was causing a settings test to fail. * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> * Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512) * Hash128 is not a cryptographic hash, replace with HMAC-SHA256. * Extend TrainingAnalytics side channel to expose configuration details * Change member function scopes and hash demo_paths * Extract tbiEvent hashing method and add test coverage * Restart crashed Unity environments (#5553) ### Change(s) Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash. When a worker receives an ENV_EXITED signal it will now: 1. Record all failures coming through the step queue and drop all other messages. 2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted. 3. Restart all failed workers (up to a configurable limit) This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values.⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times). ```yaml env_settings: # Can restart 10 times over the lifetime of the experiment. max_lifetime_restarts: 10 # Rate limit of 1 failure per 60s restarts_rate_limit_n: 1 restarts_rate_limit_period_s: 60 ``` They can of course be passed via CLI arguments as well ```bash --max-lifetime-restarts --restarts-rate-limit-n --restarts-rate-limit-period-s ``` ### Disabling this feature * Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1` * Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1` * Develop add training area replicator (#5568) * Added training area replicator to com.unity.ml-agents package. * Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas. * Changed training area replicator to size grid automatically from number of areas. * Added tests for the training area replicator. * Added setup for tests for the training area replicator. * Added comments and updated create tutorial for training area replicator. * Updated CHANGELOG. * Fixed some failing tests. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Henry Peteet <[email protected]> * Update docs/Learning-Environment-Create-New.md Co-authored-by: Henry Peteet <[email protected]> * Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs Co-authored-by: Henry Peteet <[email protected]> * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: Henry Peteet <[email protected]> * Fixing the analytics side-channel for curriculum learning. (#5586) * Fixing the analytics side-channel for curriculum learning. * Made a more robust test. * Update the changelog. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Maryam Honari <[email protected]> * Method to return stacked observations (#5547) * Method to return stacked observations * Added testing and returning staked observation flat. * Update the comment lines. * Remove brainstorm commits. * Upgrade Barracuda to 2.3.1-preview (#5591) * Upgrade to 2.3.1-preview * Fix for critic normalization bug (#5595) * Added normalization to critic during training for poca, ppo, and sac. * Fixed critic normalization bug. Added new tests to cover. * Updated CHANGELOG. * Fixed typo in test_trainers.py * Deterministic actions python training (#5619) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <[email protected]> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <[email protected]> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: cmard <[email protected]> * Revert "Deterministic actions python training (#5619)" (#5622) This reverts commit 9ea962027d4c581de7c45a07d669f96453236c5c. * Deterministic actions python training (#5626) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <[email protected]> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <[email protected]> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <[email protected]> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <[email protected]> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md * Fix the deterministic showing up all the tiime (#5621) Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: cmard <[email protected]> * Changelog cleanup. (#5627) * Release cleanup for changelog * Update CHANGELOG.md Co-authored-by: Maryam Honari <[email protected]> * Fix analytics error when compiling XBOX and PS5 (#5628) * Add ENABLE_CLOUD_SERVICES_ANALYTICS Guards To use analytics safely you must wrap usage with `#if ENABLE_CLOUD_SERVICES_ANALYTICS` See https://docs.unity3d.com/ScriptReference/Analytics.Analytics.html * Update DQN Colab example to support export for Barracuda inference (#5643) * Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference. -Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case. -Add GridWorldColab.onnx file, trained and generated with the colab. * Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene. -permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW. -update input_shape indexes and remove equivalent permute from the VisualQNetwork -fix reference to global qnet inside generate_trajectories (use the passed in version) -create wrapper network to add required constants and dummy input masks required by the runtime inference (Barracuda) -export the wrapped network to ONNX format compatible with Barracuda * Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example. * Minor edit in Colab docs. * Delete unnecessary .lighting file. * Update changelog. * Update colab/Colab_UnityEnvironment_2_Train.ipynb Co-authored-by: Maryam Honari <[email protected]> * Fix error in colab metadata name. * Add suggested comments and fix incorrect parameter ordering. * Update docs for Elo curriculum completion measure. Co-authored-by: Maryam Honari <[email protected]> * Changelog prepped for release (#5629) * Changelog prepped for release * Added missing PRs to the changelog. * Added missing CHANGELOG item. * enable 3D Ball sample import button in package manager for unity sdk (#5665) * enable 3D Ball sample import button in package manager for unity sdk * add Samples folder * add meta file * fix git ignore config * Updated versions for main. Also updated validate versions script to r… (#5668) * Updated versions for main. Also updated validate versions script to remove exp.1 from package version. * Reverted validate versions. * Updated versions. * Bumped package version for patch. * Removed unreleased section. (#5667) * Updated release versions and release links. (#5669) * Updates for package validation errors. (#5671) * Release 19 branch lts deprecation (#5673) * Updated Project and DevProject to 2020.3 LTS. * Updated supported LTS version to 2020.3. * Updated changelog. * Bumped up CI Unity version from 2020.2 to 2020.3. * Update CHANGELOG.md * Removing VS Studio dependency from manifest.json. * Update packages-lock.json * Update CHANGELOG.md * Bumped package version for patch. * Fixed build-docker and updated CHANGELOG to add unreleased section. (#5682) * Fixed build-docker and updated CHANGELOG to add unreleased section. * Updated backward compat mac training yamato test. (#5685) * Develop deprecate python 3.6 (#5) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * Updated failing yamato jobs. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Develop python api ga (#6) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <[email protected]> * Updated failing yamato jobs. * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <[email protected]> * Updated CHANGELOG. * Updated Migration guide. * Doc updates based on CR. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Fixed yamato import error. Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: jmercado1985 <[email protected]> Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: mahon94 <[email protected]> Co-authored-by: Andrew Cohen <[email protected]> * Refactoring for extraction to separate repo. * removing unused github workflows. * Updates for pre-commit and github actions. * Fixed bug with validate_meta_files.py missing a file to exclude. * Updated precommit gh action. * Refactored yamato pack. * Added yamato test yaml. * Updated to CHANGELOG and VectorSensor from mono repo before swap. * Pin dotnet version for GHA formatting. * Refactor for MLA pro support. (#1) * Refactored for MLA pro support. * Fixed bug in yamato test. * Updated the CHANGELOG. * Fixed trigger logic in yamato test. * Fixed failing tests. (#2) * Fixed failing tests. * Adding meta files. * Fix execution order (#4) * Fixed execution order. * Fixed bug with CRT build. * precommit formatting update. * Updated agent Awake docstring. * Set execution order to be negative to make sure scripts run before default time. (#5) * Added #if to exclude communicator registration in the Agent awake script (#6) * Add visibility to pro package (#7) * add coverage publish promotion yamato jobs (#3) * bump barracuda to 3.0.0 (#9) upgrade to barracuda 3.0.0 * Updated README. (#8) * add validation test dep. to publish job (#12) * add validation test dep. to publish job * fix package path * Update automated trigger rules to support oneflow develop branch (#11) * Update automated trigger rules to support oneflow develop branch * Fix release branch name for github actions * Release 2.3.0 exp.1 (#13) * upgrade package to 2.3.0-exp.1 * rebase on develop branch * update changelog * remove unreleased section (#16) * Updated experimental prerelease version and changelog. (#18) * Added submodule update gha. (#19) * fix bugs from sonar-scanner (#21) * add ml-agents-pro as a upstream for submodule update (#22) * Added support for standalone use of Academy with refactored communicator factory. (#24) * clean commandline Arguments (#25) * clean commandline Arguments * fix precommit * update unrelease section in changelog (#27) * MLA-2537 Reorder RayPerceptionSensor output to reflect its spatial structure. (#26) * Changes to support in order rays. GUI for switching to this. Updated tests. * Updated README. (#29) * Fixed samples. (#30) * Added camera sensor component checkbox for runtime enablement. update… (#31) * Added camera sensor component checkbox for runtime enablement. updated camera sensor to only be active when enabled for runtime. * Updated yamato job. * Updated CHANGELOG. * Fixed issue with Academy not building for webgl builds. (#32) * Dev bump min ver (#33) * Bumped min version of Unity to 2021.3. * Updated min version of python in pre-commit. Updated changelog. * Moving files into packages/com.unity.ml-agents directoy. * Remove com.unity.ml-agents submodule. * Added back in ml-agents package. * Removed packages temp folder. * Updated yamato protobuf gen. * Updated yamato to remove submodule update. * Updated linux tool chain. * Updated Project package dependencies. * Updated flake8 repo link in precommit. Co-authored-by: Vincent-Pierre BERGES <[email protected]> Co-authored-by: Arthur Juliani <[email protected]> Co-authored-by: Chris Elion <[email protected]> Co-authored-by: andrewcoh <[email protected]> Co-authored-by: Chris Goy <[email protected]> Co-authored-by: Marwan Mattar <[email protected]> Co-authored-by: Ruo-Ping Dong <[email protected]> Co-authored-by: Ervin Teng <[email protected]> Co-authored-by: Hunter-Unity <[email protected]> Co-authored-by: Maryam Honari <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Miguel Alonso Jr <[email protected]> Co-authored-by: Jason Bowman <[email protected]> Co-authored-by: Henry Peteet <[email protected]> Co-authored-by: cmard <[email protected]> Co-authored-by: Chingiz Mardanov <[email protected]> Co-authored-by: Jason Rupert <[email protected]> Co-authored-by: Xun Cao <[email protected]> Co-authored-by: jmercado1985 <[email protected]> Co-authored-by: mahon94 <[email protected]> Co-authored-by: Andrew Cohen <[email protected]> Co-authored-by: Xun Cao <[email protected]> Co-authored-by: Jason Rupert <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change(s)
Updates the GridWorld DQN Colab notebook to take NCHW formatted image as input, as expected by Barracuda and created a wrapper network that provides other essential tensors and ONNX export parameters to ensure compatibility for Barracuda inference.
Added new scene to the ML-Agent examples Project for GridWorld that removes goal conditioning and leaves all masks as 1 to be compatible with the DQN Colab.
Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)
https://jira.unity3d.com/browse/MLA-2315
Types of change(s)
Updated feature.
Checklist
Other comments