Updating repository (#3)

* [MLA-1768] retrain Match3 scene (Unity-Technologies#4943)

* improved settings and move to default_settings

* update models

* Release 13 versions. (Unity-Technologies#4946)

- updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation.

* Update readme for release_13. (Unity-Technologies#4951)

* Update docs to pass doc validation. (Unity-Technologies#4953)

* update defines, compile out Initialize body on non-desktop (Unity-Technologies#4957)

* Masking Discrete Actions typos (Unity-Technologies#4961) (Unity-Technologies#4964)

Co-authored-by: Philipp Siedler <[email protected]>

* Adding references to the Extensions package to help promote it. (Unity-Technologies#4967) (Unity-Technologies#4968)

Co-authored-by: Marwan Mattar <[email protected]>
Co-authored-by: Chris Elion <[email protected]>

* Fix release link validations. (Unity-Technologies#4970)

* Adding the Variable length observation to the readme and to the overview of ML-Agents

* -

* forgot a dot

* InputActuatorComponent to allow the generation of an action space from an InputActionAsset (Unity-Technologies#4881) (Unity-Technologies#4974)

* handle no plugins found (Unity-Technologies#4969) (Unity-Technologies#4973)

* Tick extension version. (Unity-Technologies#4975)

* adding a comic and readding removed feaures docs

* Update 2018 project version to fix burst errors. (Unity-Technologies#4977) (Unity-Technologies#4978)

* Add an example project for the InputSystemActuator. (Unity-Technologies#4976) (Unity-Technologies#4980)

* Update barracuda, swtich Agents in Sorter use Burst. (Unity-Technologies#4979) (Unity-Technologies#4981)

* Set ignore done=False in GAIL (Unity-Technologies#4971)

* MultiAgentGroup Interface (Unity-Technologies#4923)

* add SimpleMultiAgentGroup

* add group reward field to agent and proto

* Fix InputActuatorComponent tests asmdef. (Unity-Technologies#4994) (Unity-Technologies#4995)

* Fix asmdef? (Unity-Technologies#4994) (Unity-Technologies#4996)

* Make TrainingAnalyticsSideChannel internal (Unity-Technologies#4999)

* [MLA-1783] built-in actuator type (Unity-Technologies#4950)

* Add component menues for some sensors and actuators. (Unity-Technologies#5001)

* Add component menues for some sensors and actuators. (Unity-Technologies#5001) (Unity-Technologies#5002)

* Fixing the number of layers in the config of PyramidsRND

* Merge master -> release_13_branch-to-master

* Fix RpcCommunicator merge.

* Move the Critic into the Optimizer (Unity-Technologies#4939)

Co-authored-by: Ervin Teng <[email protected]>

* master -> main. (Unity-Technologies#5010)

* Adding links to CV/Robotics/GameSim (Unity-Technologies#5012)

* Make GridSensor a non allocating object after initialization. (Unity-Technologies#5014)

Co-authored-by: Chris Elion <[email protected]>

* Modified the model_serialization to have correct inputs and outputs

* switching from CamelCase to snake_case

* Fix gpu pytests (Unity-Technologies#5019)

* Move tensors to cpu before converting it to numpy

* Adding a name field to BufferSensorComponent

* Adding a note to the CHANGELOG about var len obs

* Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders

* Adding a custom editor for BufferSensorComponent

* Changing Sorter to use the new Editor and serialization

* adding inheritdoc

* Update cattrs dependencies to support python3.9 (Unity-Technologies#4821)

* Python Dataflow for Group Manager (Unity-Technologies#4926)

* Make buffer type-agnostic

* Edit types of Apped method

* Change comment

* Collaborative walljump

* Make collab env harder

* Add group ID

* Add collab obs to trajectory

* Fix bug; add critic_obs to buffer

* Set group ids for some envs

* Pretty broken

* Less broken PPO

* Update SAC, fix PPO batching

* Fix SAC interrupted condition and typing

* Fix SAC interrupted again

* Remove erroneous file

* Fix multiple obs

* Update curiosity reward provider

* Update GAIL and BC

* Multi-input network

* Some minor tweaks but still broken

* Get next critic observations into value estimate

* Temporarily disable exporting

* Use Vince's ONNX export code

* Cleanup

* Add walljump collab YAML

* Lower max height

* Update prefab

* Update prefab

* Collaborative Hallway

* Set num teammates to 2

* Add config and group ids to HallwayCollab

* Fix bug with hallway collab

* Edits to HallwayCollab

* Update onnx file meta

* Make the env easier

* Remove prints

* Make Collab env harder

* Fix group ID

* Add cc to ghost trainer

* Add comment to ghost trainer

* Revert "Add comment to ghost trainer"

This reverts commit 292b6ce.

* Actually add comment to ghosttrainer

* Scale size of CC network

* Scale value network based on num agents

* Add 3rd symbol to hallway collab

* Make comms one-hot

* Fix S tag

* Additional changes

* Some more fixes

* Self-attention Centralized Critic

* separate entity encoder and RSA

* clean up args in mha

* more cleanups

* fixed tests

* entity embeddings work with no max
Integrate into CC

* remove group id

* very rough sketch for TeamManager interface

* One layer for entity embed

* Use 4 heads

* add defaults to linear encoder, initialize ent encoders

* add team manager id to proto

* team manager for hallway

* add manager to hallway

* send and process team manager id

* remove print

* small cleanup

* default behavior for baseTeamManager

* add back statsrecorder

* update

* Team manager prototype  (Unity-Technologies#4850)

* remove group id

* very rough sketch for TeamManager interface

* add team manager id to proto

* team manager for hallway

* add manager to hallway

* send and process team manager id

* remove print

* small cleanup

Co-authored-by: Chris Elion <[email protected]>

* Remove statsrecorder

* Fix AgentProcessor for TeamManager
Should work for variable decision frequencies (untested)

* team manager

* New buffer layout, TeamObsUtil, pad dead agents

* Use NaNs to get masks for attention

* Add team reward to buffer

* Try subtract marginalized value

* Add Q function with attention

* Some more progress - still broken

* use singular entity embedding (Unity-Technologies#4873)

* I think it's running

* Actions added but untested

* Fix issue with team_actions

* Add next action and next team obs

* separate forward into q_net and baseline

* might be right

* forcing this to work

* buffer error

* COMAA runs

* add lambda return and target network

* no target net

* remove normalize advantages

* add target network back

* value estimator

* update coma config

* add target net

* no target, increase lambda

* remove prints

* cloud config

* use v return

* use target net

* adding zombie to coma2 brnch

* add callbacks

* cloud run with coma2 of held out zombie test env

* target of baseline is returns_v

* remove target update

* Add team dones

* ntegrate teammate dones

* add value clipping

* try again on cloud

* clipping values and updated zombie

* update configs

* remove value head clipping

* update zombie config

* Add trust region to COMA updates

* Remove Q-net for perf

* Weight decay, regularizaton loss

* Use same network

* add base team manager

* Remove reg loss, still stable

* Black format

* add team reward field to agent and proto

* set team reward

* add maxstep to teammanager and hook to academy

* check agent by agent.enabled

* remove manager from academy when dispose

* move manager

* put team reward in decision steps

* use 0 as default manager id

* fix setTeamReward

Co-authored-by: Vincent-Pierre BERGES <[email protected]>

* change method name to GetRegisteredAgents

* address comments

* Revert C# env changes

* Remove a bunch of stuff from envs

* Remove a bunch of extra files

* Remove changes from base-teammanager

* Remove remaining files

* Remove some unneeded changes

* Make buffer typing neater

* AgentProcessor fixes

* Back out trainer changes

* use delegate to avoid agent-manager cyclic reference

* put team reward in decision steps

* fix unregister agents

* add teamreward to decision step

* typo

* unregister on disabled

* remove OnTeamEpisodeBegin

* change name TeamManager to MultiAgentGroup

* more team -> group

* fix tests

* fix tests

* Use attention tests from master

* Revert "Use attention tests from master"

This reverts commit 78e052b.

* Use attention from master

* Renaming fest

* Use NamedTuples instead of attrs classes

* Bug fixes

* remove GroupMaxStep

* add some doc

* Fix mock brain

* np float32 fixes

* more renaming

* Test for team obs in agentprocessor

* Test for group and add team reward

* doc improve

Co-authored-by: Ervin T. <[email protected]>

* store registered agents in set

* remove unused step counts

* Global group ids

* Fix Trajectory test

* Remove duplicated files

* Add team methods to AgentAction

* Buffer fixes

(cherry picked from commit 2c03d2b)

* Add test for GroupObs

* Change AgentAction back to 0 pad and add tests

* Addressed some comments

* Address some comments

* Add more comments

* Rename internal function

* Move padding method to AgentBufferField

* Fix slicing typing and string printing in AgentBufferField

* Fix to-flat and add tests

* Rename GroupmateStatus to AgentStatus

* Update comments

* Added GroupId, GlobalGroupId, GlobalAgentId types

* Update comment

* Make some agent processor properties internal

* Rename add_group_status

* Rename store_group_status, fix test

* Rename clear_group_obs

Co-authored-by: Andrew Cohen <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>
Co-authored-by: Chris Elion <[email protected]>
Co-authored-by: andrewcoh <[email protected]>
Co-authored-by: Vincent-Pierre BERGES <[email protected]>

* Removing some scenes (Unity-Technologies#4997)

* Removing some scenes, All the Static and all the non variable speed environments. Also removed Bouncer, PushBlock, WallJump and reacher. Removed a bunch of visual environements as well. Removed 3DBallHard and FoodCollector (kept Visual and Grid FoodCollector)

* readding 3DBallHard

* readding pushblock and walljump

* Removing tennis

* removing mentions of removed environments

* removing unused images

* Renaming Crawler demos

* renaming some demo files

* removing and modifying some config files

* new examples image?

* removing Bouncer from build list

* replacing the Bouncer environment with Match3 for llapi tests

* Typo in yamato test

* Fix issue with queuing input events that stomp on others. (Unity-Technologies#5034)

Co-authored-by: Chris Elion <[email protected]>
Co-authored-by: Chris Goy <[email protected]>
Co-authored-by: Marwan Mattar <[email protected]>
Co-authored-by: vincentpierre <[email protected]>
Co-authored-by: andrewcoh <[email protected]>
Co-authored-by: Ruo-Ping Dong <[email protected]>
Co-authored-by: Ervin Teng <[email protected]>
Co-authored-by: Andrew Cohen <[email protected]>

.github/PULL_REQUEST_TEMPLATE.md

@@ -17,8 +17,8 @@ Describe the changes made in this PR.
 
 ### Checklist
 - [ ] Added tests that prove my fix is effective or that my feature works
-- [ ] Updated the [changelog](https://github.com/Unity-Technologies/ml-agents/blob/master/com.unity.ml-agents/CHANGELOG.md) (if applicable)
-- [ ] Updated the [documentation](https://github.com/Unity-Technologies/ml-agents/tree/master/docs) (if applicable)
-- [ ] Updated the [migration guide](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md) (if applicable)
+- [ ] Updated the [changelog](https://github.com/Unity-Technologies/ml-agents/blob/main/com.unity.ml-agents/CHANGELOG.md) (if applicable)
+- [ ] Updated the [documentation](https://github.com/Unity-Technologies/ml-agents/tree/main/docs) (if applicable)
+- [ ] Updated the [migration guide](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Migrating.md) (if applicable)
 
 ### Other comments

.github/workflows/pre-commit.yml

@@ -3,7 +3,7 @@ name: pre-commit
 on:
   pull_request:
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pre-commit:

.github/workflows/publish_pypi.yaml

@@ -16,7 +16,7 @@ jobs:
             package-path: [ml-agents, ml-agents-envs, gym-unity]
 
     steps:
-    - uses: actions/checkout@master
+    - uses: actions/checkout@main
     - name: Set up Python 3.7
       uses: actions/setup-python@v1
       with:

.github/workflows/pytest.yml

@@ -10,7 +10,7 @@ on:
     - 'test_requirements.txt'
     - '.github/workflows/pytest.yml'
   push:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:

.yamato/com.unity.ml-agents-performance.yml

@@ -20,7 +20,7 @@ Run_Mac_Perfomance_Tests{{ editor.version }}:
   triggers:
     cancel_old_ci: true
     recurring:
-      - branch: master
+      - branch: main
         frequency: daily
   artifacts:
     logs:

.yamato/com.unity.ml-agents-test.yml

@@ -60,7 +60,7 @@ all_package_tests:
   triggers:
     cancel_old_ci: true
     recurring:
-      - branch: master
+      - branch: main
         frequency: daily
 
 {% for package in packages %}
@@ -101,7 +101,7 @@ test_{{ package.name }}_{{ platform.name }}_{{ editor.version }}:
     cancel_old_ci: true
     {% if platform.name == "linux" %}
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

.yamato/compressed-sensor-test.yml

@@ -23,7 +23,7 @@ test_compressed_obs_{{ editor.version }}:
     cancel_old_ci: true
     {% if editor.extra_test == "sensor" %}
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

.yamato/gym-interface-test.yml

@@ -23,7 +23,7 @@ test_gym_interface_{{ editor.version }}:
     cancel_old_ci: true
     {% if editor.extra_test == "gym" %}
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

.yamato/protobuf-generation-test.yml

@@ -28,7 +28,7 @@ test_linux_protobuf_generation:
   triggers:
     cancel_old_ci: true
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "protobuf-definitions/**" OR

.yamato/pytest-gpu.yml

@@ -16,7 +16,7 @@ pytest_gpu:
   triggers:
     cancel_old_ci: true
     recurring:
-      - branch: master
+      - branch: main
         frequency: daily
   artifacts:
     logs:

.yamato/python-ll-api-test.yml

@@ -18,14 +18,14 @@ test_linux_ll_api_{{ editor.version }}:
       python ml-agents/tests/yamato/scripts/run_llapi.py
       python ml-agents/tests/yamato/scripts/run_llapi.py --env=artifacts/testPlayer-Basic
       python ml-agents/tests/yamato/scripts/run_llapi.py --env=artifacts/testPlayer-WallJump
-      python ml-agents/tests/yamato/scripts/run_llapi.py --env=artifacts/testPlayer-Bouncer
+      python ml-agents/tests/yamato/scripts/run_llapi.py --env=artifacts/testPlayer-Match3
   dependencies:
     - .yamato/standalone-build-test.yml#test_linux_standalone_{{ editor.version }}
   triggers:
     cancel_old_ci: true
     {% if editor.extra_test == "llapi" %}
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

.yamato/standalone-build-test.yml

@@ -15,14 +15,14 @@ test_linux_standalone_{{ editor.version }}:
     - unity-downloader-cli -u {{ editor.version }} -c editor --wait --fast
     - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux
     - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/Examples/Basic/Scenes/Basic.unity
-    - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/Examples/Bouncer/Scenes/Bouncer.unity
+    - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/Examples/Match3/Scenes/Match3.unity
     - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/Examples/WallJump/Scenes/WallJump.unity
     - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/TestScenes/TestCompressedGrid/TestGridCompressed.unity
     - python3 -u -m ml-agents.tests.yamato.standalone_build_tests --build-target=linux --scene=Assets/ML-Agents/TestScenes/TestCompressedTexture/TestTextureCompressed.unity
   triggers:
     cancel_old_ci: true
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

.yamato/standalone-build-webgl-test.yml

@@ -18,7 +18,7 @@ test_webgl_standalone_{{ editor_version }}:
   triggers:
     cancel_old_ci: true
     recurring:
-      - branch: master
+      - branch: main
         frequency: weekly
   artifacts:
     logs:

.yamato/training-backcompat-tests.yml

@@ -29,7 +29,7 @@ test_mac_backcompat_2020.1:
   triggers:
     cancel_old_ci: true
     recurring:
-      - branch: master
+      - branch: main
         frequency: daily
   artifacts:
     logs:

.yamato/training-int-tests.yml

@@ -20,7 +20,7 @@ test_linux_training_int_{{ editor.version }}:
   triggers:
     cancel_old_ci: true
     expression: |
-      (pull_request.target eq "master" OR
+      (pull_request.target eq "main" OR
       pull_request.target match "release.+") AND
       NOT pull_request.draft AND
       (pull_request.changes.any match "com.unity.ml-agents/**" OR

DevProject/Packages/packages-lock.json

@@ -31,7 +31,7 @@
       "url": "https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-candidates"
     },
     "com.unity.barracuda": {
-      "version": "1.3.0-preview",
+      "version": "1.3.1-preview",
       "depth": 1,
       "source": "registry",
       "dependencies": {
@@ -108,7 +108,7 @@
       "depth": 0,
       "source": "local",
       "dependencies": {
-        "com.unity.barracuda": "1.3.0-preview",
+        "com.unity.barracuda": "1.3.1-preview",
         "com.unity.modules.imageconversion": "1.0.0",
         "com.unity.modules.jsonserialize": "1.0.0",
         "com.unity.modules.physics": "1.0.0",
@@ -121,7 +121,7 @@
       "depth": 0,
       "source": "local",
       "dependencies": {
-        "com.unity.ml-agents": "1.7.2-preview"
+        "com.unity.ml-agents": "1.8.0-preview"
       }
     },
     "com.unity.multiplayer-hlapi": {

ML-Agents-Input-Example/Packages/packages-lock.json

@@ -31,7 +31,7 @@
       "url": "https://packages.unity.com"
     },
     "com.unity.barracuda": {
-      "version": "1.3.0-preview",
+      "version": "1.3.1-preview",
       "depth": 1,
       "source": "registry",
       "dependencies": {
@@ -99,7 +99,7 @@
       "depth": 0,
       "source": "local",
       "dependencies": {
-        "com.unity.barracuda": "1.3.0-preview",
+        "com.unity.barracuda": "1.3.1-preview",
         "com.unity.modules.imageconversion": "1.0.0",
         "com.unity.modules.jsonserialize": "1.0.0",
         "com.unity.modules.physics": "1.0.0",