Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/test-multi-gpu.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,14 @@ jobs:
# PhysX + Isaac RTX renderer
- physics: physx
renderer: isaac-rtx
task: Isaac-Cartpole-Camera-Presets-Direct-v0
task: Isaac-Cartpole-Camera-Direct-v0
extra_args: ""
trainer: skrl

# PhysX + Newton Warp renderer (hybrid)
- physics: physx
renderer: newton-warp
task: Isaac-Cartpole-Camera-Presets-Direct-v0
task: Isaac-Cartpole-Camera-Direct-v0
extra_args: "env.tiled_camera.renderer_cfg=newton_renderer"
trainer: skrl

Expand All @@ -67,14 +67,14 @@ jobs:
# Newton + Newton Warp renderer
- physics: newton
renderer: newton-warp
task: Isaac-Cartpole-Camera-Presets-Direct-v0
task: Isaac-Cartpole-Camera-Direct-v0
extra_args: "+sim=newton env.tiled_camera.renderer_cfg=newton_renderer"
trainer: skrl

# Newton + Isaac RTX renderer (hybrid)
- physics: newton
renderer: isaac-rtx
task: Isaac-Cartpole-Camera-Presets-Direct-v0
task: Isaac-Cartpole-Camera-Direct-v0
extra_args: "+sim=newton"
trainer: skrl

Expand Down
2 changes: 1 addition & 1 deletion docs/source/features/reproducibility.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ To enable deterministic RTX settings from the app launcher, pass ``--determinist
.. code-block:: bash

./isaaclab.sh -p scripts/reinforcement_learning/rl_games/train.py \
--task Isaac-Cartpole-RGB-v0 --enable_cameras --headless --deterministic
--task Isaac-Cartpole-Camera-v0 --enable_cameras --headless --deterministic

For results on our determinacy testing for RL training, please check the GitHub Pull Request `#940`_.

Expand Down
18 changes: 2 additions & 16 deletions docs/source/how-to/optimize_stage_creation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,22 +82,8 @@ Limitations
1. Isaac-Ant-Direct-v0
2. Isaac-Ant-v0
3. Isaac-Cartpole-Direct-v0
4. Isaac-Cartpole-Showcase-Box-Box-Direct-v0
5. Isaac-Cartpole-Showcase-Box-Discrete-Direct-v0
6. Isaac-Cartpole-Showcase-Box-MultiDiscrete-Direct-v0
7. Isaac-Cartpole-Showcase-Dict-Box-Direct-v0
8. Isaac-Cartpole-Showcase-Dict-Discrete-Direct-v0
9. Isaac-Cartpole-Showcase-Dict-MultiDiscrete-Direct-v0
10. Isaac-Cartpole-Showcase-Discrete-Box-Direct-v0
11. Isaac-Cartpole-Showcase-Discrete-Discrete-Direct-v0
12. Isaac-Cartpole-Showcase-Discrete-MultiDiscrete-Direct-v0
13. Isaac-Cartpole-Showcase-MultiDiscrete-Box-Direct-v0
14. Isaac-Cartpole-Showcase-MultiDiscrete-Discrete-Direct-v0
15. Isaac-Cartpole-Showcase-MultiDiscrete-MultiDiscrete-Direct-v0
16. Isaac-Cartpole-Showcase-Tuple-Box-Direct-v0
17. Isaac-Cartpole-Showcase-Tuple-Discrete-Direct-v0
18. Isaac-Cartpole-Showcase-Tuple-MultiDiscrete-Direct-v0
19. Isaac-Cartpole-v0
4. Isaac-Cartpole-Showcase-Direct-v0
5. Isaac-Cartpole-v0
20. Isaac-Factory-GearMesh-Direct-v0
21. Isaac-Factory-NutThread-Direct-v0
22. Isaac-Factory-PegInsert-Direct-v0
Expand Down
8 changes: 4 additions & 4 deletions docs/source/overview/core-concepts/sensors/camera.rst
Original file line number Diff line number Diff line change
Expand Up @@ -149,13 +149,13 @@ The active preset is selected at launch via the ``presets=`` CLI argument:
.. code-block:: bash

# Use Newton Warp renderer
python train.py task=Isaac-Cartpole-RGB-Camera-Direct-v0 presets=newton_renderer
python train.py task=Isaac-Cartpole-Camera-Direct-v0 presets=newton_renderer

# Use OVRTX renderer
python train.py task=Isaac-Cartpole-RGB-Camera-Direct-v0 presets=ovrtx_renderer
python train.py task=Isaac-Cartpole-Camera-Direct-v0 presets=ovrtx_renderer

# Use default (Isaac RTX)
python train.py task=Isaac-Cartpole-RGB-Camera-Direct-v0
python train.py task=Isaac-Cartpole-Camera-Direct-v0


Accessing camera data
Expand All @@ -174,7 +174,7 @@ When using the RTX renderer, add ``--enable_cameras`` when launching:
.. code-block:: shell

python scripts/reinforcement_learning/rl_games/train.py \
--task=Isaac-Cartpole-RGB-Camera-Direct-v0 --headless --enable_cameras
--task=Isaac-Cartpole-Camera-Direct-v0 --headless --enable_cameras


Annotators (RTX only)
Expand Down
147 changes: 16 additions & 131 deletions docs/source/overview/environments.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,10 @@ Classic environments that are based on IsaacGymEnvs implementation of MuJoCo-sty
| | | | ``segmentation``, |
| | | | ``simple_shading_*`` |
+------------------+-----------------------------+-------------------------------------------------------------------------+------------------------------+
| |cartpole| | |cartpole-resnet-link| | Move the cart to keep the pole upwards in the classic cartpole control | ``newton_mjwarp``, ``physx`` |
| | | based off of features extracted from perceptive inputs with pre-trained | |
| | |cartpole-theia-link| | frozen vision encoders. Requires running with ``--enable_cameras``. | |
| |cartpole| | |cartpole-camera-link| | Move the cart to keep the pole upwards in the classic cartpole control | ``newton_mjwarp``, ``physx`` |
| | | from raw RGB/depth observations or features extracted by pre-trained | ``rgb``, ``depth``, |
| | | frozen vision encoders. Select pipeline via ``presets=``. Requires | ``resnet18``, |
| | | running with ``--enable_cameras``. | ``theia_tiny`` |
+------------------+-----------------------------+-------------------------------------------------------------------------+------------------------------+

.. |humanoid| image:: ../_static/tasks/classic/humanoid.jpg
Expand All @@ -81,9 +82,8 @@ Classic environments that are based on IsaacGymEnvs implementation of MuJoCo-sty
.. |humanoid-link| replace:: `Isaac-Humanoid-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/humanoid/humanoid_env_cfg.py>`__
.. |ant-link| replace:: `Isaac-Ant-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/ant/ant_env_cfg.py>`__
.. |cartpole-link| replace:: `Isaac-Cartpole-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/cartpole/cartpole_env_cfg.py>`__
.. |cartpole-camera-presets| replace:: `Isaac-Cartpole-Camera-Presets-Direct-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_camera_presets_env_cfg.py>`__
.. |cartpole-resnet-link| replace:: `Isaac-Cartpole-RGB-ResNet18-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
.. |cartpole-theia-link| replace:: `Isaac-Cartpole-RGB-TheiaTiny-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
.. |cartpole-camera-presets| replace:: `Isaac-Cartpole-Camera-Direct-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_camera_presets_env_cfg.py>`__
.. |cartpole-camera-link| replace:: `Isaac-Cartpole-Camera-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__


.. |humanoid-direct-link| replace:: `Isaac-Humanoid-Direct-v0 <../../../source/isaaclab_tasks/isaaclab_tasks/direct/humanoid/humanoid_env.py>`__
Expand Down Expand Up @@ -783,146 +783,31 @@ inferencing, including reading from an already trained checkpoint and disabling
- Direct
- **rl_games** (PPO), **skrl** (IPPO, PPO, MAPPO)
-
* - Isaac-Cartpole-Camera-Showcase-Box-Box-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Box-Discrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Box-MultiDiscrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Dict-Box-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Dict-Discrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Dict-MultiDiscrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Tuple-Box-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Tuple-Discrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Showcase-Tuple-MultiDiscrete-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``
* - Isaac-Cartpole-Camera-Presets-Direct-v0 (Requires running with ``--enable_cameras``)
* - Isaac-Cartpole-Camera-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **rl_games** (PPO), **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``, ``rgb``, ``depth``, ``albedo``, ``semantic_segmentation``, ``simple_shading_constant_diffuse``, ``simple_shading_diffuse_mdl``, ``simple_shading_full_mdl``
* - Isaac-Cartpole-Direct-v0
* - Isaac-Cartpole-Camera-Showcase-Direct-v0 (Requires running with ``--enable_cameras``)
-
- Direct
- **rl_games** (PPO), **rsl_rl** (PPO), **skrl** (PPO), **sb3** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-RGB-ResNet18-v0 (Requires running with ``--enable_cameras``)
-
- Manager Based
- **rl_games** (PPO)
- ``newton_mjwarp``, ``physx``
* - Isaac-Cartpole-RGB-TheiaTiny-v0 (Requires running with ``--enable_cameras``)
- **skrl** (PPO)
- ``newton_renderer``, ``ovrtx_renderer``, ``isaacsim_rtx_renderer``, ``box_box``, ``box_discrete``, ``box_multidiscrete``, ``dict_box``, ``dict_discrete``, ``dict_multidiscrete``, ``tuple_box``, ``tuple_discrete``, ``tuple_multidiscrete``
* - Isaac-Cartpole-Camera-v0 (Requires running with ``--enable_cameras``)
-
- Manager Based
- **rl_games** (PPO)
- ``newton_mjwarp``, ``physx``
* - Isaac-Cartpole-Showcase-Box-Box-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Box-Discrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Box-MultiDiscrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Dict-Box-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Dict-Discrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Dict-MultiDiscrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Discrete-Box-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Discrete-Discrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Discrete-MultiDiscrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-MultiDiscrete-Box-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-MultiDiscrete-Discrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-MultiDiscrete-MultiDiscrete-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Tuple-Box-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Tuple-Discrete-Direct-v0
- ``newton_mjwarp``, ``physx``, ``rgb``, ``depth``, ``resnet18``, ``theia_tiny``
* - Isaac-Cartpole-Direct-v0
-
- Direct
- **skrl** (PPO)
- **rl_games** (PPO), **rsl_rl** (PPO), **skrl** (PPO), **sb3** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
* - Isaac-Cartpole-Showcase-Tuple-MultiDiscrete-Direct-v0
* - Isaac-Cartpole-Showcase-Direct-v0
-
- Direct
- **skrl** (PPO)
- ``newton_mjwarp``, ``physx``, ``ovphysx``
- ``newton_mjwarp``, ``physx``, ``ovphysx``, ``box_box``, ``box_discrete``, ``box_multidiscrete``, ``discrete_box``, ``discrete_discrete``, ``discrete_multidiscrete``, ``multidiscrete_box``, ``multidiscrete_discrete``, ``multidiscrete_multidiscrete``, ``dict_box``, ``dict_discrete``, ``dict_multidiscrete``, ``tuple_box``, ``tuple_discrete``, ``tuple_multidiscrete``
* - Isaac-Cartpole-v0
-
- Manager Based
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Memory Consumption
+====================================+================+===================+==========+===========+
| Isaac-Cartpole-Direct-v0 | |cartpole| | 4096 | 3.7 | 3.3 |
+------------------------------------+----------------+-------------------+----------+-----------+
| Isaac-Cartpole-RGB-Camera-Direct-v0| |cartpole-cam| | 1024 | 7.5 | 16.7 |
| Isaac-Cartpole-Camera-Direct-v0 | |cartpole-cam| | 1024 | 7.5 | 16.7 |
+------------------------------------+----------------+-------------------+----------+-----------+
| Isaac-Velocity-Rough-G1-v0 | |g1| | 4096 | 6.5 | 6.1 |
+------------------------------------+----------------+-------------------+----------+-----------+
Expand Down Expand Up @@ -57,7 +57,7 @@ CPU: AMD Ryzen 9 7950X 16-Core Processor
+=====================================+===================+==============+===================+====================+
| Isaac-Cartpole-Direct-v0 | 4096 | 1100000 | 910000 | 510000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Cartpole-RGB-Camera-Direct-v0 | 1024 | 50000 | 45000 | 32000 |
| Isaac-Cartpole-Camera-Direct-v0 | 1024 | 50000 | 45000 | 32000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Velocity-Rough-G1-v0 | 4096 | 94000 | 88000 | 82000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
Expand All @@ -77,7 +77,7 @@ CPU: Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
+=====================================+===================+==============+===================+====================+
| Isaac-Cartpole-Direct-v0 | 4096 | 620000 | 490000 | 260000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Cartpole-RGB-Camera-Direct-v0 | 1024 | 30000 | 28000 | 21000 |
| Isaac-Cartpole-Camera-Direct-v0 | 1024 | 30000 | 28000 | 21000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Velocity-Rough-G1-v0 | 4096 | 72000 | 64000 | 62000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
Expand All @@ -97,7 +97,7 @@ CPU: Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
+=====================================+===================+==============+===================+====================+
| Isaac-Cartpole-Direct-v0 | 4096 | 2700000 | 2100000 | 950000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Cartpole-RGB-Camera-Direct-v0 | 1024 | 130000 | 120000 | 90000 |
| Isaac-Cartpole-Camera-Direct-v0 | 1024 | 130000 | 120000 | 90000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Velocity-Rough-G1-v0 | 4096 | 290000 | 270000 | 250000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
Expand All @@ -117,7 +117,7 @@ CPU: Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
+=====================================+===================+==============+===================+====================+
| Isaac-Cartpole-Direct-v0 | 4096 | 10200000 | 8200000 | 3500000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Cartpole-RGB-Camera-Direct-v0 | 1024 | 530000 | 490000 | 260000 |
| Isaac-Cartpole-Camera-Direct-v0 | 1024 | 530000 | 490000 | 260000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
| Isaac-Velocity-Rough-G1-v0 | 4096 | 1200000 | 1100000 | 960000 |
+-------------------------------------+-------------------+--------------+-------------------+--------------------+
Expand Down
Loading