Merge pull request #14 from ARISE-Initiative/0.2.0

0.2.0

Author: amandlek

README.md

@@ -15,7 +15,8 @@
 
 -------
 ## Latest Updates
-[08/09/2021] **v0.1.0**: Initial code and paper release
+- [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder:
+- [08/09/2021] **v0.1.0**: Initial code and paper release
 
 -------
 
@@ -24,10 +25,11 @@
 Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch (offline) reinforcement learning, a lack of open-source human datasets and reproducible learning methods make assessing the state of the field difficult. The overarching goal of **robomimic** is to provide researchers and practitioners with:
 
 - a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations
+- a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations (see [this link](https://arise-initiative.github.io/robomimic-web/docs/introduction/quickstart.html#supported-datasets) for a list of supported datasets)
 - **high-quality implementations of several learning algorithms** for training closed-loop policies from offline datasets to make reproducing results easy and lower the barrier to entry
 - a **modular design** that offers great flexibility in extending algorithms and designing new algorithms
 
-This release of **robomimic** contains seven offline learning [algorithms](https://arise-initiative.github.io/robomimic-web/docs/modules/algorithms.html) and standardized [datasets](https://arise-initiative.github.io/robomimic-web/docs/introduction/results.html) collected across five simulated and three real-world multi-stage manipulation tasks of varying complexity. We highlight some features below:
+This release of **robomimic** contains seven offline learning [algorithms](https://arise-initiative.github.io/robomimic-web/docs/modules/algorithms.html) and standardized [datasets](https://arise-initiative.github.io/robomimic-web/docs/introduction/results.html) collected across five simulated and three real-world multi-stage manipulation tasks of varying complexity. We highlight some features below (for a more thorough list of features, see [this link](https://arise-initiative.github.io/robomimic-web/docs/introduction/quickstart.html#features-overview)):
 
 - **standardized datasets:** a set of datasets collected from different sources (single proficient human, multiple humans, and machine-generated) across several simulated and real-world tasks, along with a plug-and-play [Dataset class](https://arise-initiative.github.io/robomimic-web/docs/modules/datasets.html) to easily use the datasets outside of this project
 - **algorithm implementations:** several high-quality implementations of offline learning algorithms, including BC, BC-RNN, HBC, IRIS, BCQ, CQL, and TD3-BC

docs/api/robomimic.envs.rst

@@ -20,6 +20,14 @@ robomimic.envs.env\_gym module
    :undoc-members:
    :show-inheritance:
 
+robomimic.envs.env\_ig\_momart module
+-------------------------------------
+
+.. automodule:: robomimic.envs.env_ig_momart
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.envs.env\_robosuite module
 ------------------------------------
 

docs/api/robomimic.utils.rst

@@ -52,6 +52,14 @@ robomimic.utils.loss\_utils module
    :undoc-members:
    :show-inheritance:
 
+robomimic.utils.macros module
+-----------------------------
+
+.. automodule:: robomimic.utils.macros
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.utils.obs\_utils module
 ---------------------------------
 
@@ -60,6 +68,14 @@ robomimic.utils.obs\_utils module
    :undoc-members:
    :show-inheritance:
 
+robomimic.utils.python\_utils module
+------------------------------------
+
+.. automodule:: robomimic.utils.python_utils
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.utils.tensor\_utils module
 ------------------------------------
 

docs/images/modules.png

@@ -13,6 +13,7 @@ Welcome to robomimic's documentation!
    introduction/overview
    introduction/installation
    introduction/quickstart
+   introduction/features
    introduction/advanced
    introduction/examples
    introduction/datasets
@@ -25,6 +26,7 @@ Welcome to robomimic's documentation!
 
    modules/overview
    modules/dataset
+   modules/observations
    modules/algorithms
    modules/models
    modules/configs

docs/images/momart_bowl_in_sink.png

@@ -30,15 +30,15 @@ data (group)
 
   - `dones` (dataset) - done signal, equal to 1 if playing the corresponding action in the state should terminate the episode. Shape (N,) where N is the length of the trajectory.
 
-  - `obs` (group) - group for the observation modalities. Each modality is stored as a dataset.
+  - `obs` (group) - group for the observation keys. Each key is stored as a dataset.
 
-    - `<modality_1>` (dataset) - the first observation modality. Note that the name of this dataset and shape will vary. As an example, the name could be "agentview_image", and the shape could be (N, 84, 84, 3). 
+    - `<obs_key_1>` (dataset) - the first observation key. Note that the name of this dataset and shape will vary. As an example, the name could be "agentview_image", and the shape could be (N, 84, 84, 3). 
 
       ...
 
   - `next_obs` (group) - group for the next observations.
 
-    - `<modality_1>` (dataset) - the first observation modality.
+    - `<obs_key_1>` (dataset) - the first observation key.
 
       ...
 
@@ -169,6 +169,25 @@ For more details on how the released `demo.hdf5` dataset files were used to gene
 
 
 
+## MOMART Datasets
+
+<p align="center">
+  <img width="19.0%" src="../images/momart_table_setup_from_dishwasher_overview.png">
+  <img width="19.0%" src="../images/momart_table_setup_from_dresser_overview.png">
+  <img width="19.0%" src="../images/momart_table_cleanup_to_dishwasher_overview.png">
+  <img width="19.0%" src="../images/momart_table_cleanup_to_sink_overview.png">
+  <img width="19.0%" src="../images/momart_unload_dishwasher_to_dresser_overview.png">
+  <img width="19.0%" src="../images/momart_bowl_in_sink.png">
+  <img width="19.0%" src="../images/momart_dump_trash.png">
+  <img width="19.0%" src="../images/momart_grab_bowl.png">
+  <img width="19.0%" src="../images/momart_open_dishwasher.png">
+  <img width="19.0%" src="../images/momart_open_dresser.png">
+ </p>
+
+This repository is fully compatible with [MOMART](https://sites.google.com/view/il-for-mm/home) datasets, a large collection of long-horizon, multi-stage simulated kitchen tasks executed by a mobile manipulator robot. See [this link](https://sites.google.com/view/il-for-mm/datasets) for a breakdown of the MOMART dataset structure, guide on downloading MOMART datasets, and running experiments using the datasets.
+
+
+
 ## D4RL Datasets
 
 This repository is fully compatible with most [D4RL](https://github.com/rail-berkeley/d4rl) datasets. See [this link](./results.html#d4rl) for a guide on downloading D4RL datasets and running D4RL experiments.

docs/images/momart_dump_trash.png

@@ -153,4 +153,10 @@ Please see the [Config documentation](../modules/configs.html) for more informat
 
 ## Observation Networks Example
 
-The example script in `examples/simple_obs_net.py` discusses how to construct networks for taking observation dictionaries as input, and that produce dictionaries as outputs. See [this section](../modules/models.html#observation-encoder-and-decoder) in the documentation for more details.
+The example script in `examples/simple_obs_net.py` discusses how to construct networks for taking observation dictionaries as input, and that produce dictionaries as outputs. See [this section](../modules/models.html#observation-encoder-and-decoder) in the documentation for more details.
+
+
+
+## Custom Observation Modalities Example
+
+The example script in `examples/add_new_modality.py` discusses how to (a) modify pre-existing observation modalities, and (b) add your own custom observation modalities with custom encoding. See [this section](../modules/models.html#observation-encoder-and-decoder) in the documentation for more details about the encoding and decoding process.

docs/images/momart_grab_bowl.png

@@ -0,0 +1,41 @@
+# Features Overview
+
+## Summary
+
+In this section, we briefly summarize some key features and where you should look to learn more about them.
+
+1. **Datasets supported by robomimic**
+   - See a list of supported datasets [here](./features.html#supported-datasets).<br><br>
+2. **Visualizing datasets**
+   - Learn how to visualize dataset trajectories [here](./datasets.html#view-dataset-structure-and-videos).<br><br>
+3. **Reproducing paper experiments**
+   - Easily reproduce experiments from the following papers
+     - robomimic: [here](./results.html)
+     - MOMART: [here](https://sites.google.com/view/il-for-mm/datasets)<br><br>
+4. **Making your own dataset**
+   - Learn how to make your own collected dataset compatible with this repository [here](./datasets.html#dataset-structure). 
+   - Note that **all datasets collected through robosuite are also readily compatible** (see [here](./datasets.html#converting-robosuite-hdf5-datasets)).<br><br>
+5. **Using filter keys to easily train on subsets of a dataset**
+   - See [this section](./datasets.html#filter-keys-and-train-valid-splits) on how to use filter keys.<br><br>
+6. **Running hyperparameter scans easily**
+   - See [this guide](./advanced.html#using-the-hyperparameter-helper-to-launch-runs) on running hyperparameter scans.<br><br>
+7. **Using pretrained models in the model zoo**
+   - See [this link](./model_zoo.html) to download and use pretrained models.<br><br>
+8. **Getting familiar with configs**
+   - Learn about how configs work [here](../modules/configs.html).<br><br>
+9. **Getting familiar with operations over tensor collections**
+   - Learn about using useful tensor utilities [here](../modules/utils.html#tensorutils).<br><br>
+10. **Creating your own observation modalities**
+    - Learn how to make your own observation modalities and process them with custom network architectures [here](../modules/observations.html).<br><br>
+11. **Creating your own algorithm**
+    - Learn how to implement your own learning algorithm [here](../modules/algorithms.html#building-your-own-algorithm).<br><br>
+
+## Supported Datasets
+
+This is a list of datasets that we currently support, along with links on how to work with them. This list will be expanded as more datasets are made compatible with robomimic.
+
+- [robomimic](./results.html#downloading-released-datasets)
+- [robosuite](./datasets.html#converting-robosuite-hdf5-datasets)
+- [MOMART](./datasets.html#momart-datasets)
+- [D4RL](./results.html#d4rl)
+- [RoboTurk Pilot](./datasets.html#roboturk-pilot-datasets)

docs/images/momart_open_dishwasher.png

@@ -15,11 +15,11 @@
 
 Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch (offline) reinforcement learning, a lack of open-source human datasets and reproducible learning methods make assessing the state of the field difficult. The overarching goal of **robomimic** is to provide researchers and practitioners with:
 
-- a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations
+- a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations (see [this link](./features.html#supported-datasets) for a list of supported datasets)
 - **high-quality implementations of several learning algorithms** for training closed-loop policies from offline datasets to make reproducing results easy and lower the barrier to entry
 - a **modular design** that offers great flexibility in extending algorithms and designing new algorithms
 
-This release of **robomimic** contains seven offline learning [algorithms](../modules/algorithms.html) and standardized [datasets](./results.html) collected across five simulated and three real-world multi-stage manipulation tasks of varying complexity. We highlight some features below:
+This release of **robomimic** contains seven offline learning [algorithms](../modules/algorithms.html) and standardized [datasets](./results.html) collected across five simulated and three real-world multi-stage manipulation tasks of varying complexity. We highlight some features below (for a more thorough list of features, see [this link](./features.html#features-overview)):
 
 - **standardized datasets:** a set of datasets collected from different sources (single proficient human, multiple humans, and machine-generated) across several simulated and real-world tasks, along with a plug-and-play [Dataset class](../modules/datasets.html) to easily use the datasets outside of this project
 - **algorithm implementations:** several high-quality implementations of offline learning algorithms, including BC, BC-RNN, HBC, IRIS, BCQ, CQL, and TD3-BC

docs/images/momart_open_dresser.png

@@ -108,6 +108,3 @@ Instead of storing the observations, which can consist of high-dimensional image
 ```sh
 python run_trained_agent.py --agent /path/to/model.pth --n_rollouts 50 --horizon 400 --seed 0 --dataset_path /path/to/output.hdf5
 ```
-
-
-

docs/images/momart_table_cleanup_to_dishwasher_overview.png

@@ -22,6 +22,24 @@ $ python train.py --config ../exps/paper/core/lift/ph/low_dim/bc.json
 
 See the [downloading released datasets](./results.html#downloading-released-datasets) section below for more information on downloading different datasets, and the [results on released datasets](./results.html#results-on-released-datasets) section below for more detailed information on reproducing different results from the study.
 
+## Quick Example
+
+In this section, we show a simple example of how to reproduce one of the results from the study - the BC-RNN result on the Lift (Proficient-Human) low-dim dataset.
+
+```sh
+# default behavior for download script - just download lift proficient-human low-dim dataset to robomimic/../datasets
+$ python download_datasets.py
+
+# generate json configs for running all experiments at robomimic/exps/paper
+$ python generate_paper_configs.py --output_dir /tmp/experiment_results
+
+# the training command can be found in robomimic/exps/paper/core.sh
+# Training results can be viewed at /tmp/experiment_results (--output_dir when generating paper configs).
+$ python train.py --config ../exps/paper/core/lift/ph/low_dim/bc.json
+```
+
+See the [downloading released datasets](./results.html#downloading-released-datasets) section below for more information on downloading different datasets, and the [results on released datasets](./results.html#results-on-released-datasets) section below for more detailed information on reproducing different results from the study.
+
 ## Downloading Released Datasets
 
 Released datasets can be downloaded easily by using the `download_datasets.py` script. **This is the preferred method for downloading the datasets**, because the script will also set up a directory structure for the datasets that works out of the box with examples for reproducing some benchmark results with the repository. A few examples of using this script are provided below.

docs/images/momart_table_cleanup_to_sink_overview.png

@@ -43,7 +43,7 @@ We also list additional suggested contributing guidelines that we adhered to dur
 
 - When creating new networks (e.g. subclasses of `Module` in `models/base_nets.py`), always sub-modules into a property called `self.nets`, and if there is more than one sub-module, make it a module collection (such as a `torch.nn.ModuleDict`). This is to ensure that the pattern `model.to(device)` works as expected with multiple levels of nested torch modules. As an example of nesting, see the `_create_networks` function in the `VAE` class (`models/vae_nets.py`) and the `MIMO_MLP` class (`models/obs_nets.py`).
 
-- Do not use default mutable arguments -- they can lead to terrible bugs and unexpected behavior (see [this link](https://florimond.dev/blog/articles/2018/08/python-mutable-defaults-are-the-source-of-all-evil/) for more information). For this reason, in functions that expect optional dictionaries and lists (for example, the `visual_core_kwargs` argument in the  `obs_encoder_factory` function, or the `layer_dims` argument in the `MLP` class constructor), we use a default argument of `visual_core_kwargs=None` or an empty tuple (since tuples are immutable) `layer_dims=()`.
+- Do not use default mutable arguments -- they can lead to terrible bugs and unexpected behavior (see [this link](https://florimond.dev/blog/articles/2018/08/python-mutable-defaults-are-the-source-of-all-evil/) for more information). For this reason, in functions that expect optional dictionaries and lists (for example, the `core_kwargs` argument in the  `obs_encoder_factory` function, or the `layer_dims` argument in the `MLP` class constructor), we use a default argument of `core_kwargs=None` or an empty tuple (since tuples are immutable) `layer_dims=()`.
 
 - Prefer `torch.expand` over `torch.repeat` wherever possible, for memory efficiency. See [this link](https://discuss.pytorch.org/t/expand-vs-repeat-semantic-difference/59789) for more details.
 

docs/images/momart_table_setup_from_dishwasher_overview.png

@@ -8,6 +8,7 @@ A list of projects and papers that use **robomimic**. If you would like to add y
 
 ## Imitation Learning and Batch (Offline) Reinforcement Learning
 
+- [Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation](https://arxiv.org/abs/2112.05251) Josiah Wong, Albert Tung, Andrey Kurenkov, Ajay Mandlekar, Li Fei-Fei, Silvio Savarese, Roberto Martín-Martín
 - [Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control](https://arxiv.org/abs/2103.00375) Chen Wang, Rui Wang, Danfei Xu, Ajay Mandlekar, Li Fei-Fei, Silvio Savarese
 - [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://arxiv.org/abs/2012.06733) Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, Silvio Savarese
 - [Learning Multi-Arm Manipulation Through Collaborative Teleoperation](https://arxiv.org/abs/2012.06738) Albert Tung, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, Silvio Savarese

docs/images/momart_table_setup_from_dresser_overview.png

@@ -141,7 +141,7 @@ config, _ = config_from_checkpoint(algo_name=algo_name, ckpt_dict=ckpt_dict)
 model = algo_factory(
     algo_name,
     config,
-    modality_shapes=ckpt_dict["shape_metadata"]["all_shapes"],
+    obs_key_shapes=ckpt_dict["shape_metadata"]["all_shapes"],
     ac_dim=ckpt_dict["shape_metadata"]["ac_dim"],
     device=device,
 )

docs/images/momart_unload_dishwasher_to_dresser_overview.png

@@ -30,7 +30,7 @@ dataset = SequenceDataset(
 - `hdf5_path`
 	- The absolute / relative path to the hdf5 file containing training demonstrations. See [datasets](../introduction/datasets.html) page for the expected data structure.
 - `obs_keys`
-	- A list of strings specifying which observation modalities to read from the dataset. This is typically read from the config file: our implementation pools observation keys from `config.observation.modalities.obs.low_dim` and `config.observation.modalities.obs.images`.
+	- A list of strings specifying which observation modalities to read from the dataset. This is typically read from the config file: our implementation pools observation keys from `config.observation.modalities.obs.low_dim` and `config.observation.modalities.obs.rgb`.
 - `dataset_keys`
 	- Keys of non-observation data to read from a demonstration. Typically include `actions`, `rewards`, `dones`.
 - `seq_length`