Sceniris: a Fast Procedural Scene Generation Framework

Jinghuan Shang¹, Harsh Patel², Ran Gong¹, Karl Schmeckpeper¹

¹Robotics and AI Institute ²University of Waterloo

[Paper]

Sceniris is a procedural scene generation framework that generates a massive amount of collision-free environment instances in a short time. How fast?. The framework is developed on top of scene_synthesizer and curobo. Sceniris also supports more spatial relationships than scene_synthesizer. The speedup mainly comes from three aspects: batched sampling, batched pose computation & forward kinematics, and curobo's parallel collision checking.

How fast is Sceniris

To generate scenes from similar env configs:

Sceniris: 16384 env instances (13000+ valid) in just 30 seconds

Scene_synthesizer (with 10 multiprocessing): 64 env instances in around 38 seconds (our device is a VM with NVIDIA L4 GPU)

Installation

1. Install curobo. Note: before installing curobo, make sure you have pyTorch installed under the same cuda version of your system (can be checked by nvcc -V).
2. Install sceniris pip install git+https://github.com/bdaiinstitute/sceniris.git. You may need to review the licence terms mentioned in scene_synthesizer, and the [licence] of curobo
3. [Optional] if you want to deal with USD assets but there is no OpenUSD installed, pip install usd-core
4. [Optional] if you want to use the robot reachability feature, run git submodule update --init --recursive to pull rm4d, and install it with cd rm4d && pip install -e .

Usage

• From config

import sceniris
from sceniris.scene import Scene
import os
import logging
logging.basicConfig(level=logging.INFO, force=True)
example_asset_folder = os.path.join(os.path.dirname(sceniris.__file__), "example_assets")
# give a config
cfg = {
    "env": {
        "num_envs": 4,
        "env_size": 1.0,
    },
    "objects": [
        {
            "id": "orange",
            "asset_path": os.path.join(example_asset_folder, "orange/orange.usd"),
            "constraints": [
                {
                    "anchor_object_ids": ["drawer", "mug"],
                    "direction": "middle",
                }
            ]
        },
        {
            "id": "drawer",
            "asset_path": os.path.join(example_asset_folder, "drawer_unit_1/drawer_unit_1.usd"),
            "joint_states": [
                {
                    "joint_ids": ["drawer_joint_lower"],
                    "max": 0.3,
                    "min": 0.1,
                    "distribution": "uniform",
                },
                {
                    "joint_ids": ["drawer_joint_upper"],
                    "max": 1.0,
                    "min": 0.7,
                    "distribution": "uniform",
                },
            ]
        },
        {
            "id": "banana",
            "asset_path": os.path.join(example_asset_folder, "banana/banana.usd"),
            "parent_id": "drawer/unit_1_upper_drawer", # might need to optimize the part name in trimesh scene
            "relation_to_parent": "top",
        },
        {
            "id": "mug",
            "asset_path": os.path.join(example_asset_folder, "mug/mug.usd"),
            "constraints": [
                {
                    "anchor_object_ids": ["drawer"],
                    "direction": "front",
                    "distance": 0.5,
                    "distance_type": "greater",
                    "relation_axis_transform": "drawer",
                }
            ]
        },
    ]
}
scene = Scene.gen_from_cfg(cfg) # initialize a scene
scene.gen() # generate!!!
poses, joint_states, valid_env_mask = scene.export_scene_to_poses_and_joint_states() # optionally, dump object poses, joint states, and valid environments data
# scene.show(env_ids=[0,1,2,3]) # it will pop up a window for visualizing the generated (without texture)

You can also refer to our demo script.

If the script works, it will generate the scenes similar to the image below:

Acknowledgement

scene_synthesizer, curobo

Maintainance Disclaimer

The project will be under light maintenance. No feature development guaranteed. We welcome any bug reports or PRs that help us to improve the project.