Skip to content

Improve lift rl training example#529

Open
qianl-nv wants to merge 7 commits intomainfrom
qianl/lift_rl_training_fix
Open

Improve lift rl training example#529
qianl-nv wants to merge 7 commits intomainfrom
qianl/lift_rl_training_fix

Conversation

@qianl-nv
Copy link
Copy Markdown
Collaborator

@qianl-nv qianl-nv commented Apr 1, 2026

Summary

Update the lift object RL example to have a high success rate model

Detailed description

  • What was the reason for the change?
    Existing lift object RL training model success rate is low (~30%), and the arm motion is unnatural.
  • What has been changed?
    Add a franka joint control embodiment for RL training to avoid the weird arm motion from IK version
    Update the observation to include joint and target poses only
    Fix a bug in the base rls policy so the target pose (task_obs in addition to policy) is passed to the actor/critic model
    Fix a bug resulting ~0 success rate for parallel eval due to incorrect object/target frame in the success term
    Update RL docs with latest models and commands
  • What is the impact of this change?
    The RL model now gets 70~80% success rate within 1.5h

@qianl-nv qianl-nv changed the title Improve lift rl training fix Improve lift rl training example Apr 1, 2026
@qianl-nv qianl-nv requested review from viiik-inside and xyao-nv April 1, 2026 17:03
@qianl-nv qianl-nv marked this pull request as ready for review April 1, 2026 17:06
# Uses FRANKA_PANDA_CFG (gravity on, stiffness=80, damping=4) instead of HIGH_PD.
_FRANKA_JOINT_POS_CFG = FRANKA_PANDA_CFG.copy()
_FRANKA_JOINT_POS_CFG.spawn.usd_path = (
f"{ISAACLAB_STAGING_NUCLEUS_DIR}/Arena/assets/robot_library/franka_panda_hand_on_stand.usd"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it share the same path as above _FRANKA_CFG.spawn.usd_path

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes they share the same. updated to use _FRANKA_CFG.spawn.usd_path directly

class FrankaEmbodiment(EmbodimentBase):
"""Embodiment for the Franka robot."""

name = "franka"
Copy link
Copy Markdown
Collaborator

@xyao-nv xyao-nv Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not now. But can you add a TODO?
Rename it to franka_ik, add a base class where action_cfg is missing, and refactor child classes for both franka_joint_pos and franka_ik.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, great suggestion! Added the todo comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants