Questions about the opening cabinet task and joint physics

[vlongle]

Hello,

Thank you for this great work! I'm looking at the open_cabinet_drawer env. It seems that the handle_link is extracted by looking for handle_ meshes within each link.

We're interested in using our own automatically generated assets. The entire mesh of the object is generated by a diffusion model, which is then part-segmented. We would have a drawer link of the object but won't have fine-grained link like handle. Would ManiSkill/SAPIEN still work? Can the simulator rely on the fact that there is a protruding region of the drawer to simulate grasping motion without explicit handle annotation?

Secondly, the PartNet-Mobility assets seem to not have physics. Normally, in Mujoco, we'd have to do something like

<joint name="joint_2" axis="-1 0 0" type="slide" range="0 0.8" frictionloss="1" />

and randomize frictionloss for sim2real. But how are physical properties handled in Maniskill? Is it using some default parameters? If so, in your experience, do your default physic parameters transfer well to a real Franka arm?

Thank you.

[vlongle]

Regarding the first question, I have made a version of the cabinet environment that doesn't rely on handle and tele-operated the trajectory in the attached video. Seems that the grasping physics still works.

maniskill_no_handle.mp4

[StoneT2000]

For domain randomization see this: https://maniskill.readthedocs.io/en/latest/user_guide/tutorials/domain_randomization.html

Generally do all the domain randomizations during the load scene function.

I don't recall the partnet mobility dataset having material based labels. In MS1/MS2 we did domain randomization on joint properties like this by modifying friction and damping values

    def _set_joint_physical_parameters(self):
        for joint in self.cabinet.get_active_joints():
            joint.set_friction(...)
            joint.set_drive_property(
                stiffness=0, damping=...
            )

You can do something similar in MS3.

I think @callmeray ran the first sim2real experiments with cabinet opening back in 2021, he can chime in maybe on sim2real for articulated objects.