Skip to content

TaskCreation

Jump to bottom
Hiroki Yamada edited this page Jan 21, 2026 · 5 revisions

Execution Mode

The application starts in the competition mode if the executionMode of InteractiveCleanupConfig.json is 0, it starts in the data generation mode if it is 1.

In the mode for the competition, the pre-generated EnvironmentInfoXX.json file is used to relocate objects and AvatarMotionXX.dat is used to reproduce the movements of the human avatar. In this case, if these files are not found, an error will occur.

In the data generation mode, you need to prepare a VR headset(Meta Quest 2). The position and orientation of relocatable objects are determined randomly upon execution, and the graspable objects are determined by the objects pointed at by the human avatar. Then EnvironmentInfoXX.json and AvatarMotionXX.dat will be output.

How to Create Tasks

You can generate EnvironmentInfoXX.json and AvatarMotionXX.dat using Meta Quest 2/3.
On the Windows PC, connect your Meta Quest 2/3 to the PC via Meta Horizon Link, and leave it in standby mode.
Note that EnvironmentInfo01.json and AvatarMotion01.dat will be updated when you perform the following data generation procedure.

  1. Start the ROS side application.
  2. Set the executionMode of InteractiveCleanupConfig.json to 1 and start the application.
  3. Proceed as usual until receiving "I_am_ready" message.
  4. Perform the pointing action using Meta Quest Touch Controller as follows:
    1. Press the A button or the X button. (Start recording)
    2. Control the thumbstick to move the avatar as needed. And press the middle finger trigger to display a laser pointer. The laser is just a guide for pointing.
    3. Select the grasping target with the laser pointer and press the index finger trigger.
    4. Select the destination with the laser pointer and press the index finger trigger.
    5. Move back to initial position if you moved. (This is for re-pointing)
    6. Press the A button or X button. (End recording)
  • The default number of the graspable objects is 13.
  • If you want to change the positions of the graspable objects, change the positions of Layout2019IC01/GraspingCandidatesPositions/GraspingCandidatesPositionXX.
    (The graspable objects will be placed randomly at the positions of GraspingCandidatesPositionXX.)
  • There are white spheres at the initial position of the human avatar as a landmark.

The competition data is output in the above procedure, and it is also possible to verify in real time how the robot moves according to the movements of the human who is pointing.

Clone this wiki locally