Immersive and Intelligent Humanoid Robot Control System

We have built a complete robot control system that does not require big and heavy wearable in order to acquire operator’s movement. Instead, we integrated MS Kinect v2 into our system for recognizing operator’s movement. The movement recognize by Kinect is noisy, so we developed a dedicated filter module to smooth the signals. To let the operator have an immersive experience and see what the robot can see, we also integrated VR technology. On our VR device, the operator can see the real-time video capture stream from the robot’s camera. They can also move their head to see around.

To make our system intelligent, we developed an algorithm that enable our robot to guess an undetected object’s location in a realtime video frame. If we want the robot to find an object, it can always provide hint on where the object will be most likely to appear.