Conference Agenda

Natural and intuitive human–robot interaction is increasingly important as robotic systems are progressively integrated into human environments. Conventional teleoperation approaches often rely on dedicated interfaces or wearable sensors, which increase system complexity and limit accessibility.

This work proposes a vision-based framework for robot teleoperation that exploits a human digital twin (HDT) as an intermediate layer between motion capture and robot control. Human arm motion is acquired using an RGB-D camera (Luxonis OAK-D Pro) combined with a YOLOv8-based pose estimation pipeline, enabling markerless tracking of upper-limb joints. The extracted trajectories are processed in MATLAB/Simscape to drive a multibody human model via inverse kinematics, generating biomechanically consistent motion. The resulting wrist trajectory is then used as input for the inverse kinematics of a robotic manipulator.

Unlike conventional leader–follower schemes, the proposed architecture introduces a digital twin layer that enables visualization, validation, and filtering of human motion prior to execution. A pick-and-place experiment is conducted to evaluate the framework and assess the influence of camera placement on tracking accuracy. Results show that an oblique camera configuration significantly improves performance compared to frontal and top-down views, highlighting the importance of sensing geometry in vision-based teleoperation systems.