Stereo-Based Single-Shot Hand-to-Eye Calibration for Robot Arms

Robot hand-to-eye calibration is a necessary process for a robot arm to perceive and interact with its environment. Past approaches required collecting multiple images using a calibration board placed at different locations relative to the robot. When the robot or camera is displaced from its calibrated position, hand-eye calibration must be redone using the same tedious process. In this research, we developed a novel method that performs hand-to-eye calibration using a stereo camera to automatically generate a transformation matrix from the world to the camera coordinate frame from a single stereo image. We use a robot-pointer tool attached to the robot’s end-effector to establish a relationship between the world and the robot coordinate frame. Then, we establish the relationship between the camera and the robot using a transformation matrix that maps points observed in the stereo image frame from two-dimensional space to the robot’s three-dimensional coordinate frame. Our analysis of the stereo calibration showed a reprojection error of 0.26 pixels. An evaluation metric was developed to test the camera-to-robot transformation matrix, and the experimental results showed root mean square errors of less than 1 mm in the x and y directions, and less than 2 mm in the z directions in the robot coordinate frame. The results show that with this work, we contribute a hand-to-eye calibration method that uses three non-collinear points in a single stereo image to map camera-to-robot coordinate-frame transformations.