Yaskawa Robotics – How does the Robot Perception System work?

In addition to the need to perceive mechanical quantities related to their own working state, such as displacement, velocity, acceleration, force and torque, visual perception technology is an important aspect of industrial robot perception.

The visual servo system uses visual information as a feedback signal to control and adjust the position and posture of the robot. Applications in this area are mainly found in the semiconductor and electronics industries. Machine vision systems are also widely used in all aspects of quality inspection, workpiece identification, food sorting, and packaging.

Usually, robot visual servo control is a position-based visual servo or an image-based visual servo, which is also known as a three-dimensional visual servo and a two-dimensional visual servo, respectively, and these two methods have their own advantages and applicability, but also have some defects.

The location-based visual servo system uses the parameters of the camera to establish the mapping relationship between the image information and the position/attitude information of the robot end effector, and realizes the closed-loop control of the position of the robot end effector. The position and attitude error of the end effector are estimated by the position information of the end effector extracted from the real-time shooting image and the geometric model of the positioning target, and then the new pose parameters of each joint are obtained based on the position and attitude error. Position-based visual servos require that the end-effector should always be observable in the visual scene and calculate its 3D position and attitude information. Eliminating interference and noise in the image is the key to ensuring accurate calculation of position and attitude errors.

The 2D visual servo derives an error signal by comparing the features of the image captured by the camera with the given image (not the 3D geometric information). Then, the robot is made to complete servo control by the joint controller, the vision controller and the current working state of the robot. Compared with the three-dimensional visual servo, the two-dimensional visual servo has strong robustness to the calibration error of the camera and the robot, but in the design of the visual servo controller, it will inevitably encounter problems such as the singularity of the image Jacobian matrix and the local minimity.

In view of the limitations of 3D and 2D visual servo methods, a 2.5D visual servo method was proposed. It decouples the translational displacement of the camera from the closed-loop control of rotation, reconstructs the azimuth and imaging depth ratio in the three-dimensional space of the object based on the image feature points, and the translational part is represented by the feature point coordinates on the image plane. This method can successfully combine the image signal with the pose signal based on image extraction, and synthesize the error signal generated by them for feedback, which largely solves the problems of robustness, singularity and local minimity. However, there are still some problems to be solved in this method, such as how to ensure that the reference object is always within the camera's field of view during servo process, and the problem that the solution is not unique when decomposing the monogy matrix.

When building a vision controller model, it is necessary to find a suitable model to describe the mapping relationship between the end effector and the camera of the robot. The method of image Jacobian matrix is a widely used method in the field of robot vision servo research. The Jacobian matrix of an image is time-varying and therefore needs to be calculated or estimated online.

Yaskawa Robotics – How does the Robot Perception System work?

Yaskawa Robotics – How does the Robot Perception System work?

Sections viewed

Contact

working hours