Breakthrough in Vision-Guided Robotics

Hand-eye Integration: A Breakthrough in Visually Guided Robots

Having sharp claws, there should be sharp eyes. Like the eagle soaring in the sky, sharp eyes with flexible and powerful talons, nothing to lose. The success of the cake sorting line in Malaysia has been very encouraging. We realized early on that a flexible robot should be equipped with an accurate visual recognition system. Deep vision recognition system can help robots locate and capture target objects more accurately, improve operation efficiency and accuracy, so as to meet the needs of different application scenes.

To put eyes on sharp claws

In 2020, an Italian client contacted Utrust Pack and wanted us to provide a production line for sorting sachet packs. The client's main products are health products in sachets packaging, which are first packed into strip packs by automatic packaging machines, and then the outer packaging is completed by manual counting and boxing. Since the outer packing required a lot of labor work and coincided with the pandemic outbreak, there was a serious shortage of labor. The client wanted to reduce the need for labor by upgrading the automation of the outer packaging process.

Those who succeed are hard to achieve but easy to lose, and those who are in time are hard to get but easy to lose. After ten years of sharpening a sword, it's time to try Frost Blade.

Wait a minute! First, let's sort out the requirements...

1. Sorting

Loose and messy sachets packages need to be separated and independent.

2. Identification

The direction of the sachets must be consistent.

3. Counting

The strips should be organized one by one according to the set number (e.g. 20 strips) in order to be put into the box.

4. Generalization

There is a wide variety of products in the strip pack, and the size, shape and quantity requirements may be different.

5. Speed

The production efficiency is 300~500 packs/min depending on the size of the strips.

A pile of messy sachets packs needs to be arranged neatly in a direction and in a fixed number

These requirements are unprecedented! Conventional equipment has been very mature for automatic counting and boxing of single kinds of sachets, but it also has great limitations. The sorting equipment can only organize the messy sachet packs into a single column without overlapping, but there is no way to unify the direction of the printing pattern on the sachet packs. The counting machine cannot be compatible with any number of counts, and the compatible count values must be multiplied, such as 6, 12, 18, etc., but 6 and 7 are not compatible.The box opening and entering equipment is only compatible with one or two kinds of boxes within a certain size. However, customers require compatibility with various sizes of sachets and various counting quantities, and some strips need to be placed in the box according to the direction of the uniform printing pattern.

The ideal is really full, the sachets but a little thin.

The above problems cannot be solved by non-depth vision. In contrast, a fine overall solution is less difficult to design.

The results of the program stage first surprised the Italian client

The next few months were spent on continuous research and experimentation. Data acquisition and labeling, hardware matching, simulation and optimization of the operating environment, image processing in motion…There are so many difficult problems that need to be broken through, which are too complicated.

Our vision engineers were confronted with thousands of sachet packets of different types, shapes and sizes, training and testing them repeatedly to ensure the accuracy and stability of the algorithm. Each advancement towards the target finally resulted in a recognition accuracy rate of 99.99% or more.

After the model training is completed, the R&D team starts system integration. This requires a seamless connection between the visual recognition system and the robotic arm control system to ensure stability and coordination between the two systems.

The test results show that the system can accurately identify and classify different packages, and guide the delta robot arm to perform precise grabbing and placing operations. At the same time, it can maintain stable and reliable performance on high-speed production lines. 2.5 seconds, 2 seconds 1.5 seconds, 1.2 seconds, 1 second! The recognition result command was sent, and the speed of the manipulator's grasping and precise placement also met the set requirements.

The long test debugging is a major challenge to willpower

For various reasons, we have temporarily abandoned the automated implementation of the sachets packing into the boxes and still keep the manual operation.

This was enough for our Italian customer to solve the most boring and tedious manual picking process.

Italy, a global design center, has a natural pickiness about design. We did not get carried away with the joy of conquering technology.

The product texture reflects the concept of humanization, artistic design, heavy but not romantic. We also highly value the recognition of our Italian customers who are spirited yet rigorous, free yet literary.

One day on the video link, our Italian customer was humming a tune. When showing us their equipment in operation on-site, it was all so worthwhile and comforting.