When the shortbread is crisp and the pralines glisten in their tray, you know the quality is right. What delights consumers and manufacturers alike can be traced back to systems that are much less prominent than conveyor belts or robots. And yet camera-based scanners are capable of great things when it comes to inspection. They continuously check the shape, colour, dimensions and height of a wide range of products. The technology has become increasingly sophisticated over the decades – in no small part thanks to the innovative strength of leading packaging machine manufacturers.
Camera-based scanners are essential to the critical infrastructure of many packaging and picking lines, as they detect everything that takes place on the conveyor belts below them. Failing to do so would be counterproductive. Products that are damaged or incorrectly placed in the packaging inevitably lead to unwanted rejects, machine downtime or even costly recalls – to the detriment of manufacturing companies and consumers. Defective products ultimately pose a safety risk that needs to be avoided at all costs. To this end, scanners play an important role. Different types enable companies in many different industries to carry out quality control in line with their very specific production requirements.
Clarity thanks to image processing
Camera-based control systems have been around since the mid-1980s – with the introduction of the Pattern Considering System, or Pacos for short, the first solution of its kind on the market. It revolutionised the world of mechanical packaging lines, which had previously managed without machine image processing. Pacos was the first to offer a program-controlled image analysis system that could recognise unsorted objects on a conveyor belt during the packaging of small parts. This paved the way for the first quality controls, for example in terms of position and orientation. With the advent of the first picker lines – and with them, wider product belts – the technology proved to be inadequate, as it reached its optical limits at widths of over 600 millimetres. This led to increasing demand for more powerful systems that could reliably inspect products even in large-scale lines – and capture different parameters such as colour, shape and dimensions in the process.
2D reflected light scanners
In the early 2000s, 2D scanners marked a turning point, as the wider conveyor belts used by many baked goods manufacturers called for the use of line scan cameras. Unlike area scan cameras, such as those found in smartphones, they have only one row or line of pixels. Line scan cameras can take fast, one-dimensional images of moving objects – and efficiently process even large amounts of data. The devices accomplish this with a very low resolution. They generate individual images of one pixel at a time, and can repeat this process many times in a very short timeframe. By combining many successive lines, a high-resolution colour image is created that provides detailed insight into the quality of the products. Deviations, for example in shape and colour, can be reliably detected thanks to this ‘top view’.
The high quality is also ensured by an optical module with a Fresnel lens. Thanks to its ring-shaped structure, this type of lens is thinner and more lightweight than conventional optical lenses, but it focuses and directs light just as efficiently to a specific focal point. This prevents an effect known as parallax, which can make image processing more difficult. The phenomenon refers to the apparent changing position of an object in space when it is viewed from two different points. In image processing, this shift out of the optical axis could cause the products being inspected to appear different – making unambiguous quality assessments a challenge.
A clear view from above
With a Fresnel lens and top-view inspection, 2D scanners can overcome this hurdle. Products are always telecentric, i.e. directly under the optical axis of the cameras – so there is no parallax. The scanners capture colour, shape and dimensions, but not height – and therefore cannot be used to represent a 3D object an object in space. Instead, for the first time ever, they enabled a modular design that could keep pace with the ever-widening product ranges. Schubert played a major role in initiating this development. The company’s engineers and programmers redesigned the 2D scanner so that the optical modules with camera and sensors measure just 200 millimetres. Depending on the Pickerline, several of these modules can be set up in a row. Even belt widths of up to 1,800 millimetres are not a problem for the 2D scanner.
Product control in three dimensions
However, a view from above is not always sufficient. If products, such as sandwich biscuits, should have a specific height, manufacturers also need to be able to assess this so that the biscuits reach the consumer in the desired dimensions. A 2D view would only show the top side of each sandwich biscuit, and therefore always deliver the same image. Beginning in 2014, 3D scanners from Schubert brought about a decisive change – and cleverly exploited the parallax for this purpose. By having several cameras capture the products from different observation points, their fields of view overlap. The multiple images provide the foundation for a stereoscopic overall image. The scanners calculate the individual segments of the subsequent elevation image from two images with different perspectives.
The patented light pattern approach also contributed to the advancement of 3D technology. Using a special projector, the 3D scanner projects light strips in a specific arrangement onto the product belt. As products pass through this area, they distort the light pattern and provide information about their height. The scanners calculate the 3D image from these distortions, which they pass on to the pick & place robots along with further coordinates such as product position and orientation. This is what makes the robots operable in the first place.
Keeping close track of the products
This interaction between robotics and scanners opens up considerable room for optimisation. In Pickerlines and packaging lines, several 2D and 3D scanners often check the quality of different products and pass the information on to the robots. Unlike the position in space, however, quality proves to be a constant: once detected at the beginning of the line, it can be assumed that essential product properties remain until the end of the process. The position of the products, however, can change continuously, for example due to mechanical influences in the machine, such as belt slippage or belt drift. The product coordinates therefore need to be recorded continuously and, if necessary, corrected and transmitted to the respective robot. So-called tracking scanners can perform these tasks. Unlike reflected light or 3D scanners, they do not check the quality, but merely determine where the products are located within the packaging line or picking line.
Each robot cell features a tracking scanner. There is also a one-off 2D or 3D scanner at the beginning of the line to check quality. This enables manufacturers to capture the coordinates more cost-effectively than with several 2D or 3D scanners per line. Technologically, such tracking scanners can already be implemented: cost-efficient area scan cameras and corresponding sensor technology are available on the market. However, leading manufacturers rely on their own developments with a high degree of vertical added value. This is how they achieve cost-efficient scanners with two cameras per module that can be variably retrofitted. The flexible tracking scanners teach robots to see, so to speak, by providing precise information about the position of the products to be packaged.
Leveraging the potential of AI
Especially when it comes to challenging product characteristics, different scanner types soon reach their limits – at least if they are based on algorithms. Variable phenomena such as stains or discolouration, for example, are difficult to describe using fixed parameters. This is why line manufacturers are focusing on scanners based on artificial intelligence. Instead of programming the scanners with algorithms – and with them, rules of action for clearly defined cases – they ‘train’ an underlying vision AI with image data. In the case of stains, this would be images of the relevant products, which can be quickly processed and used to build neural networks thanks to energy-efficient technologies such as embedded GPUs (graphics processing units). Over the long term, these should enable scanners to quickly and accurately capture even products with variable characteristics, including bulk materials, ensuring consistently high quality throughout the entire packaging process.
Length of text: 8 961 characters (with blank spaces) / 1 400 words
Information box image processing (1 242 characters)
Over the past forty years, Gerhard Schubert GmbH has firmly established itself as a leading manufacturer of patented image processing systems. The company was the first packaging machine manufacturer to systematically dedicate itself to image processing. Since 1985 has had its own development department – a pioneering move for the industry at the time. In the mid-1980s, the company laid the cornerstone for this with a collaboration that brought together scientists from the Karlsruhe Nuclear Research Centre and experts from Crailsheim. Together, Dr Abdelmalek Nasraoui and Dr Josef Pecht – both initially at the nuclear research centre and later working at Schubert – and Schubert designed image-processing solutions such as Pacos. The 2D and 3D scanners developed and patented by Schubert are based on this groundwork, but they took into account changing production requirements. Increasingly wider conveyor belts called for modular and efficient solutions to reliably inspect even large product volumes at high speeds. The modular design of the scanners, the light pattern for height detection and picker lines with counter-flow lanes for products and packaging materials are among Schubert’s patented solutions that have since set industry standards.
About Gerhard Schubert GmbH
Gerhard Schubert GmbH is a globally recognised market leader in top-loading packaging machines (TLM). For its digital, robot-based packaging machines, the family-owned company based in Crailsheim (Baden-Württemberg, Germany) builds on an interplay of simple mechanics, intelligent control technology and high modularity. With this philosophy and a highly developed culture of innovation, the company has been pursuing an entirely independent technological path for more than 50 years.
With its TLM technology, the machine manufacturer provides its customers with future-proof packaging machine solutions that are easy to operate, flexible in terms of format conversion, high-performance and stable in function. The TLM packaging machines pack products of all types and from all sectors – from food, confectionery, beverages, pharmaceuticals and cosmetics to technical products – into trays, cartons, boxes or into flow-wrap bags.
Well-known global companies in the consumer goods and pharmaceutical industries rely on Schubert’s automation solutions, as do numerous small, medium-sized and family-owned companies. Founded in 1966, the second generation of the company now employs 1,700 people.
Learn more at:
Homepage: www.schubert.group
Facebook: www.facebook.com/GerhardSchubertGmbh
LinkedIn: www.linkedin.com/company/gerhardschubertgmbhverpackungsmaschinen
Gerhard Schubert GmbH packaging machines
Industriegebiet Südost
Hofäckerstraße 7
74564 Crailsheim
Germany
Tel.: +49 7951 400-0
E-Mail: info@gerhard-schubert.de
www.schubert.group
Press/ media contact:
Ruess International GmbH – Member of Ruess Group
Benoît Wolff
Lindenspürstraße 22
70176 Stuttgart
Germany
Tel.: +49 711 16446-69
E-Mail: benoit.wolff@ruess-group.com
www.ruess-group.com
