Machine learning optimizes real-time inspection of instant noodle packaging

2023-08-25

Instant noodles can be found in almost every grocery store in China. To reduce the number of products with packaging errors and the resulting customer complaints, a large Chinese instant noodle producer decided to use Beckhoff’s control technology, including the TwinCAT Machine Learning machine Learning technology. This made possible an intelligent and reliable real-time inspection of the quality of the packages.

Tianjin FengYuLingKong the Electrical and Mechanical Equipment Co., Ltd., a system integrator specializing in industrial automation, was awarded the contract to supply a state-of-the-art packaging inspection system to one of the largest fast food and beverage producers in China. According to this food manufacturer, they have the largest instant noodle production lines in the world, with an average throughput of about 500 containers per minute and a total annual production of up to 4,800 million containers.

High customer satisfaction thanks to error-free packaging

Packaging and sealing are essential tasks in the manufacture of instant noodles. A package of noodles usually contains a block of pre-cooked noodles along with several sachets with spices, for example, flavoring powders, sauces and dried vegetables. On the production line, these individual sachets are arranged on a conveyor belt moving at high speed, placed on the corresponding noodle block and then sent together to the transverse sealing and packaging machine.

During the production process, there are several factors that can cause the seasoning sachets to slide between two noodle blocks and be sliced by the slicing machine, or to be packaged separately in two side-by-side packages. These defective products would lead to consumer complaints and damage the reputation of the company, so it is necessary to reduce the delivery of such products to distributors as much as possible. Since the type of machine updated by Tianjin FengYu has already produced with a very low error rate before, there is another aspect of quality control that is essential: it must be ensured that only defective products are reliably excluded and not good ones.

A large Chinese food company uses TwinCAT Machine Learning

to achieve the highest possible level of quality in the packaging of instant noodles.

Photo: ThamKC – stock.adobe.com .

Since it is not possible to see the processes inside the sealing and packaging machine, and moreover possible packaging defects are often also not visible from the outside during the subsequent optical inspection, it is very difficult to find out the exact cause of the aforementioned errors. Therefore, it is practically impossible to avoid defective products on principle. Despite this, and in order not to deliver low-quality products, instant noodle manufacturers use highly automated quality inspection devices with minimal latency.

Tianjin FengYu assisted the end user in this project with the rapid implementation of a high-performance quality control system. First of all, the mechanical and electrical parts of the machine were examined. According to Tianjin FengYu, in the process they discovered that, due to its transparency and flexibility, Beckhoff’s PC-based control technology is ideal for acquiring the necessary analysis data. The system integrator installed several sensors inside the machine and was able to acquire the first data sets for the analysis of the prototype quickly and easily through TwinCAT Scope View. The subsequent analysis of the data showed a certain sensitivity in the measured values that occurred when a machine mistakenly cut a seasoning sachet. However, these problematic events that lead to defective products cannot be reliably detected with conventional engineering methods. The reasons are various uncertainty factors that can affect the data, such as machine vibrations, changes in packaging material, transport speed and shear stress. To find a solution to this demanding analytics application, Tianjin FengYu decided to use TwinCAT Machine Learning as a means to implement data-driven engineering with ML (machine learning).

Machine learning in industrial applications

The basis of a product inspection is the classifiers, which make it possible to distinguish good products from defective ones. If ML-based classifiers are used, a mathematical model is learned from example data so that correct decisions can be made from there without the need for explicit programming.

Beckhoff offers several tools and open interfaces to support the entire engineering cycle, from data acquisition and model learning to the implementation of the learned model in the control system itself:

• Data acquisition: The quantity and quality of data have crucial effects on ML applications. A wide range of Beckhoff I/O and software products allows the acquisition of almost any data. Various functions of the TwinCAT software, such as Scope View, Database Server, Data Agent and Analytics Logger, allow data to be saved on an industrial PC, in local or remote databases or in the cloud.
• Model learning: The acquired data must be pre-processed initially to find a correlation between the data and the desired results, or to reinforce that correlation. Subsequently, an ML algorithm suitable for learning the model is identified and parameterized. Beckhoff recommends the use of open and established ML frameworks for this purpose, such as PyTorch, Keras or Scikit-learn. Finally, the learned model can be saved as a file in the standard exchange format ONNX (Open Neural Network Exchange). The ONNX file describes the operations and parameters of the learned model and can then be converted to a binary format (BML) more suitable for serialization in TwinCAT.
• Using models: The TwinCAT execution environment for machine learning (TF3800 and TF3810) can load the learned model files (in BML format) into the control systems dynamically, from there the models can be executed in real time with execution cycles of less than 1 ms. In this way, the results of the inference (execution of a memorized ML model) can be processed directly, transferred to the output devices via ultra-fast EtherCAT communication, and thus the machine can be controlled in real time.

Lu Peng (left), project manager of Tianjin FengYu, Wan Pinlei (center),

system application engineer of Beckhoff China (with the CX51xx embedded PC as the hardware center of the TwinCAT

Machine Learning solution) and Xie Shaowei (right), technical support engineer of Beckhoff China. Photo: Beckhoff.

The detection of defective products in the instant noodle production line was implemented exactly according to the three-step method described above. Firstly, the sensor data was captured via the EtherCAT EL1xxx or EL3xxx digital and analogue input terminals and TwinCAT Scope View. Subsequently, the ML model was learned using the open source Scikit-learn framework and the model description file was generated from it. The necessary pre-processing of the sensor data was implemented with TwinCAT Condition Monitoring in the control system. The corresponding BML file was then used on a CX51xx embedded PC, which executes the model in real time with the help of the TwinCAT Machine Learning execution instance and outputs the results of the inference for the identification of defective products through an EtherCAT EL2xxx digital output terminal. According to Tianjin FengYu, the transparency of the system is a great advantage of Beckhoff’s control technology and was very beneficial in this case, as it was able to integrate with the existing third-party main controller on the production line without much effort.

The open platform accelerates the development of algorithms

The validation of ML algorithms is usually a long and laborious process due to the necessary tests and the frequent visits that must be made to the end customers. However, with the TwinCAT open software platform, algorithms can be effectively validated without the need for direct access to the machine. The data recorded on a production machine is divided into learning data and validation data before starting the learning of the model. Then only the learning data set is used to memorize the ML algorithm. The validation dataset may initially be used in the learning environment to test the performance of the memorized algorithm on unknown data.

After successfully transferring the data preprocessing and integrating the ML algorithm into TwinCAT, validation based on the production code can be performed. The code is executed on a test system (or on the industrial or embedded PC being used) and the validation data set is transmitted to the TwinCAT real-time environment for use as a virtual data source using the functions of the TwinCAT database server. The same sampling frequency is adopted as with the on-site sensors, which allows an optimal simulation of the scenario at the end customer’s facilities. Similarly, the new data sets collected on the production machine can be used in the test environment to explore a wide variety of situations. Ultimately, the test data serves to validate the entire ML application on the embedded PC, evaluate it and ensure safe operation.

Thanks to the development, verification and validation using the TwinCAT open platform, it is no longer necessary to test ML algorithms in industrial plants and the implementation phase is considerably accelerated. According to Tianjin FengYu, this has contributed to the progress of the joint project of this food company was hardly affected by the covid-19 pandemic.

Multi-tasking and multi-core capabilities ensure real-time machine learning

The ML algorithm is executed on the CX51xx in three steps:

• Acquisition of data from sensors,
• Data preprocessing,
• Execution of ML models for the detection of defective products

On the one hand, it must be taken into account that the inspection of the product requires a high sampling frequency to acquire the data from the sensors during the entire cutting process. On the other hand, a lower execution frequency is necessary to be able to process the acquired data and execute the ML model. However, this apparent contradiction for two real-time sequences in one PLC task can be solved very well thanks to the multitasking and multicore capabilities of TwinCAT, as this ensures both reliable execution of multiple tasks on different processor cores and error-free data exchange between multiple PLC tasks. Another advantage is that these functions can be implemented by a simple configuration and through predefined PLC function blocks that hardly require development. In the described project for the production of instant noodles, the two-step execution of the ML algorithm by using two PLC tasks and two processor cores was reliably implemented.

ML workflow (above), as well as an anomaly detected

and displayed in TwinCAT Scope View (below, marked in red). Photo: Beckhoff.

Machine learning and PC-based control technology minimize effort

Through machine learning concepts and with the help of data mining, production problems can be solved faster and more efficiently, thereby saving R&D costs. Tianjin FengYu’s experiences with the instant noodle production line have shown that TwinCAT Machine Learning is superior to traditional engineering methods in anomaly detection. The open TwinCAT platform made it possible to automate the entire workflow for data acquisition, learning and ML inference. By the end of 2020, the new quality inspection system has already been operating fully automatically on several production lines of the end customer for about four months. The customer’s conclusion: Thanks to Beckhoff’s open solution, it was possible to implement the inspection system without modifying the existing main control system. In addition, defective products are detected quickly and reliably, resulting in an effective reduction of customer complaints.

System integrator Tianjin FengYu pointed out that the use of the machine has become much simpler and more flexible thanks to Beckhoff’s CX51xx products, EtherCAT and TwinCAT I/O. Despite the complex mechanisms of the local production line, the commissioning and maintenance were very simple thanks to the great support of Beckhoff engineers. In addition, PC-based open control technology has solved the problem of packaging quality inspection, also by using ML methods to acquire the data of existing noodle machines, regardless of their brand. Tianjin FengYu is convinced that more and more end users will benefit from TwinCAT Machine Learning in the near future.