Menu
As computer and mechanical automation technologies advance, machine vision-based non-destructive testing technology finds use in a multitude of domains. Non-destructive testing technologies can be used on apple sorting equipment to decrease apple damage while simultaneously increasing sorting efficiency. As a result, the apple sorting machine’s image identification system now incorporates machine vision technology. The automatic classification of apple grades is accomplished by gathering, processing, extracting, and computing the contour features of apple photographs using preset sorting levels. The automatic control system then sorts apples of different grades to designated locations, thus achieving the automation of apple sorting. Tests were run on the sorting machine’s image recognition system to confirm the solution’s viability. The outcomes demonstrate that the sorting machine can effectively classify fruit automatically based on their perimeter, which is important for fruit sorting automation.
The field of agricultural automation and computer vision has advanced significantly in recent years, especially in the area of non-destructive testing methods [1]. One prominent application of this progress is observed in the development of automated fruit sorting systems, which not only enhance efficiency but also mitigate potential damage to the produce. Apple grading detection is a crucial step in the commercial processing of apples, having an immediate effect on apple packing, shipping, storage, and income [2]. Traditional methods involve weighing apples and then grading them, using manual and mechanical devices, resulting in low sorting efficiency and potential damage to apples. Moreover, these approaches cannot provide a realistic evaluation of the colour and surface flaws of apples. Machine vision-based non-destructive testing technology has been used in a number of industries recently. Apple sorting devices that use machine vision into their design can successfully increase apple sorting efficiency and accomplish non-destructive apple testing [3].
The pivotal process of grading apples holds immense importance in the commercial handling of the fruit, directly impacting packaging, transportation, storage, and sales revenue. Traditionally, methods involved weighing apples and subsequently categorizing them. However, the utilization of manual labor and mechanical devices in these processes not only results in low sorting efficiency but also poses a risk of damage to the apples [4]. Furthermore, these methods fail to provide a comprehensive assessment of color and surface defects, limiting their effectiveness.
The integration of machine vision-based non-destructive testing techniques presents a transformative opportunity to address these challenges. By incorporating machine vision into the design of apple sorting machines, it becomes possible to significantly improve sorting efficiency and achieve non-destructive testing of apples. This marks a departure from traditional methods and opens avenues for a more comprehensive and nuanced evaluation of apples based on color, shape, and surface characteristics [5, 6, 7].
Central to the success of the image recognition system is the application of non-destructive testing techniques. These techniques play a critical role in the detailed analysis of apple images, enabling the system to identify color variations and surface defects accurately. By avoiding physical contact and damage to the apples during the sorting process, non-destructive testing ensures that the produce retains its quality and market value. The significance of non-destructive testing extends beyond sorting efficiency to the preservation of apple quality. The ability to identify and categorize apples without causing harm not only enhances the overall efficiency of the sorting process but also contributes to reducing post-sorting losses and maintaining the integrity of the fruit [8]. While the integration of machine vision and non-destructive testing techniques presents a promising avenue for automating apple sorting, several challenges persist in the current research landscape.
Apples exhibit a wide range of characteristics, including varying colors, shapes, and surface features. The complexity of these characteristics poses a challenge in developing a universal sorting criteria that accommodates the diversity observed in different apple varieties [9]. The efficiency of automated sorting systems is contingent upon real-time image processing capabilities. Achieving rapid and accurate image analysis, especially in high-throughput scenarios, remains a formidable challenge that requires continuous research and technological advancements [10]. To enhance the accuracy of sorting, integrating data from multiple sensors, such as color sensors and surface defect detectors, is essential. However, the seamless integration and synchronization of data from diverse sources present technical challenges that demand innovative solutions.
Addressing these challenges requires a concerted research effort, involving advancements in machine learning algorithms, sensor technologies, and system integration methodologies [11]. The ultimate goal is to develop a robust and versatile automated sorting system that can revolutionize the fruit industry by enhancing efficiency, reducing losses, and ensuring the delivery of high-quality produce to consumers.
This paper delves into the integration of machine vision technology into the design of an apple sorting machine, with a focus on image recognition systems. The objective is to explore the feasibility and efficacy of employing non-destructive testing techniques to not only elevate the sorting efficiency of apples but also to evaluate them without causing harm.
The agricultural machinery image recognition system constitutes the core of machine vision. Visual recognition typically involves the initial acquisition of corresponding images by visual sensors. Subsequently, these images undergo processing by a machine vision system. After completing processes like feature extraction and picture segmentation, the system determines the image’s final content [9]. The process flow of agricultural image recognition systems advances from basic image collecting and preprocessing to intermediate processing layers that are capable of segmenting images and extracting features. Finally, it culminates in high-level processing that utilizes various algorithms to achieve plane recognition of the image.
The collection and acquisition of images rely on visual sensors. After obtaining the images, preprocessing steps are necessary, including converting RGB values to HIS mode, enhancing defect features, and optimizing image resolution. Image preprocessing primarily involves four aspects. Firstly, grayscale processing transforms RGB tricolors, converting the acquired color image into a grayscale image, and reducing the 24-bit image to an 8-bit image [12]. Secondly, uniformization processing is applied to standardize the size of crop images, allowing for more characteristic points for subsequent recognition through uniform scaling. Subsequently, image enhancement is performed to improve clarity and strengthen recognition effects. Techniques such as dilation and erosion can be used to enhance image resolution or filter out interference in the image. Finally, for convenient image processing, binarization can be applied using the Niblack algorithm, determining the central coordinates on crop images and achieving binarization through calculating pixel point variances and means [11, 13].
To divide an image into several important pieces is the aim of image segmentation. Three primary segmentation techniques-based on image edges, image thresholds, and image regions-are available. Using edge detection algorithms, one of the segmentation techniques based on picture edges is to compute the grey value, colour, and texture of the image region with irregular edges, then utilise these edge contours to identify the image region [14].
Threshold-based image segmentation relies on determining a threshold based on the emission or absorption of light on the object’s surface. There are two main approaches to threshold-based segmentation. The first is manual segmentation, a fundamental method that uses histogram techniques to identify the optimal threshold and isolate the segmented target from the background. Manual segmentation is influenced by subjective factors, requiring repeated monitoring and adjustments to achieve satisfactory segmentation results. In agricultural production, crops are significantly affected by lighting conditions, necessitating threshold adjustments when changes occur [15, 16]. Frequent adjustments, however, may impact the effectiveness of threshold segmentation.
The second approach is automatic threshold segmentation, which eliminates the need for human intervention. In automatic thresholding, the threshold is automatically incorporated into the histogram information during the target segmentation process. Based on pre-set parameters, the segmentation operation is performed. However, threshold segmentation encounters challenges when dealing with images lacking distinct peaks and valleys, leading to difficulties in calculating effective thresholds and resulting in incomplete region segmentation.
Region-based segmentation is a method of segmentation based on combining representative pixel content within a region, resulting in a segmentation approach that performs less effectively in noise identification. This method is mostly applied to identify crops with distinct characteristics on the crop surface, focusing on recognition in areas such as the overall crop, roots, and defects. The segmentation results are illustrated in Figure 1.
One fundamental skill in art and painting is the ability to interpret images, and correctly recognizing and interpreting paintings allows one to grasp the intention and deeper meaning of the artwork. The visual tendencies in contemporary art are closely related to the rapid development of visual culture. Currently, museums house a vast number of paintings, and different aesthetic approaches yield varying visual communication effects. The study of visual symbols in artworks is a topic worthy of in-depth exploration. If the methods of semiotics were introduced into the study of art and painting, effective identification methods for artworks could be developed, ultimately enhancing the ability to interpret artistic works [17].
The efficiency of sorting apples would be greatly increased by creating an apple sorting machine using machine vision based on the interpretation of paintings and other visual media [18]. This approach, utilizing non-destructive testing, can effectively reduce damage to apples during the sorting process. Apples that are sorted according to size will work better for processing, processing, and storage. Apart from that, apples with a form grade will be more marketable. Figure 2 shows the steps involved in sorting apples using machine vision technology.
Taking pictures of the apples is the initial stage in the apple sorting machine. Subsequently, machine vision systems are employed to process these images. Finally, based on the grading standards, the apples’ grades are determined. Once the apple grades are established, an automatic control system is utilized to sort apples of different grades into designated devices, thus achieving non-destructive testing and automated classification of apples [19, 20].
Manual sorting would lead to significant labour waste and lengthier sorting cycles in the apple sorting process. Mechanical sorting, while efficient, poses a risk of damaging the fruit [21]. Non-destructive testing, however, offers the advantage of improving sorting efficiency without causing harm to the fruit. During non-destructive testing of fruit quality, it is essential to assess factors such as apple size, surface defects, color, and shape, which serve as the basis for apple grading. Figure 3 depicts the sorting machine system’s schematic diagram.
During non-destructive sorting, the primary process involves capturing intact images of apples. Subsequently, image processing, feature extraction, and contour analysis are employed to accomplish quality grading. The assessment of grades is based on factors such as apple size, color, and surface defects. The primary method used in apple size grading to calculate the diameter and circumference of the apple is to extract the shape contour from the image. Apple color grading involves converting RGB values to HIS mode, obtaining the hue histogram, and evaluating color characteristics. Apple surface flaws are evaluated by extracting visual characteristics and applying grayscale or image model techniques. As shown in Figure 4, there are six primary processes in the post-image acquisition processing: image grayscale processing, image normalisation, image enhancement, image binarization, and image feature extraction [22, 23].
The image’s basic properties are revealed by image processing, and these can be used to determine the apple image’s quality based on its size and surface characteristics. The following are the steps involved in the image processing process:
The primary purpose of image grayscale processing is to gather colour images of apples. The weighted average method is used in this procedure to convert 24-bit images to 8-bit images. The formula is:
\[\label{e1}\tag{1} {{C = xR + yG + zB, x + y + z = 1}},\] where \(C\) is the colour after conversion, \(x\), \(y\), and \(z\) are the three most fundamental colour coefficients, and \(R\), \(G\), and \(B\) are the three primary colours.
The apple image that needs to be detected can be normalised during image processing to give it a consistent size. Assuming that the scaling ratios in the X- and Y-axes directions are \({f_x}\) and \({f_y}\), respectively, normalisation takes the form of uniform image scaling. The expression for the point (\({x_1},{y_1}\)) in the scaled image \(g(x,y)\) corresponding to a point (\({x_0},{y_0}\)) in the original apple image to be recognised is \(f(x,y)\).
\[\label{e2}\tag{2} {x_1} = {f_x} \times {x_0},{y_1} = {f_y} \times {y_0}.\]
As feature points are rearranged within the range of [0, 255], the normalisation procedure can increase the accuracy with which the apple image is detected. The normalisation formula for the grey scale is:
\[\label{e21}\tag{3} p(i,j) = {{p(i,j) – {G_1}} \over {{G_2} – {G_1}}} \times 255.\]
Additional image processing can be used to increase the apple photos’ identification efficacy. This may involve filtering operations to eliminate interference from the image, as well as operations such as dilation and erosion to enhance the image’s resolution, thereby improving the accuracy of apple grading [18].
For convenient image processing and enhanced recognition efficiency, binarization of the image is performed by setting the grayscale values to 0 or 255. Niblack algorithm can be used to achieve binarization [19]. This algorithm determines the central coordinates \((x,y)\) on the apple image to be identified and calculates pixel variance and mean within the surrounding neighborhood of size \((r*r)\) facilitating the binarization process.
\[\label{e3}\tag{4} T(x,y) = m(x,y) + k \times s(x,y),\]
\[\label{e4}\tag{5} m(x,y) = {1 \over {{r^2}}}\sum\limits_{i = x – {r \over 2}}^{x + {r \over 2}} {\sum\limits_{j = y – {r \over 2}}^{y + {r \over 2}} f } (i,j),\]
\[\label{e5}\tag{6} {\rm{s}}(x,y) = \sqrt {{1 \over {{r^2}}}\sum\limits_{i = x – {r \over 2}}^{x + {r \over 2}} {\sum\limits_{j = y – {r \over 2}}^{y + {r \over 2}} {[f} } (i,j) – m(x,y){]^2}} ,\] where \({\rm{T}}(x,y)\) is the threshold value of pixel \((x,y)\); \(m(x,y)\) is the mean value of pixel in \((r*r)\) neighbourhood; \(s(x,y)\) is the standard deviation of pixel in \((r*r)\) neighbourhood; \(k\) is the correction coefficient, which can be binarised to set pixel grey scale to 0 or 255.
Intersection points and endpoints are crucial to the matching process in the recovered texture skeleton picture of the apple image [21]. The specific algorithm for extracting feature points is assuming that the \(3*3\) template region is:
\[\begin{gathered} \begin{matrix} {{P_1}} & {{P_2}} & {{P_3}} \\ {{P_g}} & {{P_0}} & {{P_4}} \\ {{P_7}} & {{P_6}} & {{P_5}} \\ \end{matrix} \quad \end{gathered}\tag{7}\] where \({P_1}\) represents the centroid of the region and its formula can be expressed as follows:
\[\label{e7}\tag{8} N_{\text {trans }}=\sum_{i=1}^g\left|p_{i+1}-p_i\right|,\] where \(P_1=P_9\) in this case. Once the image’s features have been extracted, the apples can be ranked according to the diameter, area, and edge features of the image. After classifying each grade into pertinent standards, the apples are sorted based on the outcomes of the image recognition process.
The most important part of the apple sorter is the picture recognition system, which is based on the PLC control system’s recognition results for automatic apple sorting [14]. This time, as seen in Figure 5, we primarily test the sorter’s image recognition system using apples as the test object [23].
The feasibility of the recognition system was assessed by selecting apples as the identification objects, focusing on four apples with clearly distinct features for sorting. Initially, one of the apples underwent image processing, and the result after gray scale processing is depicted in Figure 6.
Apple diameter and circumference can be extracted based on contour data. This time, as illustrated in Figure 7, the primary method of classification for apples is the circumference of their outlines. Four apples need to be categorised.
The perimeter of the contours created after the apple photographs were processed was the main grading factor for the four apples that were graded. The grading outcomes, which are displayed in Table 1, were produced by processing the four apple photos and counting the perimeters.
| Apple signal to be tested | Measured circumference/cm | Rating |
|---|---|---|
| 1 | 225 | 3 |
| 2 | 315 | 3 |
| 3 | 350 | 2 |
| 4 | 420 | 1 |
Table 1 clearly shows that the perimeter data of apples can be obtained by effectively using the specified image processing approach. Through empirical grade assessment, the quality grade of apples can ultimately be determined. In the actual operation of the apple sorting machine, after obtaining the assessed grades, To categorise various apples and transport them to separate sites for packaging and processing, the control system must be used.
Figure 8 shows the detection effect and classification detection effect of oranges. To achieve a better recognition effect, a binarized image is used as an auxiliary. Binarized images are frequently used in image processing and computer vision due to their simplicity and utility. They help to simplify data, extract features, reduce noise, segment, and support thresholding. Binarization may assist in data compression and preprocessing for complex computer vision tasks. However, it may result in some information loss, making the selection of the binarization method critical, and its application is dependent on task-specific requirements. After using the background subtraction method to remove the background, according to the RBG and HIS (Hue, Saturation, Intensity) conditions, determine the area where the orange appears, and select it immediately. Using the background subtraction method, a computer vision algorithm detects objects in video content [8].
The algorithm compares objects to the image’s background and foreground. During the no-object-frame comparison, the objective is to isolate foreground objects. The background subtraction method is an effective technique for detecting moving objects in a series of frames from static cameras. The method detects foreground objects by comparing incoming frames to a model of the scene’s background. The background subtraction method is generally fast and requires less power for computation. Figure 7 shows the original greyed-out image on the top left, the RPN output effect on the top right, the RGB mapping map on the bottom left, and the output map of the entire model—that is, the discrimination of the fruit levels—on the bottom right. Different colours correspond to different levels.As can be seen from Figure 7, compared with the traditional L/C (brightness/contrast) method, the orange detection effect of the algorithm in this paper is better.
Therefore, after comprehensive consideration, the average hue is used to classify the red-orange colour, in which the average hue of the superior grade is less than or equal to 26, and the average hue of the first- and second-class grades is 18 and 13, respectively. Therefore, the specific grading parameters of the tone mean value among each grade are shown in Table 2
| Superior products | First class | Second class | Substandard |
|---|---|---|---|
| Mean value \(\le\) 26 | 13\(<\)mean value \(\le\) 18 | 8\(<\)mean value \(\le\) 13 | mean value<8 |
Machine vision technology has been included into the apple sorting system’s design to increase apple sorting efficiency and enable non-destructive testing. Through processes such as grayscale treatment, binarization, enhancement, and feature extraction on apple images, automated grading of apples has been achieved based on established grading standards. The apple sorting machine’s image recognition technology was tested in order to confirm the solution’s viability. The findings show that the sorting image recognition system can effectively classify grades and conduct non-destructive testing by sorting apples according to their perimeter features. This holds great significance as a point of reference for the advancement of automated fruit sorting mechanisms.
This work no funding supported.
The author declares no conflict of interests.