Title : Localisation and classification of broccoli heads
Abstract:
Fresh market broccoli is traditionally harvested selectively, in a labour-intensive, manual process. Only broccoli heads of the right size and right quality are harvested. The size assessment is done visually by the harvesters. As a first step trying to automate the selective broccoli harvesting process, the presentation describes how we have selected and used a depth camera and neural network to localise and measure the size of the broccoli heads.
The first step is to record in the field a series of images, annotate them and then to use a deep learning algorithm to detect broccoli heads in an image. Using a customized data set, based on the existing University of Lincoln broccoli dataset, we trained and compared the performance of two modern deep learning algorithms (YOLO V4 and YOLACT).
In a comparison between five different depth cameras, we selected the Intel Realsense stereo camera D435I as the most suitable for our purpose to perceive a broccoli in the field. After training on a custom data set, YOLO was found to have an Average Precision (AP) score of 70.48%, and a run-time performance of object detection within 32 ms. YOLACT was found to be better at accurately identifying and locating the broccoli head (AP score of 73.82%), however, YOLACT performs slower with a detection time of 51 ms.
A user interface was developed to display all detection information in real time. With the help of a class diagram, we will present an object-oriented overview of the proof-of-concept system and a sequence diagram to show how the objects work together.
With a subsequent field experiment, the recognition algorithm was tested on 50 new broccoli heads, which were scanned and measured to establish the ground truth in diameter and depth. During the field experiment it was shown that the diameter measurement is accurate to 9.36 mm and the depth measurement to 8.98 mm. In further work towards a machine that can selectively harvest broccoli, we will study how the vision system can be integrated with a harvesting arm to harvest broccoli in the field.
