With the rapid development of China’s economic level and the significant improvement of people’s living standard, the quality issue of peaches has become more and more strict. In this paper, based on deep learning algorithm, we propose the recognition method of peach fruit color, size and fruit shape features, combined with near-infrared spectroscopy detection technology, to quantify the peach fruit components and discriminate its maturity. Differential algorithm, standard normal transform, and multiple scattering correction are applied to pre-process peach fruit data. Based on M-YOLOv5s target detection framework, spectral analysis and image characterization techniques were used to jointly detect the degree of peach fruit disease. The distribution of peach fruit quality parameters was investigated, and the test results showed that 39.19% of the samples with measured values of fruit size were concentrated at 1.60-6.40 cm, and 61.79% of the samples with predicted values were concentrated at 2.50-7.50 cm, which was located at around the mean value of 4.763 cm.The classification accuracies of the information modeling set and validation set for the combination of the spectral analysis and image eigenvalue detection techniques were 91.439% and 88.487%, respectively, and the combined use of the two techniques had a high accuracy for the differentiation of diseased peach fruits. Based on the experimental results, the application of spectral detection technology in food freshness detection as well as pesticide residues and illegal additives is explored.