Target tracking is a fundamental task in the field of computer vision, which has a wide range of applications in real-life video image signal processing. This paper proposes target tracking optimization technique based on the principle of multi-scale convolutional neural network and multitarget tracking algorithm. The basic structure is designed using VGG16 network, the ROI align method is used to reduce the number of features for feature fusion, and the improved Hungarian algorithm is adopted to associate the fused features and obtain the target tracking results. In the tracking performance experiments, the target tracking optimization technique in this paper is more discriminative in terms of extracted features, and also has higher tracking results under challenging factors such as background clustering (BC), scale variation (SV), and out-of-view (OV). As for the target tracking experiments on mobile network video images, the average tracking accuracy and average tracking success rate of this paper’s method are 97.89% and 96.02%, which are better than DS_v2 and FFT16, and the average error between the target tracking results and the target’s actual motion trajectory is 4.12mm, while possessing the smallest error amplitude.