The rapid development of artiϐicial intelligence algorithms makes them play an important auxiliary role in college English education. This paper deeply analyzes the application of artiϐicial intelligence algorithms in English education in colleges and universities, and constructs a method of analyzing students’ behavior in college and university English classrooms with computer vision as the algorithmic representative, which assists teachers to understand the state of students in the whole classroom.YOLOv7 network carries out multi-target detection in the classroom and improves the network for the deϐiciencies in the classroom environment. The improved K-means algorithm is then introduced to improve the DeepSORT multi-target tracking algorithm. Obtain the surveillance video data in the English classroom of Q college and build the dataset by itself, and design different experiments to verify the effectiveness of this paper’s algorithm respectively. Finally, the classroom behavior analysis method based on computer vision in this paper is applied to teaching practice to explore the practical application effect of the method. The results show that the improved method of this paper can signiϐicantly improve the performance of the target detection and tracking model, and the application of this paper’s method to the classroom time can accurately capture the classroom state of different students, and assist teachers in formulating different teaching strategies according to different classroom stages.