E-commerce classroom teaching is an important means to improve the quality and teaching effect of e-commerce teaching, and effective interaction in teaching is an important carrier of e-commerce teaching classroom activities. This study combines pan-reinforcement learning and reinforcement Q learning algorithms to recognize and analyze speech data in e-commerce teaching classroom, and uses head posture estimation algorithm to recognize interactive behaviors in e-commerce teaching classroom video, and combines the video and speech interaction data to get the e-commerce teaching interactive behavior recognition model. The model is then equipped with web application technology to design a visual analysis system for e-commerce teaching interaction, and the optimization strategy of e-commerce teaching interaction is realized with the assistance of this system. The results of the study show that the interactive behavior recognition model proposed in this paper can accurately identify the interactive behavior of teachers and students in each course of e-commerce teaching. It is also found that after the implementation of interaction optimization strategy in college e-commerce teaching classroom, the frequency of effective interaction behaviors of teachers and students increases from 351 to 391 times, and the meaningless classroom silence time is reduced. And the learners’ cognition of knowledge is also improved under the influence of the improvement of the effect of interactive behavior. The visual analysis system of teaching interaction proposed in this paper based on reinforcement learning algorithm is of great significance for optimizing the effective interactive behaviors of teachers and students in e-commerce teaching and improving the degree of students’ knowledge cognition.