Rhythm matching of music and dance is an important research area in cross-modal analysis. In this paper, a music and dance rhythm matching algorithm based on time series analysis is proposed to extract the time series features of music and dance, and a genetic algorithm is used to determine the correspondence between music and dance movements to reflect the degree of correlation between changes in music and dance rhythm movements. In order to improve the matching and smoothing degree between the dance movement time series and the music time series, a constraint-based dynamic programming algorithm is introduced. The experimental results show that the model performs well in the matching degree and matching efficiency enhancement between dance movement time series and music time series, and its matching efficiency is 2-3 times of the traditional method. It shows high practicality in dance choreography and music matching, and can match any music clip with smooth and beautiful dance movements. The research in this paper provides new technical means for dance choreography and music matching, which will further optimize the transition harmony between music time series and dance movement time series.