Starting from the essence of dynamic programming algorithms, the terminology in dynamic programming algorithms, the applicability conditions of the algorithms, and common sub-problem models are summarized. The Belman optimal algorithm is used to split the multilevel problems in dynamic planning into simple single-level problems and solve them one by one, combined with the function approximation structure to approximate the performance index function, to construct the adaptive dynamic planning algorithm, and to apply it in the resource balancing optimization of integrated teaching. The results show that the adaptive dynamic programming algorithm has better resource balancing effect than other algorithms, and the number of convergence and running time are reduced by 6-53 times and 48.92-90.34 seconds respectively. The introduction of the adaptive dynamic programming algorithm improved the resource balancing accuracy of university teaching and learning management by 4.0%-17.4% in each subject group. As the number of resources increased, the time consumption required when balancing resources decreased by 50%-83.33% for test groups 3, 4 and 5, and the efficiency of the test improved by 75%-100%. This shows that the algorithm proposed in this paper is effective when dealing with balancing online and offline teaching resources in higher education.