The traditional Japanese language teaching mode in colleges and universities has been unable to meet the requirements of Japanese language majors in various industries, and colleges and universities should use certain methods to carry out a reasonable reform of the teaching mode of Japanese language majors. Firstly, an error correction model based on UniLM model framework is proposed, using natural language processing technology to extract features, and fine-tuning training for the model after initialization. The model framework based on UniLM+CRF and the seq2seq model framework based on UniLM are built to realize the Japanese text grammar error annotation task and the Japanese text grammar error correction task respectively. Then a multi-task learning error correction method is proposed to integrate the grammar error labeling task and the grammar error correction task, so as to improve the accuracy of the error correction model. Finally, a specific Japanese grammar error correction system architecture is designed, a Japanese language knowledge base is established, and utterance synthesis rules are formulated to realize the innovative teaching of Japanese language in colleges and universities. The average grammatical error correction precision, recall, and F1 value of the model in this paper reached a good level in the students’ Japanese composition correction. The error between the average score of teacher correction and the average score of model correction is only 0.19 points, and the related experiments show that the innovative teaching model studied in this paper can effectively improve students’ mastery of Japanese syntactic ability. The above data illustrate that the Japanese error correction system based on UmiLM framework designed in this paper has certain application value and can realize the innovation of Japanese language teaching mode.