Prediction of legal decisions using machine learning and artificial intelligence techniques has gradually become an important part of smart court technology. In addition the crime prediction and law recommendation also face the problem of easily confusing crimes. In order to solve these problems, this paper unites multi-task learning models and proposes a model fusion legal verdict prediction model. An attention neural network fusing Transformer Encoder and DPCNN encodes the key semantic information in the case description. The TF-IDF algorithm and TextRank algorithm are applied to extract the keywords of the charge, and the forward propagation network is used as a classifier to constitute a multi-task learning legal verdict prediction model. Using 9 CAIL2018 legal datasets as experimental data, the metrics performance of the multi-task learning legal judgment prediction model proposed in this paper is measured on three subtasks (offense prediction, legal provision prediction, and punishment duration prediction) in LJP. Combining real case information for legal verdict prediction as well as charge differentiation. The verdict prediction results on the CAILBig-Multi dataset show that the mean MP value of the comparison algorithms is 82.925% in the charge prediction. And the MP index of the charge prediction of the multitask learning legal verdict prediction model proposed in this paper is 89.13%, which is significantly higher than the mean value of the comparison algorithms. And the multitask learning model incorporating the keyword information of charges in case analysis can effectively solve the problem of confusing charges.