Based on the excellent achievements of deep learning technology in image recognition in several fields, then the convolutional neural network is expected to play its superior performance in the recognition of micro features of Chinese herbal medicines. The object of analysis in this paper is the microscopic feature images of Chinese herbal medicines, and the residual network will be improved in order to improve the recognition accuracy of the network model on the microscopic images of herbs. On the basis of the traditional CNN network model, CBAM based on mixed domain attention mechanism is added, and residual connection is introduced to increase the transfer of gradient and information flow, preserve image feature data and reduce feature loss. Improved from the traditional residual structure to moving inverted bottleneck convolution (MBConv), the SE module and SAM module are added to the MBConv stage respectively to optimize the feature extraction performance and improve the accuracy of the classification of microscopic features of traditional Chinese medicine. The effect of the addition of the attention mechanism on the network model is analyzed, and the network model is examined in conjunction with the constructed dataset of powdered microscopic images of commonly used Chinese medicinal herbs.The average accuracy of the Attention-TCM-Net network model on the test set reaches 96.47%, which is an improvement of 0.85 percentage points compared with that of the ResNet34 network, and meanwhile, the convergence of the model is significantly better than other models.