Chinese alt text writing based on deep learning

Abstract

To generate coherent and readable Chinese image caption, this paper designs an Chinese image captioning model based on Inception-ResNet-v2, a deep convolutional neural network (DCNN) based on residual blocks, and the double-layer gated recurrent unit (GRU) network. The proposed model extracts the features from the original image with the Inception-ResNetv2. To overcome the stochasticity of random text encoding, the neural network modelling was performed to create word embedding features for sparse word codes. Next, the extracted deeply convoluted image features were mapped to the word embedding feature space. Finally, the double-layer GRU network was trained with the image features and word embedding features, yielding the Chinese image captioning model. The proposed model was proved through experiment as capable of generating Chinese text for images. In addition, our model performed excellently in the objective evaluation with indices like Perplexity, BLEU and ROUGE-L. Specifically, the Perplexity score of our model was 4.922, the BLEU-1, BLEU-2, BLEU-3 and BLEU-4 results were 0.674, 0.533, 0.416 and 0.330, respectively, and the ROUGE-L was 0.635. All of these were better than the results of the other models like the natural image captioning (NIC) model.