Logo BSU

Please use this identifier to cite or link to this item: https://elib.bsu.by/handle/123456789/261155
Title: Chinese alt text writing based on deep learning
Authors: Xie, J.
Li, R.
Lv, S.
Wang, Y.
Wang, Q.
Vorotnitsky, Y.I.
Keywords: ЭБ БГУ::ЕСТЕСТВЕННЫЕ И ТОЧНЫЕ НАУКИ::Физика
Issue Date: 2019
Publisher: International Information and Engineering Technology Association
Citation: Trait Signal 2019;36(2):161-170.
Abstract: To generate coherent and readable Chinese image caption, this paper designs an Chinese image captioning model based on Inception-ResNet-v2, a deep convolutional neural network (DCNN) based on residual blocks, and the double-layer gated recurrent unit (GRU) network. The proposed model extracts the features from the original image with the Inception-ResNetv2. To overcome the stochasticity of random text encoding, the neural network modelling was performed to create word embedding features for sparse word codes. Next, the extracted deeply convoluted image features were mapped to the word embedding feature space. Finally, the double-layer GRU network was trained with the image features and word embedding features, yielding the Chinese image captioning model. The proposed model was proved through experiment as capable of generating Chinese text for images. In addition, our model performed excellently in the objective evaluation with indices like Perplexity, BLEU and ROUGE-L. Specifically, the Perplexity score of our model was 4.922, the BLEU-1, BLEU-2, BLEU-3 and BLEU-4 results were 0.674, 0.533, 0.416 and 0.330, respectively, and the ROUGE-L was 0.635. All of these were better than the results of the other models like the natural image captioning (NIC) model.
URI: https://elib.bsu.by/handle/123456789/261155
Scopus: 10.18280/ts.360206
metadata.dc.identifier.scopus: 85071915946
Sponsorship: Project Supported by Natural Science Foundation of Heilongjiang Province (LH2019E058); University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province (UNPYSCT-2017091); Fundamental Research Fundation for Universities of Heilongjiang Province (LGYC2018JC027).
Appears in Collections:Статьи факультета радиофизики и компьютерных технологий

Files in This Item:
File Description SizeFormat 
36.02_06.pdf2,16 MBAdobe PDFView/Open


PlumX

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.