Automated image captioning with a ResNet-LSTM encoder-decoder for distributed surveillance scenes
Abstract
Automatic image captioning is a fundamental artificial intelligence task at the intersection of computer vision and natural language processing. This paper presents a generative encoder-decoder model for automatic caption generation from images of military exercises. The model employs an ImageNet-pre-trained 152-layer Residual Network (ResNet-152) encoder to extract visual representations and a long short-term memory (LSTM) decoder to generate corresponding textual descriptions. The original model was trained on the Microsoft Common Objects in Context (MS COCO) dataset and subsequently fine-tuned on a domain-specific collection of military-exercise images. Five pseudo-reference captions per image were generated automatically using Florence-2 and used to supervise domain adaptation. Experimental evaluation was performed on a held-out military-domain test set containing 25 images. The fine-tuned model achieved Bilingual Evaluation Understudy scores of 70.87, 49.67, 33.61, and 21.94 for BLEU-1, BLEU-2, BLEU-3, and BLEU-4, respectively, together with a Recall-Oriented Understudy for Gisting Evaluation based on the longest common subsequence (ROUGE-L) score of 45.52. On the same military test set, the original MS COCO checkpoint achieved 52.00, 33.10, 23.00, 15.80, and 37.05, respectively. These results indicate that domain-specific fine-tuning improves agreement with the available military-domain reference captions. However, because the test set is small and its reference captions were generated automatically rather than independently written by domain experts, the reported results should be interpreted as preliminary evidence rather than a definitive estimate of performance on the broader military-image domain. The proposed architecture may provide a basis for future investigation of onboard semantic reporting and bandwidth-efficient communication on unmanned platforms, while its practical contribution can be further experimentally validated through the evaluation of execution speed, energy consumption, and communication savings on a real unmanned aerial vehicle.