Hybrid SOM based cross-modal retrieval exploiting Hebbian learning.

Abstract

Lately, cross-modal retrieval has attained plenty of attention due to enormous multi-modal data generation every day in the form of audio, video, image, and text. One vital requirement of cross-modal retrieval is to reduce the heterogeneity gap among various modalities so that one modality's results can be efficiently retrieved from the other. So, a novel unsupervised cross-modal retrieval framework based on associative learning has been proposed in this paper where two traditional SOMs are trained separately for images and collateral text and then they are associated together using the Hebbian learning network to facilitate the cross-modal retrieval process. Experimental outcomes on a popular Wikipedia dataset demonstrate that the presented technique outshines various existing state-of-the-art approaches.