برچسب‌زنی مقیاس‌پذیر تصاویر با خلاصه‌سازی نمونه‌ها به نماینده‌های برچسب‌دار

با افزایش روز‌افزون تصاویر، اندیس‌گذاری و جستجوی سریع آنها در پایگاه داده‌های بزرگ، یک امر ضروری است. یکی از راه‌کارهای موثر، نسبت‌دادن یک یا چند برچسب به هر تصویر با هدف توصیف محتوای درون آن است. با وجود کارایی روش‌های خودکار برچسب‌زنی، یکی از چالش‌های اساسی آنها مقیاس‌پذیری با افزایش تصاویر پایگاه داده است. در این مقاله، با هدف حل این چالش، ابتدا براساس توصیف‌گر بصری تصاویر که از شبکه‌های یادگیری عمیق استخراج می‌شوند،‌ نمایندگان مناسبی به‌دست می‌آیند. سپس، با استفاده از رویه انتشار برچسب بر روی گراف، برچسب ‌های معنایی از تصاویر آموزشی به نمایندگان منتشر می‌شوند. با این راه‌کار، به یک مجموعه نمایندگان برچسب ‌دار دست خواهیم یافت که می‌توان عمل برچسب‌زنی هر تصویر آزمون را بر اساس این نمایندگان انجام داد. برای برچسب‌زنی، یک رویکرد مبتنی بر آستانه‌گذاری وفقی پیشنهاد شده است. با روش پیشنهادی، می‌توان اندازه مجموعه‌داده آموزشی را به 22.6 درصد اندازه اولیه کاهش داد که منجر به تسریع حداقل 4.2 برابری زمان برچسب‌زنی خواهد شد. همچنین، کارایی برچسب زنی بر روی مجموعه‌داده‌ های مختلف برحسب سه معیار دقت، یادآوری و f1 در حد مطلوبی حفظ شده است.

Scalable Image Annotation by Summarizing Training Samples into Labeled Prototypes

By increasing the number of images, it is essential to provide fast search methods and intelligent filtering of images. To handle images in large datasets, some relevant tags are assigned to each image to for describing its content. Automatic Image Annotation (AIA) aims to automatically assign a group of keywords to an image based on visual content of the image. AIA frameworks have two main stages; Feature Extraction and Tag Assignment which are both important in order to reach a proper performance. In the first stage of our proposed method, we utilize deep models to obtain a visual representation of images. We apply different pretrained architectures of Convolutional Neural Networks (CNN) to the input image including Vgg16, Dense169, and ResNet 101. After passing the image through the layers of CNN, we obtain a single feature vector from the layer before the last layer, resulting into a rich representation for the visual content of the image. One advantage of deep feature extractor is that it substitutes a single feature vector instead of multiple feature vectors and thus, there is no need for combining multiple features. In the second stage, some tags are assigned from training images to a test image which is called ldquo;Tag Assignment rdquo;. Our approach for image annotation belongs to the searchbased methods which have high performance in spite of simple structure. Although it is even more timeconsuming due to its method of comparing the test image to every training in order to find similar images. Despite the efficiency of automatic Image annotation methods, it is challenging to provide a scalable method for largescale datasets. In this paper, to solve this challenge, we propose a novel approach to summarize training database (images and their relevant tags) into a small number of prototypes. To this end, we apply a clustering algorithm on the visual descriptors of training images to extract the visual part of prototypes. Since the number of clusters is much smaller than the number of images, a good level of summarization will be achieved using our approach. In the next step, we extract the labels of prototypes based on the labels of input images in the dataset. because of this, semantic labels are propagated from training images to the prototypes using a label propagation process on a graph. In this graph, there is one node for each input image and one node for each prototypes. This means that we have a graph with union of input images and prototypes. Then, to extract the edges of graph, the visual feature of each node on graph is coded using other nodes to obtain its Knearest neighbors. This goal is achieved by using Localityconstraints Linear Coding algorithm. After construction the above graph, a label propagation algorithm is applied on the graph to extract the labels of prototypes. Based on this approach, we achieve a set of labeled prototypes which can be used for annotating every test image. To assign tags for an input image, we propose an adaptive thresholding method that finds the labels of a new image using a linear interpolation from the labels of learned prototypes. The proposed method can reduce the size of a training dataset to 22.6% of its original size. This issue will considerably reduce the annotation time such that, compared to the stateoftheart searchbased methods such as 2PKNN, the proposed method is at least 4.2 times faster than 2PKNN, while the performance of annotation process in terms of Precision, Recall and F1 will be maintained on different datasets.