Deep Visual-Semantic embedding for Multi-label image Classi cation

Abstract

The classi cation is one of Machine Learning techniques, that aims to categorize data into one or more prede ned classes (labels). When the data is a set of images, we talk about image classi cation, which done basing on their visual content. The image clas- si cation can be categorized into two categories which are single-label classi cation and multi-label classi cation.The multi-label image classi cation (MLC) aims to rstly learn from training set of images, where each one can belong to multiple classes and so after be able to predict more than one class label simultaneously for a new tested image.In this thesis, we present a multi-label image classi cation method that contains three modules: word embedding module, visual embedding module and transformation mod- ule. The rst module consist of a word embedding model that maps words (labels) into a semantic embedding space of d-dimension, where each semantic related labels are close to each other. The second one, is a CNN framework that learn a transformation ma- trix A with dimensions k d from an input image I and the embedding vectors of its corresponding labels. The last module receive results of the two previous modules, to transform labels from d-dimensional space to a k-dimensional visual-semantic embedding space, which separated the relevant and irrelevant labels to the image I.