Im2Vec : A Language Model Approach to Understanding Image Classification
Date:
Im2Vec is a language model approach to visualizing and understanding images in image classification task. In this work we developed a new way to understanding images in image classification task via image embeddings and visualized with PCA. We have encoded relationship among different classes in CIFAR-10 dataset utilizing Word2Vec approach in word embedding.
Abstract
In this work we present Im2Vec, a novel approach to understanding the images in image classification task. Whereas conventional methods for weight visualization have tried to generate images from the weights of trained models, we sought to derive a numerical representation image embedding vectors. The idea is adopted from Word2Vec, a method to encode relationship among words into word embedding vector. Im2Vec uses the pixel values of each image as the context for its classification, similar to the context words in skip-gram model. By doing so, the generated image embedding vectors can reflect the relative importance of each pixel for each class as well as show relationship between different classes via the proximity between vectors of corresponding classes
Summer 2020