This volume does much more than survey modern advanced color processing. Starting with a historical perspective on ways we have classified color, it sets out the latest numerical techniques for analyzing and processing colors, the leading edge in our search to accurately record and print what we see. The human eye perceives only a fraction of available light wavelengths, yet we live in a multicolor world of myriad shining hues. Colors rich in metaphorical associations make us "purple with rage" or "green with envy" and cause us to "see red." Defining colors has been the work of centuries, culminating in today's complex mathematical coding that nonetheless remains a work in progress: only recently have we possessed the computing capacity to process the algebraic matrices that reproduce color more accurately. With chapters on dihedral color and image spectrometers, this book provides technicians and researchers with the knowledge they need to grasp the intricacies of today's color imaging.

A search engine for font recognition is presented and evaluated. The intended usage is the search in very large font databases. The input to the search engine is an image of a text line, and the output is the name of the font used when rendering the text. After pre-processing and segmentation of the input image, a local approach is used, where features are calculated for individual characters. The method is based on eigenimages calculated from edge filtered character images, which enables compact feature vectors that can be computed rapidly. In this study the database contains 2763 different fonts for the English alphabet. To resemble a real life situation, the proposed method is evaluated with printed and scanned text lines and character images. Our evaluation shows that for 99 % of the queries, the correct font name can be found within the five best matches.

We investigate the emotional response to colors in ordinary multicolored images. In psychophysical experiments, using both category scaling and interval scaling, observers are asked to judge images using three emotion factors: activity, weight, and heat. The color emotion metric was originally developed for single colors, and later extended to include pairs of colors. The same metric was recently used in image retrieval. The results show that people in general perceive color emotions for multi-colored images in similar ways, and that observer judgments correlate with the recently proposed method used in image retrieval. The intended usage is in retrieval systems publicly available on the Internet, where both the user and the viewing environment is unknown, which requires novel ways of conducting the psychophysical experiments.

Traditional content based image indexing aims at developing algorithms that can analyze and index images based on their visual content. A typical approach is to measure image attributes, like colors or textures, and save the result in image descriptors, which then can be used in recognition and retrieval applications. Two topics within content based image indexing are addressed in this thesis: Emotion based image indexing, and font recognition.

The main contribution is the inclusion of high-level semantics in indexing of multi-colored images. We focus on color emotions and color harmony, and introduce novel emotion and harmony based image descriptors, including global emotion histograms, a bag-of-emotions descriptor, an image harmony descriptor, and an indexing method based on Kobayashi's Color Image Scale. The first three are based on models from color science, analyzing emotional properties of single colors or color combinations. A majority of the descriptors are evaluated in psychophysical experiments. The results indicate that observers perceive color emotions and color harmony for multi-colored images in similar ways, and that observer judgments correlate with values obtained from the presented descriptors. The usefulness of the descriptors is illustrated in large scale image classification experiments involving emotion related image categories, where the presented descriptors are compared with global and local standard descriptors within this field of research. We also investigate if these descriptors can predict the popularity of images. Three image databases are used in the experiments, one obtained from an image provider, and two from a major image search service. The two from the search service were harvested from the Internet, containing image thumbnails together with keywords and user statistics. One of them is a traditional object database, whereas the other is a unique database focused on emotional image categories. A large part of the emotion database has been released to the research community.

The second contribution is visual font recognition. We implemented a font search engine, capable of handling very large font databases. The input to the search engine is an image of a text line, and the output is the name of the font used when rendering the text. After pre-processing and segmentation of the input image, eigenimages are used, where features are calculated for individual characters. The performance of the search engine is illustrated with a database containing more than 2700 fonts. A system for visualizing the entire font database is also presented.

Both the font search engine, and the descriptors that are related to emotions and harmony are implemented in publicly available search engines. The implementations are presented together with user statistics.

We investigate the use of standard image descriptors and a supervised learning algorithm for estimating the popularity of images. The intended application is in large scale image search engines, where the proposed approach can enhance the user experience by improving the sorting of images in a retrieval result. Classification methods are trained and evaluated on real-world user statistics recorded by a major image search engine. The conclusion is that for many image categories, the combination of supervised learning algorithms and standard image descriptors results in useful popularity predictions.

We propose a color-based image descriptor that can be used for image indexing based on high-level semantic concepts. The descriptor is based on Kobayashi's Color Image Scale, which is a system that includes 130 basic colors combined in 1170 three-color combinations. Each combination is labeled with one of 180 high-level semantic concepts, like "elegant", "romantic", "provocative", etc. Moreover, words are located in a two-dimensional semantic space, and arranged into groups based on perceived similarity. From a modified approach for statistical analysis of images, involving transformations of ordinary RGB-histograms, a semantic image descriptor is derived, containing semantic information about both color combinations and single colors in the image. We show how the descriptor can be translated into different levels of semantic information, and used in indexing of multi-colored images. Intended applications are, for instance, image labeling and retrieval.

We introduce two large databases consisting of 750 000 and 1.2 million thumbnail-sized images, labeled with emotion-related keywords. The smaller database consists of images from Matton Images, an image provider. The larger database consists of web images that were indexed by the crawler of the image search engine Picsearch. The images in the Picsearch database belong to one of 98 emotion related categories and contain meta-data in the form of secondary keywords, the originating website and some view statistics. We use two psycho-physics related feature vectors based on the emotional impact of color combinations, the standard RGB-histogram and two SIFT-related descriptors to characterize the visual properties of the images. These features are then used in two-class classification experiments to explore the discrimination properties of emotion-related categories. The clustering software and the classifiers are available in the public domain, and the same standard configurations are used in all experiments. Our findings show that for emotional categories, descriptors based on global image statistics (global histograms) perform better than local image descriptors (bag-of-words models). This indicates that content-based indexing and retrieval using emotion-based approaches are fundamentally different from the dominant object- recognition based approaches for which SIFT-related features are the standard descriptors.

In this paper we describe how to include high level semantic information, such as aesthetics and emotions, into Content Based Image Retrieval. We present a color-based emotion-related image descriptor that can be used for describing the emotional content of images. The color emotion metric used is derived from psychophysical experiments and based on three variables: activity, weight and heat. It was originally designed for single-colors, but recent research has shown that the same emotion estimates can be applied in the retrieval of multi-colored images. Here we describe a new approach, based on the assumption that perceived color emotions in images are mainly affected by homogenous regions, defined by the emotion metric, and transitions between regions. RGB coordinates are converted to emotion coordinates, and for each emotion channel, statistical measurements of gradient magnitudes within a stack of low-pass filtered images are used for finding interest points corresponding to homogeneous regions and transitions between regions. Emotion characteristics are derived for patches surrounding each interest point, and saved in a bag-of-emotions, that, for instance, can be used for retrieving images based on emotional content.

A predictive model for estimating the perceived harmony of ordinary multi-colored images is proposed and evaluated. The model is based on earlier research concerning two-color harmonies. Color regions of images are extracted using mean shift segmentation. Global and local harmony scores are derived for two-color combinations included in different subsets of all segmented regions. Statistical measurements of the obtained harmony scores are used for predicting the perceived overall harmony. The model is validated in a psychophysical experiment, where human observers are judging images on a harmony scale. The findings show that humans do perceive harmony in multi-colored images in similar ways, and that the proposed model results in useful predictions of harmony. The model can be applied in automatic labeling or classification of images.