In this paper we introduce the representation theory of the symmetric group S(3) as a tool to investigate thestructure of the space of RGB-histograms. We show that the theory reveals that typical histogram spaces arehighly structured and that these structures originate partly in group theoretically defined symmetries. Thealgorithms exploit this structure and constructs a PCA like decomposition without the need to construct correlationor covariance matrices and their eigenvectors. We implemented these algorithms and investigate theirproperties with the help of two real-world databases (one from an image provider and one from a image searchengine company) containing over one million images.