The automatic detection and classification of stance (e.g., certainty or agreement) in text data using natural language processing and machine learning methods create an opportunity to gain insight into the speakers' attitudes towards their own and other people's utterances. However, identifying stance in text presents many challenges related to training data collection and classifier training. In order to facilitate the entire process of training a stance classifier, we propose a visual analytics approach, called ALVA, for text data annotation and visualization. ALVA's interplay with the stance classifier follows an active learning strategy in order to select suitable candidate utterances for manual annotation. Our approach supports annotation process management and provides the annotators with a clean user interface for labeling utterances with multiple stance categories. ALVA also contains a visualization method to help analysts of the annotation and training process gain a better understanding of the categories used by the annotators. The visualization uses a novel visual representation, called CatCombos, which groups individual annotation items by the combination of stance categories. Additionally, our system makes a visualization of a vector space model available that is itself based on utterances. ALVA is already being used by our domain experts in linguistics and computational linguistics in order to improve the understanding of stance phenomena and to build a stance classifier for applications such as social media monitoring.