We present a visual analytics approach for multi-level visual exploration of users' interaction strategies in an interactive digital environment. The use of interactive touchscreen exhibits in informal learning environments, such as museums and science centers, often incorporate frameworks that classify learning processes, such as Bloom's taxonomy, to achieve better user engagement and knowledge transfer. To analyze user behavior within these digital environments, interaction logs are recorded to capture diverse exploration strategies. However, analysis of such logs is challenging, especially in terms of coupling interactions and cognitive learning processes, and existing work within learning and educational contexts remains limited. To address these gaps, we develop a visual analytics approach for analyzing interaction logs that supports exploration at the individual user level and multi-user comparison. The approach utilizes algorithmic methods to identify similarities in users' interactions and reveal their exploration strategies. We motivate and illustrate our approach through an application scenario, using event sequences derived from interaction log data in an experimental study conducted with science center visitors from diverse backgrounds and demographics. The study involves 14 users completing tasks of increasing complexity, designed to stimulate different levels of cognitive learning processes. We implement our approach in an interactive visual analytics prototype system, named VISID, and together with domain experts, discover a set of task-solving exploration strategies, such as “cascading” and “nested-loop', which reflect different levels of learning processes from Bloom's taxonomy. Finally, we discuss the generalizability and scalability of the presented system and the need for further research with data acquired in the wild.

Automation in Air Traffic Control (ATC) is gaining an increasing interest. Possible relevant applications are in automated decision support tools leveraging the performance of the Air Traffic Controller (ATCO) when performing tasks such as Conflict Detection and Resolution (CD & R). Another important area of application is in ATCOs training by aiding instructors to assess the trainees strategies. From this perspective, models that capture the cognitive processes and reveal ATCOs work strategies need to be built. In this work, we investigated a novel approach based on topic modelling to learn controllers work patterns from temporal event sequences obtained by merging eye movement data with data from simulation logs. A comparison of the work phases exhibited by the topic models and the Conflict Life Cycle (CLC) reference model, derived from post-simulation interviews with the ATCOs, indicated that there was a correspondence between the phases captured by the proposed method and the CLC framework. Another contribution of this work is a method to assess similarities between ATCOs work strategies. A first proof-of-concept application targeting the CD & R task is also presented.

Real-world event sequence data, such as activity logs, eye-tracking data, simulation data, and electronic health records, often share characteristics such as a large alphabet of events, fragmentation, noise, and high complexity which makes them difficult to analyze in their raw form. Because of this, simplification and preprocessing through various data transformations are commonly required before the data can be effectively visualized and analyzed. Existing methods for such data transformation are either manually applied and rely heavily on user expertise, or use algorithmic approaches to apply bulk operations which can imply the loss of potentially important information without users being aware. To bridge this gap, we propose a visual analytics approach that aims to successively increase the quality of noisy event sequences by supporting an interactive, context-aware application of data transformations. This is achieved by providing cues concerning the potential loss of information that transformation operations may imply and allowing users to explore, and visually assess their impact on the data. Therefore, a central feature of the approach is that users can tune the data transformation process so that important identified data characteristics are preserved. We motivate the proposed approach in the domain of air traffic control and illustrate it through a usage example, using event sequences derived by merging eye-tracking and simulator data from a human-in-the-loop simulation experiment with 14 air traffic controllers.

The objective of en-route control is to ensure minimum separation between aircraft in the sector under all circumstances. Exploring and understanding the related work patterns of air traffic controllers on how to successfully perform this task is crucial for the future development and implementation of automation solutions. Future automation must match the logic of decision making and the need for information at the right time in identifying and resolving conflicts. The objective of this paper is to identify “decision cues” that are relevant during decision-making and its relation to the controller’s intention. A retrospective think aloud method was applied in which en-route controllers commented their own work behaviour after playing a simple conflict scenario in the simulator. A set of decision cues were identified using 13 controllers and classified using a Conflict Life Cycle-model, dividing the task into four work steps. The result shows clear differences in the compilation of decision cues used between work steps. Large differences were found among controllers, indicating personal preferences in consideration of information, timing, and chosen conflict resolution. The results further show that the “conflict resolution probing” step is the most challenging task because it contains the most decision cues. The high inter-individual variance in the cue composition of this step indicates a high degree of individual skill development on which the adoption and selection of conflict solutions is based. The results support the future hypothesis-driven verification of controllers’ work pattern and intention of decision-making and related automated solutions.

This paper presents an interactive sequence mining approach for exploring long duration event-sequences and identifying interesting patterns within them. The approach extends previous work on exploratory sequence mining by using a sliding window to split the sequence prior to mining. Patterns are interactively grown and visualized through a tree representation, while a set of accompanying views allows for identified patterns to be explored in the context in which they occur. The approach is motivated and exemplified in the domain of air traffic control and, in particular, air traffic controller training.

Eye-Tracking experiments have proven to be of great assistance in understanding human computer interaction across many fields. Most eye-tracking experiments are non-intrusive and so do not affect the behaviour of the subject. Such experiments usually last for just a few minutes and so the spatio- temporal data generated by the eye-tracker is quite easy to analyze using simple visualization techniques such as heat maps and animation. Eye tracking experiments in air traffic control, or maritime or driving simulators can, however, last for several hours and the analysis of such long duration data becomes much more complex. We have developed an analysis pipeline, where we identify visual spatial areas of attention over a user interface using clustering and hierarchical cluster merging techniques. We have tested this technique on eye tracking datasets generated by air traffic controllers working with Swedish air navigation services, where each eye tracking experiment lasted for ∼90 minutes. We found that our method is interactive and effective in identification of interesting patterns of visual attention that would have been very difficult to locate using manual analysis.

Eye tracking is used to analyze and compare user behaviour within numerous domains, but long duration eye tracking experiments across multiple users generate millions of eye gaze samples, making the data analysis process complex. Usually the samples are labelled into Areas of Interest (AoI) or Objects of Interest (OoI), where the AoI approach aims to understand how a user monitors different regions of a scene while OoI identification uncovers distinct objects in the scene that attract user attention. Using scalable clustering and cluster merging techniques that require minimal user input, we label AoIs across multiple users in long duration eye tracking experiments. Using the common AoI labels then allows direct comparison of the users as well as the use of such methods as Hidden Markov Models and Sequence mining to uncover common and distinct behaviour between the users which, until now, has been prohibitively difficult to achieve.

Sequential pattern mining finds applications in numerous diverging fields. Due to the problem's combinatorial nature, two main challenges arise. First, existing algorithms output large numbers of patterns many of which are uninteresting from a user's perspective. Second, as datasets grow, mining large number of patterns gets computationally expensive. There is, thus, a need for mining approaches that make it possible to focus the pattern search towards directions of interest. This work tackles this problem by combining interactive visualization with sequential pattern mining in order to create a "transparent box" execution model. We propose a novel approach to interactive visual sequence mining that allows the user to guide the execution of a pattern-growth algorithm at suitable points through a powerful visual interface. Our approach (1) introduces the possibility of using local constraints during the mining process, (2) allows stepwise visualization of patterns being mined, and (3) enables the user to steer the mining algorithm towards directions of interest. The use of local constraints significantly improves users' capability to progressively refine the search space without the need to restart computations. We exemplify our approach using two event sequence datasets; one composed of web page visits and another composed of individuals' activity sequences.

Eye-tracking has become an invaluable tool for the analysis of working practices in many technological fields of activity. Typically studies focus on short tasks and use static expected areas of interest (AoI) in the display to explore subjects’ behaviour, making the analyst’s task quite straightforward. In long-duration studies, where the observations may last several hours over a complete work session, the AoIs may change over time in response to altering workload, emergencies or other variables making the analysis more difficult. This work puts forward a novel method to automatically identify spatial AoIs changing over time through a combination of clustering and cluster merging in the temporal domain. A visual analysis system based on the proposed methods is also presented. Finally, we illustrate our approach within the domain of air traffic control, a complex task sensitive to prevailing conditions over long durations, though it is applicable to other domains such as monitoring of complex systems. 

We present the results from a particle-in-cell (PIC) simulation that models the interaction between a spatially localized electron-positron cloud and an electron-ion plasma. The latter is permeated by a magnetic field that is initially spatially uniform and aligned with the mean velocity vector of the pair cloud. The pair cloud expels the magnetic field and piles it up into an electromagnetic piston. Its electromagnetic field is strong enough to separate the pair cloud from the ambient plasma in the direction that is perpendicular to the cloud propagation direction. The piston propagates away from the spine of the injected pair cloud and it accelerates the protons to a high nonrelativistic speed. The accelerated protons form an outer cocoon that will eventually become separated from the unperturbed ambient plasma by a fast magnetosonic shock. No electromagnetic piston forms at the front of the cloud and a shock is mediated here by the filamentation instability. The final plasma distribution resembles that of a hydrodynamic jet. Collisionless plasma jets may form in the coronal plasma of accreting black holes and the interaction between the strong magnetic field of the piston and the hot pair cloud may contribute to radio emissions by such objects.