In the near future, transportation systems will include both autonomous vehicles and human-operated vehicles sharing the same traffic conditions. Human drivers will have difficulty predicting the actions of autonomous vehicles and the latter will face challenges due to complex decision-making algorithms and dynamic environments. The lack of standardized interaction protocols between autonomous vehicles and human drivers further complicates safe decision-making. This paper proposes an AI-based advisory framework to enhance human driving skills in mixed autonomy traffic and improve autonomous vehicles in a Human–AI teaming fashion. Our framework is composed of both a centralized component and a decentralized component. The centralized component primarily identifies driving style and trajectory parameters that impact traffic efficiency across large-scale traffic networks shared by human drivers and autonomous vehicles. At the local level, however, our proposed framework features a decentralized, agent-based strategy to enable effective coordination between human and autonomous vehicles—especially at complex intersections. An initial prototype is modeled and implemented in a desktop virtual reality environment for testing and training. 

Industrial applications of Artificial Intelligence (AI) can be hindered by the issues of explainability and trust from end users. Human-computer interaction and eXplainable AI (XAI) concerns become imperative in such scenarios. However, the prior evidence of applying more general principles and techniques in specialized industrial scenarios is often limited. In this case study, we focus on designing interactive interfaces of XAI solutions for operators in the pulp and paper industry. The explanation techniques supported and compared include counterfactual and feature importance explanations. We applied the user-centered design methodology, including the analysis of requirements elicited from operators during site visits and interactive interface prototype evaluation eventually conducted on site with five operators. Our results indicate that the operators preferred the combination of counterfactual and feature importance explanations. The study also provides lessons learned for researchers and practitioners.

As the levels of automation and reliance on modern artificial intelligence (AI) approaches increase across multiple industries, the importance of the human-centered perspective becomes more evident. Various actors in such industrial applications, including equipment operators and decision makers, have their needs and preferences that often do not align with the decisions produced by black-box models, potentially leading to mistrust and wasted productivity gain opportunities. In this paper, we examine these issues through the lenses of visual analytics and, more broadly, interactive visualization, and we argue that the methods and techniques from these fields can lead to advances in both academic research and industrial innovations concerning the explainability of AI models. To address the existing gap within and across the research and application fields, we propose a conceptual framework for visual analytics design and evaluation for such scenarios, followed by a preliminary roadmap and call to action for the respective communities.

In the near future, transportation systems will includeboth autonomous vehicles and human-operated vehicles sharing thesame traffic conditions. Human drivers may have difficulties in predictingthe actions of autonomous vehicles, and the latter will facechallenges due to complex decision-making algorithms and dynamicenvironments. The lack of standardized communication between autonomousvehicles and human drivers complicates safe decisionmaking.This paper proposes a centralized generative AI-based advisoryframework to enhance human driving skills in mixed-autonomytraffic, and also to improve the autonomous vehicles in a Human-AIteaming fashion. The framework includes visual aids, explanations,and a virtual reality environment for testing and training.

An essential task of an air traffic controller is to manage the traffic flow by predicting future trajectories. Complex traffic patterns are difficult to predict and manage and impose cognitive load on the air traffic controllers. In this work we present an interactive visual analytics interface which facilitates detection and resolution of complex traffic patterns for air traffic controllers. The interface supports air traffic controllers in detecting complex clusters of aircraft and further enables them to visualize and simultaneously compare how different re-routing strategies for each individual aircraft yield reduction of complexity in the entire sector for the next hour. The development of the concepts was supported by the domain-specific feedback we received from six fully licensed and operational air traffic controllers in an iterative design process over a period of 14 months.

In industrial applications where the physical parameters are highly interconnected, keeping the process flow steady is a major concern for the operators. This is caused by the sensitivity of system to the process dynamics. As a result, a slight adjustment to a control parameter can significantly affect the efficiency of the system and thus impact the financial gain. Paper pulp production is an example of such a process, where operators continuously investigate the potential of changes in the process and predict the consequences of an adjustment before making a decision. Process parameter adjustments prescribed by simulated control models cannot be fully trusted as the external disturbances and the process inherent variabilities cannot be fully incorporated into the simulations. Therefore, to assess the viability of a strategy, operators often compare the situation with the historical records and trends during which the processes in the plant ran steadily. While previous research has mostly focused on developing advanced control models to simulate complex pulp production process, this work aims to support operators analytical reasoning by provision of effective data visualization. The contributions of our design study include a domain problem characterization and a linked-view visual encoding design, which aims to enhance operator's mental models independent of particular users or scenarios. Finally, by reflecting on the advantages of our choice of task abstraction technique, inherited from the ecological interface design framework [5], we reason for the generalizability of our approach to similar industrial applications.

Operators in air traffic control facing time- and safety-critical situations call for efficient, reliable and robust real-time processing and interpretation of complex data. Automation support tools aid controllers in these processes to prevent separation losses between aircraft. Issues of current support tools include limited what-if and what-else probe functionalities in relation to vertical solutions. This work presents the design and evaluation of two visual analytics interfaces that promote contextual awareness and support what-if and what-else probes in the spatio-temporal domain aiming to improve information integration and support controllers in prioritising conflict resolution. Both interfaces visualize vertical solution spaces against a time-altitude graph. The main contributions of this paper are: (a) the presentation of two interfaces for supporting conflict solving; (b) the novel representation of how vertical information and aircraft rate of climb and descent affect conflicts and (c) an evaluation and comparison of the interfaces with a traditional air traffic control support system. The evaluation study was performed with domain experts to compare the effects of visualization concepts on operator engagement in processing solutions suggested by the tools. Results show that the visualizations support operators ability to understand and resolve conflicts. Based on the results, general design guidelines for time-critical domains are proposed.

Operational demands in safety-critical systems impose a risk of failure to the operators especially during urgent situations. Operators of safety-critical systems learn to make decisions effectively throughout extensive training programs and many years of experience. In the domain of air traffic control, expensive training with high dropout rates calls for research to enhance novices' ability to detect and resolve conflicts in the airspace. While previous researchers have mostly focused on redesigning training instructions and programs, the current paper explores possible benefits of novel visual representations to improve novices' understanding of the situations as well as their decision-making process. We conduct an experimental evaluation study testing two ecological visual analytics interfaces, developed in a previous study, as support systems to facilitate novice decision-making. The main contribution of this paper is threefold. First, we describe the application of an ecological interface design approach to the development of two visual analytics interfaces. Second, we perform a human-in-the-loop experiment with forty-five novices within a simplified air traffic control simulation environment. Third, by performing an expert-novice comparison we investigate the extent to which effects of the proposed interfaces can be attributed to the subjects' expertise. The results show that the proposed ecological visual analytics interfaces improved novices' understanding of the information about conflicts as well as their problem-solving performance. Further, the results show that the beneficial effects of the proposed interfaces were more attributable to the visual representations than the users' expertise. 

Automation in today’s world supports human operators to accomplish several tasks in limited time. With more advanced automation and autonomous systems, the hu-mans’ role is shifting from hands-on operational tasks to supervisory tasks. In complex environments such as air traffic control, supervisory tasks become difficult to manage during unexpected situations as the operator needs to have a clear understanding of various resolution strategies and their consequences and make decisions about them in a limited amount of time (i.e. within a couple of minutes). In such environments, interface designers must carefully consider how information should be presented to the operators. An improper way of presenting information could, wastefully consume operators’ cognitive resources resulting in inefficient decision-making and an increased risk of failure. 

By designing ecological visual analytics interfaces, this thesis addresses the problem of real-time decision-making in the domain of air traffic control. The aim of this thesis has been to apply ecological design theories to the design and evaluation of visual representations to better support controllers’ analytical capabilities and decision-making. Four novel visual analytics interfaces were designed, developed, and tested over the course of this research project. To understand how the designed visual representations affected the operators’ decision-making processes, evaluation studies with air traffic controllers as well as novices without ATC experience were conducted for two of the designed interfaces and the results were analyzed. 

The contribution of this thesis to the field of air traffic control and visualization design is fourfold. First, the thesis contributes knowledge on what information should be visualized and how, to achieve functional goals of conflict detection and resolution task of air traffic control. Second, evolved through a series of design studies, a final interactive visual analytics interface is proposed that visualizes information about the available solution space for solving conflict situations between airborne traffic and the traffic complexity. The interface supports controllers’ decision-making process for resolving conflicts and ability to reduce the traffic complexity. Third, the method developed for evaluating the interface designs contribute with knowledge on how interfaces tailored to safety-critical systems can be tested. Fourth, findings show that the integration of ecological interface design with the development of visual representations can shape novice and expert operators’ decision-making towards domain-specific functional goals, while allowing them to follow their own problem-solving strategies. 

Automation hjälper mänskliga operatörer att utföra uppgifter parallellt, på begränsad tid. Med mer avancerad automation och autonoma system förändras människans roll, och går över från mer direkt kontroll till övervakning- och intervention. I komplexa miljöer som flygledning, blir övervakningsuppgifter svåra att hantera under oväntade situationer eftersom operatören behöver ha en klar förståelse för olika strategier, deras konsekvenser, och fatta beslut om dem under en begränsad tid (inom ett par minuter). I sådana miljöer måste en visualiseringsdesigner noga överväga hur information ska presenteras för operatörerna. Ett felaktigt sätt att presentera information belastar operatörernas kognitiva resurser, vilket leder till ineffektivt beslutsfattande och en ökad risk för misslyckanden.  

Genom att utforma ekologiska visuella analytiska gränssnitt tar denna avhandling upp problemet med beslutsfattande i realtid, i oplanerade situationer, inom området för flygledning. Syftet med denna avhandling har varit att tillämpa ekologiska designteorier på design och utvärdering av visuella representationer. Målet var att bättre stödja operatörers analytiska förmåga och beslutsfattande. Fyra nya visuella analysgränssnitt utformades, utvecklades och testades. För att förstå hur de visuella representationerna påverkade operatörernas beslutsprocesser genomfördes utvärderingsstudier med såväl flygledare som nybörjare utan domänerfarenhet för två av gränssnitten och resultaten analyserades.  

Avhandlingen ger fyra bidrag till området flygledning och till området visualiseringsdesign. För det första bidrar den med kunskap om vilken information som ska visualiseras och hur man uppnår funktionella mål för att upptäcka och lösa konflikter i flygledning. För det andra, utvecklat genom en serie designstudier, föreslås ett interaktivt visuellt analytiskt gränssnitt. Det visualiserar information om det tillgängliga lösningsutrymmet för att lösa konfliktsituationer mellan luftburen trafik, samt visar trafikens komplexitet. Gränssnittet utformades för att stödja beslutsprocesser för att lösa konflikter och för att minska komplexitet i trafik. För det tredje bidrar den metod som utvecklats för att utvärdera gränssnittsdesignerna, med kunskap om hur gränssnitt skräddarsydda för säkerhetskritiska system kan testas. För det fjärde bidrar avhandlingen med resultat angående integrationen av ekologisk gränssnittsdesign med utveckling av visuella representationer. Den visar hur gränssnitten kan påverka nybörjare och operatörers (experters) beslutsfattande angående domänspecifika funktionella mål, samtidigt som gränssnitten tillåter dem att följa sina egna problemlösningsstrategier.   

Air traffic control is a safety critical high-risk environment where operators need to analyse and interpret traffic dynamics of spatio-temporal data in real-time. To support the air traffic controller in safely separating traffic, earlier research has applied real-time visualisation techniques that explore the constraints and solution spaces of separation problems. Traditionally, situation displays for conflict detection and resolution have used visualisations that convey information about the relative horizontal position between aircraft. Although vertical solutions for solving conflicts are common, and often a preferred among controllers, visualisations typically provide limited information about the vertical relationship between aircraft. This paper presents a design study of an interactive conflict detection and resolution support tool and explores techniques for real-time visualisation of spatio-temporal data. The design evolution has incorporated several activities, including an initial work domain analysis, iterative rounds of programming, design, and evaluations with a domain expert, and an evaluation with eight active controllers. The heading-time-altitude visualisation system is developed based on formulating and solving aircraft movements in a relative coordinate system. A polar-graph visualisation technique is used to construct a view of conflicting aircraft vertical solution spaces in the temporal domain. Using composite glyphs, the final heading-time-altitude visualisation provides a graphical representation of both horizontal and vertical solution spaces for the traffic situation.