Real-world sequential decision-making tasks are usually complex, and require trade-offs between multiple–often conflicting–objectives. However, the majority of research in reinforcement learning (RL) and decision-theoretic planning assumes a single objective, or that multiple objectives can be handled via a predefined weighted sum over the objectives. Such approaches may oversimplify the underlying problem, and produce suboptimal results. This extended abstract outlines the limitations of using a semi-blind iterative process to solve multi-objective decision making problems. Our extended paper [4], serves as a guide for the application of explicitly multi-objective methods to difficult problems.

This article explores the complex challenges of co-designing an AI- and learning analytics (LA)-integrated learning management system (LMS). While co-design has been proposed as a human-centred design approach for scaling AI and LA adoption, our understanding of how these design processes play out in real-life settings remains limited. This study is based on ethnographic fieldwork in primary and secondary schools and employs a relational materialist approach to trace, visualise, and analyse the increasingly complex and transformative relations between a growing number of actors. The findings shed light on the intricate ecosystem in which AI and LA are being introduced and on the marketisation of K-12 education. Instead of following a rational and sequential approach that can be easily executed, the co-design process emerged as a series of events, shifting from solely generating ideas with teachers to integrating and commercialising the LMS into a school market with an already high prevalence of educational technology (EdTech). AI and LA in education, co-design and data-driven schooling served as negotiating ideas, boundary objects, which maintained connectivity between actors, despite limited AI and LA implementation and the development of a stand-alone app. Even though teachers and students were actively involved in the design decisions, the co-design process did not lead to extensive adoption of the LMS nor did it sufficiently address the ethical issues related to the unrestricted collection of student data.

One-step reinforcement learning explanation methods account for individual actions but fail to consider the agents future behavior, which can make their interpretation ambiguous. We propose to address this limitation by providing hierarchical goals as context for one-step explanations. By considering the current hierarchical goal as a context, one-step explanations can be interpreted with higher certainty, as the agents future behavior is more predictable. We combine reward decomposition with hierarchical reinforcement learning into a novel explainable reinforcement learning framework, which yields more interpretable, goal-contextualized one-step explanations. With a qualitative analysis of one-step reward decomposition explanations, we first show that their interpretability is indeed limited in scenarios with multiple, different optimal policies-a characteristic shared by other one-step explanation methods. Then, we show that our framework retains high interpretability in such cases, as the hierarchical goal can be considered as context for the explanation. To the best of our knowledge, our work is the first to investigate hierarchical goals not as an explanation directly but as additional context for one-step reinforcement learning explanations.

Artificial intelligence (AI) is often highlighted as a transformative technology that can“address some of the biggest challenges in education today”(UNESCO, 2019). Introducing data-driven AI in classrooms also raises pedagogical and ethical concerns related to students’, teachers’ and teacher educators’ understanding of how AI works in theory and practice (Holmes, 2022; Sperling et al., 2022). This extended abstract presents initial findings from the first study conducted within the WASP-HS1-funded research project: "AI Literacy for Swedish Teacher Education - A Participatory Design Approach". The project aims to establish a scientific foundation for teaching AI literacy in teacher education (TE) programs.

Many real-world problems require a trade-off between multiple conflicting objectives. Decision-makers’ preferences over solutions to such problems are determined by their utility functions, which convert multi-objective values to scalars. In some settings, utility functions change over time, and the goal is to find methods that can efficiently adapt an agent’s policy to changes in utility. Previous work on learning with dynamic utility functions has focused on model-free methods, which often suffer from poor sample efficiency. In this work, we instead propose a model-based actor-critic, which explores with diverse utility functions through imagined rollouts within a learned world model between interactions with the real environment. An experimental evaluation on Minecart, a well-known benchmark for multi-objective reinforcement learning, shows that by learning a model of the environment the quality of the agent’s policy is improved compared to model-free algorithms.

Many real-world problems require a trade-off between multiple conflicting objectives. Decision-makers’ preferences over solutions to such problems are determined by their utility functions, which convert multi-objective values to scalars. In some settings, utility functions change over time, and the goal is to find methods that can efficiently adapt an agent’s policy to changes in utility. Previous work on learning with dynamic utility functions has focused on model-free methods, which often suffer from poor sample efficiency. In this work, we instead propose a model-based actor-critic, which explores with diverse utility functions through imagined rollouts within a learned world model between interactions with the real environment. An experimental evaluation shows that by learning a model of the environment the performance of the agent can be improved compared to model-free algorithms.

Imagine placing an online order on your way to the grocery store, then being able to pick the collected basket upon arrival or shortly after. Likewise, imagine placing any online retail order, made ready for pickup in minutes instead of days. In order to realize such a low-latency automatic warehouse logistics system, solvers must be made to be basketaware. That is, it is more important that the full order (the basket) is picked timely and fast, than that any single item  in the order is picked quickly. Current state-of-the-art methods are not basket-aware. Nor are they optimized for a positive customer experience, that is; to prioritize customers based on queue place and the difficulty associated with  picking their order. An example of the latter is that it is preferable to prioritize a customer ordering a pack of diapers over a customer shopping a larger order, but only as long as the second customer has not already been waiting for  too long. In this work we formalize the problem outlined, propose a new method that significantly outperforms the state-of-the-art, and present a new realistic simulated benchmark. The proposed method is demonstrated to work in an on-line and real-time setting, and to solve the on-demand multi-agent basket picking problem for automated shopping stores under realistic conditions.

Silver et al.[14] posit that scalar reward maximisation is sufficient to underpin all intelligence and provides a suitable basis for artificial general intelligence (AGI). This extended abstract summarises the counter-argument from our JAAMAS paper [19].

The concept of transparency is fragmented in artificial intelligence (AI) research, often limited to transparency of the algorithm alone. We propose that AI transparency operates on three levels-algorithmic, interaction, and social-all of which need to be considered to build trust in AI. We expand upon these levels using current research directions, and identify research gaps resulting from the conceptual fragmentation of AI transparency highlighted within the context of the three levels.

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

Coverage Path Planning is the work horse of contemporary service task automation, powering autonomous floor cleaning robots and lawn mowers in households and office sites. While steady progress has been made on indoor cleaning and outdoor mowing, these environments are small and with simple geometry compared to general urban environments such as city parking garages, highway bridges or city crossings. To pave the way for autonomous road sweeping robots to operate in such difficult and complex environments, a benchmark suite with three large-scale 3D environments representative of this task is presented. On this benchmark we evaluate a new Coverage Path Planning method in comparison with previous well performing algorithms, and demonstrate state-of-the-art performance of the proposed method. Part of the success, for all evaluated algorithms, is the usage of automated domain adaptation by in-the-loop parameter optimization using Bayesian Optimization. Apart from improving the performance, tedious and bias-prone manual tuning is made obsolete, which makes the evaluation more robust and the results even stronger.

The high operational cost of aircraft, limited availability of air space, and strict safety regulations make training of fighter pilots increasingly challenging. By integrating Live, Virtual, and Constructive simulation resources, efficiency and effectiveness can be improved. In particular, if constructive simulations, which provide synthetic agents operating synthetic vehicles, were used to a higher degree, complex training scenarios could be realised at low cost, the need for support personnel could be reduced, and training availability could be improved. In this work, inspired by the recent improvements of techniques for artificial intelligence, we take a user perspective and investigate how intelligent, learning agents could help build future training systems. Through a domain analysis, a user study, and practical experiments, we identify important agent capabilities and characteristics, and then discuss design approaches and solution concepts for training systems to utilise learning agents for improved training value.

As a result of the increased digitalisation, many countries have introduced programming in their primary education curricula. One main objective is to give all children equal opportunities to develop the skills needed to be an active participant and producer in a digitalized society. This also addresses another important objective, that of increased diversity and broadened participation. Despite technology being a natural part in our everyday lives, stereotypical views of programming as a primarily male activity still exist. In this paper, we explore girls’ and boys’ experiences of programming at school and in their spare time. The study is situated in primary school classrooms in Sweden, where programming was introduced in a cross-curricular manner as part of digital competence in 2018. While most students reported having some programming experience, it was quite limited. The results show that, compared to the girls, boys in grades 4-9 are somewhat more positive towards programming and get more programming experience both at school and in their spare time. Similarly, boys rated their self-perceived programming skills higher than the girls. In grades 1-3, no gender disparity was found in students’ attitudes, experiences or skills. However, the gender differences in grades 4-9 were not reflected to an equally high extent in the students’ programming skills, as girls and boys did equally well on many skills related tasks. The analysis highlights the importance of well planned, motivating and relevant tasks in order to provide positive experiences of programming in the classroom.

This Research Full Paper explore inclusion in programming in primary school. Education plays a crucial role in engaging a diverse group of students with different social backgrounds and interests. Therefore, this study aims to shed light upon inclusion in programming in primary school, focusing on gender to increase the knowledge regarding inclusion in programming didactics. The following research questions have guided the study: How are programming activities designed in primary school? How do pupils approach the programming tasks given? Can any gender differences be observed, and what are the consequences for the teaching practice? The theoretical framework used to analyse the empirical material is at the intersection between multimodal social semiotics [1] and a design-oriented perspective [2]. The empirical material consists of classroom video observations. Programming lessons in grades 4-8 have been observed and videos was recorded during 2019-2020. The pupils have worked on eight different programming tasks during the lessons. Analysis of these programming activities (tasks, instructions and resources used) focusing on gender has been made. Findings show two aspects 1) interest and position and 2) representations of knowledge. Regarding interest and position, the study of programming activities shows both similarities and differences between girls’ and boys’ approach to the task. Similarities are shown regarding the learning activities. No differences in coding strategies or creativity are observed if the task has an open design. The differences are shown in the guided tasks, where boys tend to engage in the tasks from their interests rather than following instructions and girls tend to follow the instructions given by the teacher. From a gender perspective, the boys might find programming more creative and fun, and the girls might feel less engaged as their interest falls into the background. Secondly, knowledge representations might affect who is seen as an expert within the CS field. For example, in grades 4 and 5, a male voice was represented in the video clips and a guest teacher used when presenting programming activities. The resources used in the lessons can be seen as representations of knowledge. In this case, they are always connected to a social and cultural domain [3], an environment foremost represented by males in this case.

n/a

The recent paper “Reward is Enough” by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial, and provides a suitable basis for the creation of artificial general intelligence. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and computational intelligence, and argue in favour of explicitly multi-objective models of reward maximisation. Furthermore, we contend that even if scalar reward functions can trigger intelligent behaviour in specific cases, this type of reward is insufficient for the development of human-aligned artificial general intelligence due to unacceptable risks of unsafe or unethical behaviour.

Programming has been part of Swedish elementary school curriculum for six years and the aim of this full paper is to find out how teachers can design programming activities so that students engage and learn. A mix-methods research project with a social semiotic, multimodal theoretical framework – Designs for learning – is used to investigate teaching and learning in a class during three years. The results in this small-scale study indicate that collaboration is a successful didactic design for programming lessons in school. Computational thinking is prevalent and both digital skills (such as coding) and digital competencies (such as understanding the impact of technology in society) are practiced and met in programming lessons merging Science, Technology, Engineering, Arts, and Mathematics.

Machine learning and other artificial intelligence (AI) technologies are predicted to play a transformative role in primary education, where these technologies for automation and personalization are now being introduced to classroom instruction. This article explores the rationales and practices by which machine learning and AI are emerging in schools. We report on ethnographic fieldwork in Sweden, where a machine learning teaching aid in mathematics, the AI Engine, was tried out by 22 teachers and more than 250 primary education students. By adopting an Actor-Network Theory approach, the analysis focuses on the interactions within the network of heterogeneous actors bound by the AI Engine as an obligatory passage point. The findings show how the actions and accounts emerging within the complex ecosystem of human actors compensate for the unexpected and undesirable algorithmic decisions of the AI Engine. We discuss expectations about AI in education, contradictions in how the AI Engine worked and uncertainties about how machine learning algorithms ‘learn’ and predict. These factors contribute to our understanding of the potential of automation and personalisation—a process that requires continued re-negotiations. The findings are presented in the form of a fictional play in two acts, an ethnodrama. The ethnodrama highlights controversies in the use of AI in education, such as the lack of transparency in algorithmic decision-making—and how this can play out in real-life learning contexts. The findings of this study contribute to a better understanding of AI in primary education.

AI Education for K-12 and in particular AI literacy gained huge interest recently due to the significantly influence in daily life, society, and economy. In this paper we discuss this topic of early AI education along four dimensions: (1) formal versus informal education, (2) cooperation of researchers in AI and education, (3) the level of education, and (4) concepts and tools.

Optimal simultaneous coalition structure generation and assignment is computationally hard. The state-of-the-art can only compute solutions to problems with severely limited input sizes, and no effective approximation algorithms that are guaranteed to yield high-quality solutions are expected to exist. Real-world optimization problems, however, are often characterized by large-scale inputs and the need for generating feasible solutions of high quality in limited time. In light of this, and to make it possible to generate better feasible solutions for difficult large-scale problems efficiently, we present and benchmark several different anytime algorithms that use general-purpose heuristics and Monte Carlo techniques to guide search. We evaluate our methods using synthetic problem sets of varying distribution and complexity. Our results show that the presented algorithms are superior to previous methods at quickly generating near-optimal solutions for small-scale problems, and greatly superior for efficiently finding high-quality solutions for large-scale problems. For example, for problems with a thousand agents and values generated with a uniform distribution, our best approach generates solutions 99.5% of the expected optimal within seconds. For these problems, the state-of-the-art solvers fail to find any feasible solutions at all.

The high operational cost of aircraft, limited availability of air space, and strict safety regulations make training of fighter pilots increasingly challenging. By integrating Live, Virtual, and Constructive simulation resources, efficiency and effectiveness can be improved. In particular, if constructive simulations, which provide synthetic agents operating synthetic vehicles, were used to a higher degree, complex training scenarios could be realized at low cost, the need for support personnel could be reduced, and training availability could be improved. In this work, inspired by the recent improvements of techniques for artificial intelligence, we take a user perspective and investigate how intelligent, learning agents could help build future training systems. Through a domain analysis, a user study, and practical experiments, we identify important agent capabilities and characteristics, and then discuss design approaches and solution concepts for training systems to utilize learning agents for improved training value.

Digital competence and programming are actively highlighted areas in education worldwide. They are becoming part of curricula all over the world, including the Swedish elementary school curriculum, Children are expected to develop computational thinking through programming activities, mainly in mathematics-which are supposed to be based on both proven experience and scientific grounds. Both are lacking in the lower grades of elementary school. This article gives unique insight into pupils learning during the first programming lessons based on a group of Swedish pupils experiences when entering school. The goal of the article is to inform education policy and practice. The large interdisciplinary, longitudinal research project studies approximately 1500 students aged 6-16 and their teachers over three years, using video documentation, questionnaires, and focus group interviews. This article reports on empirical data collected during the first year in one class with 30 pupils aged 6-7 years. The social semiotic, multimodal theoretical framework "Design for Learning" is used to investigate potential signs of learning in pupils multimodal representations when they, for example, use block programming in the primary and secondary transformation unit. We show that young pupils have positive attitudes to programming and high self-efficacy, and that pupils signs of learning in programming are multimodal and often visible in social interactions.

Lattice-based motion planning is a hybrid planning method where a plan is made up of discrete actions, while simultaneously also being a physically feasible trajectory. The planning takes both discrete and continuous aspects into account, for example action pre-conditions and collision-free action-duration in the configuration space. Safe motion planning rely on well-calibrated safety-margins for collision checking. The trajectory tracking controller must further be able to reliably execute the motions within this safety margin for the execution to be safe. In this work we are concerned with introspective learning and reasoning about controller performance over time. Normal controller execution of the different actions is learned using machine learning techniques with explicit uncertainty quantification, for safe usage in safety-critical applications. By increasing the model accuracy the safety margins can be reduced while maintaining the same safety as before. Reasoning takes place to both verify that the learned models stays safe and to improve collision checking effectiveness in the motion planner using more accurate execution predictions with a smaller safety margin. The presented approach allows for explicit awareness of controller performance under normal circumstances, and detection of incorrect performance in abnormal circumstances. Evaluation is made on the nonlinear dynamics of a quadcopter in 3D using simulation.

We present, analyze and benchmark two algorithms for simultaneous coalition structure generation and assignment: one based entirely on dynamic programming, and one anytime hybrid approach that uses branch-and-bound together with dynamic programming. To evaluate the algorithms’ performance, we benchmark them against both CPLEX (an industry-grade solver) and the state-of-the-art using difficult randomized data sets of varying distribution and complexity. Our results show that our hybrid algorithm greatly outperforms CPLEX, pure dynamic programming and the current state-of-the-art in all of our benchmarks. For example, when solving one of the most difficult problem sets, our hybrid approach finds optimum in roughly 0.1% of the time that the current best method needs, and it generates 98% efficient interim solutions in milliseconds in all of our anytime benchmarks; a considerable improvement over what previous methods can achieve.

As the impact and importance of artificial intelligence (AI) grows, there is a growing trend to teach AI in primary and secondary education (K-12). To provide an international perspective, we have conducted three interviews with practitioners and policy makers from AI4K12 in the US (D. Touretzky, C. Gardner-McCune, and D. Seehorn), from Singapore (L. Liew) and from the European Commission (F. Benini).

Reinforcement learning techniques have successfully been applied to solve challenging problems. Among the more famous examples are playing games such as Go and real-time computer games such as StarCraft II. In addition, reinforcement learning has successfully been deployed in cyber-physical systems such as robots playing a curling-based game. These are all important and significant achievements indicating that the techniques can be of value for the aerospace industry. However, to use these techniques in the aerospace industry, very high requirements on verification and validation must be met. In this position paper, we outline four key problems for verification and validation of reinforcement learning techniques. Solving these are an important step towards enabling reinforcement learning techniques to be used in safety critical domains such as the aerospace industry.

Simulation-based training has the potential to significantly improve training value in the air combat domain. However, synthetic opponents must be controlled by high-quality behavior models, in order to exhibit human-like behavior. Building such models by hand is recognized as a very challenging task. In this work, we study how multi-agent deep reinforcement learning can be used to construct behavior models for synthetic pilots in air combat simulation. We empirically evaluate a number of approaches in two air combat scenarios, and demonstrate that curriculum learning is a promising approach for handling the high-dimensional state space of the air combat domain, and that multi-objective learning can produce synthetic agents with diverse characteristics, which can stimulate human pilots in training.

An important research problem in artificial intelligence is how to organize multiple agents, and coordinate them, so that they can work together to solve problems. Coordinating agents in a multi-agent system can significantly affect the systems performance-the agents can, in many instances, be organized so that they can solve tasks more efficiently, and consequently benefit collectively and individually. Central to this endeavor is coalition formation-the process by which heterogeneous agents organize and form disjoint groups (coalitions). Coalition formation often involves finding a coalition structure (an exhaustive set of disjoint coalitions) that maximizes the systems potential performance (e.g., social welfare) through coalition structure generation. However, coalition structure generation typically has no notion of goals. In cooperative settings, where coordination of multiple coalitions is important, this may generate suboptimal teams for achieving and accomplishing the tasks and goals at hand. With this in mind, we consider simultaneously generating coalitions of agents and assigning the coalitions to independent alternatives (e.g., tasks/goals), and present an anytime algorithm for the simultaneous coalition structure generation and assignment problem. This combinatorial optimization problem hasmany real-world applications, including forming goal-oriented teams. To evaluate the presented algorithms performance, we present five methods for synthetic problem set generation, and benchmark the algorithm against the industry-grade solver CPLEXusing randomized data sets of varying distribution and complexity. To test its anytime-performance, we compare the quality of its interim solutions against those generated by a greedy algorithm and pure random search. Finally, we also apply the algorithm to solve the problem of assigning agents to regions in a major commercial strategy game, and show that it can be used in game-playing to coordinate smaller sets of agents in real-time.

Robot safety is of growing concern given recent developments in intelligent autonomous systems. For complex agents operating in uncertain, complex and rapidly-changing environments it is difficult to guarantee safety without imposing unrealistic assumptions and restrictions. It is therefore necessary to complement traditional formal verification with monitoring of the running system after deployment. Runtime verification can be used to monitor that an agent behaves according to a formal specification. The specification can contain safety-related requirements and assumptions about the environment, environment-agent interactions and agent-agent interactions. A key problem is the uncertain and changing nature of the environment. This necessitates requirements on how probable a certain outcome is and on predictions of future states. We propose Probabilistic Signal Temporal Logic (ProbSTL) by extending Signal Temporal Logic with a sub-language to allow statements over probabilities, observations and predictions. We further introduce and prove the correctness of the incremental stream reasoning technique progression over well-formed formulas in ProbSTL. Experimental evaluations demonstrate the applicability and benefits of ProbSTL for robot safety.

Team training in complex domains often requires a substantial amount of resources, e.g., instructors, role-players and vehicles. For this reason, it may be difficult to realize efficient and effective training scenarios in a real-world setting. Instead, intelligent agents can be used to construct synthetic, simulationbased training environments. However, building behavior models for such agents is challenging, especially for the end-users of the training systems, who typically do not have expertise in artificial intelligence. In this PhD project, we study how machine learning can be used to simplify the process of constructing agents for simulation-based training. As a case study we use a simulation-based air combat training system. By constructing smarter synthetic agents the dependency on human training providers can be reduced, and the availability as well as the quality of training can be improved.

In March 2017, the Swedish government decided to introduce digital competence - including programming - in primary school. As a consequence, every math and technology teacher in grades 1-9 in Sweden is expected to integrate programming in their teaching. Furthermore, the Swedish school law requires that teaching is based on scientific evidence and proven experience. In addition to professional development for teachers, it is hence crucial to also conduct research on different aspects of programming in the classroom. In this paper, we describe the process of developing a scientifically grounded instrument for assessing students' programming skills, as part of a longitudinal research project investigating how students in primary school learn programming. We also present the main findings related to the suitability of the instrument based on a pilot study conducted in spring 2019, collecting data from 310 students.

Cyber-physical systems such as robots and intelligent transportation systems are heavy producers and consumers of trajectory data. Making sense of this data and putting it to good use is essential for such systems. When industrial robots, intelligent vehicles and aerial drones are intended to co-exist, side-by-side, with people in human-tailored environments safety is paramount. Safe operations require uncertainty-aware motion pattern recognition, incremental reasoning and rapid decision-making to manage collision avoidance, monitor movement execution and detect abnormal motion. We investigate models and techniques that can support and leverage the interplay between these various trajectory-based capabilities to improve the state-of-the-art for intelligent autonomous systems.

This paper presents preliminary work on using deep neural networksto guide general-purpose heuristic algorithms for performing utilitarian combinatorial assignment. In more detail, we use deep learning in an attempt to produce heuristics that can be used together with e.g., search algorithms to generatefeasible solutions of higher quality more quickly. Our results indicate that ourapproach could be a promising future method for constructing such heuristics.

Stream reasoning can be defined as incremental reasoning over incrementally-available information. The formula progression procedure for Metric Temporal Logic (MTL) makes use of syntactic formula rewritings to incrementally evaluate formulas against incrementally-available states. Progression however assumes complete state information, which can be problematic when not all state information is available or can be observed, such as in qualitative spatial reasoning tasks or in robotics applications. In those cases, there may be uncertainty as to which state out of a set of possible states represents the ‘true’ state. The main contribution of this paper is therefore an extension of the progression procedure that efficiently keeps track of all consistent hypotheses. The resulting procedure is flexible, allowing a trade-off between faster but approximate and slower but precise progression under uncertainty. The proposed approach is empirically evaluated by considering the time and space requirements, as well as the impact of permitting varying degrees of uncertainty.

Validation and verification of supervised machine learning models is becoming increasingly important as their complexity and range of applications grows. This paper describes an extension to Bayesian optimization which allows for selecting both training and validation data, in cases where data can be generated or calculated as a function of a spatial location.

Exploration is an important aspect of robotics, whether it is for mapping, rescue missions or path planning in an unknown environment. Frontier Exploration planning (FEP) and Receding Horizon Next-Best-View planning (RH-NBVP) are two different approaches with different strengths and weaknesses. FEP explores a large environment consisting of separate regions with ease, but is slow at reaching full exploration due to moving back and forth between regions. RH-NBVP shows great potential and efficiently explores individual regions, but has the disadvantage that it can get stuck in large environments not exploring all regions. In this work we present a method that combines both approaches, with FEP as a global exploration planner and RH-NBVP for local exploration. We also present techniques to estimate potential information gain faster, to cache previously estimated gains and to exploit these to efficiently estimate new queries.

The tactical systems and operational environment of modern fighter aircraft are becoming increasingly complex. Creating a realistic and relevant environment for pilot training using only live aircraft is difficult, impractical and highly expensive. The Live, Virtual and Constructive (LVC) simulation paradigm aims to address this challenge. LVC simulation means linking real aircraft, ground-based systems and soldiers (Live), manned simulators (Virtual) and computer controlled synthetic entities (Constructive). Constructive simulation enables realization of complex scenarios with a large number of autonomous friendly, hostile and neutral entities, which interact with each other as well as manned simulators and real systems. This reduces the need for personnel to act as role-players through operation of e.g. live or virtual aircraft, thus lowering the cost of training. Constructive simulation also makes it possible to improve the availability of training by embedding simulation capabilities in live aircraft, making it possible to train anywhere, anytime. In this paper we discuss how machine learning techniques can be used to automate the process of constructing advanced, adaptive behavior models for constructive simulations, to improve the autonomy of future training systems. We conduct a number of initial experiments, and show that reinforcement learning, in particular multi-agent and multi-objective deep reinforcement learning, allows synthetic pilots to learn to cooperate and prioritize among conflicting objectives in air combat scenarios. Though the results are promising, we conclude that further algorithm development is necessary to fully master the complex domain of air combat simulation.

The creation of behavior models for computer generated forces (CGF) is a challenging and time-consuming task, which often requires expertise in programming of complex artificial intelligence algorithms. This makes it difficult for a subject matter expert with knowledge about the application domain and the training goals to build relevant scenarios and keep the training system in pace with training needs. In recent years, machine learning has shown promise as a method for building advanced decision-making models for synthetic agents. Such agents have been able to beat human champions in complex games such as poker, Go and StarCraft. There is reason to believe that similar achievements are possible in the domain of military simulation. However, in order to efficiently apply these techniques, it is important to have access to the right tools, as well as knowledge about the capabilities and limitations of algorithms.   

This paper discusses efficient applications of deep reinforcement learning, a machine learning technique that allows synthetic agents to learn how to achieve their goals by interacting with their environment. We begin by giving an overview of available open-source frameworks for deep reinforcement learning, as well as libraries with reference implementations of state-of-the art algorithms. We then present an example of how these resources were used to build a reinforcement learning environment for a CGF software intended to support training of fighter pilots. Finally, based on our exploratory experiments in the presented environment, we discuss opportunities and challenges related to the application of reinforcement learning techniques in the domain of air combat training systems, with the aim to efficiently construct high quality behavior models for computer generated forces.

n/a

Agent-based simulation is a powerful tool for studying complex systems of interacting agents. To achieve good results, the behavior models used for the agents must be of high quality. Traditionally these models have been handcrafted by domain experts. This is a difficult, expensive and time consuming process. In contrast, reinforcement learning allows agents to learn how to achieve their goals by interacting with the environment. However, after training the behavior of such agents is often static, i.e. it can no longer be affected by a human. This makes it difficult to adapt agent behavior to specific user needs, which may vary among different runs of the simulation. In this paper we address this problem by studying how multi-objective reinforcement learning can be used as a framework for building tunable agents, whose characteristics can be adjusted at runtime to promote adaptiveness and diversity in agent-based simulation. We propose an agent architecture that allows us to adapt popular deep reinforcement learning algorithms to multi-objective environments. We empirically show that our method allows us to train tunable agents that can approximate the policies of multiple species of agents.

A fundamental problem in artificial intelligence is how to organize and coordinate agents to improve their performance and skills. In this paper, we consider simultaneously generating coalitions of agents and assigning the coalitions to independent tasks, and present an anytime algorithm for the simultaneous coalition structure generation and assignment problem. This optimization problem has many real-world applications, including forming goal-oriented teams of agents. To evaluate the algorithm’s performance, we extend established methods for synthetic problem set generation, and benchmark the algorithm against CPLEX using randomized data sets of varying distribution and complexity. We also apply the algorithm to solve the problem of assigning agents to regions in a major commercial strategy game, and show that the algorithm can be utilized in game-playing to coordinate smaller sets of agents in real-time.

The Swedish government has recently introduced digital competence including programming in the Swedish K-9 curriculum starting no later than fall 2018. This means that 100 000 teachers need to learn programming and digital competence in less than a year. In this paper we report on our experience working with professional teacher training in Sweden's fifth largest city. The city has about 150 000 inhabitants and about 50 schools with about 14 000 students in primary education. The project has been carried out in close cooperation with the municipality.

The work started in the fall of 2014 with a pilot study with 10 teachers in different subjects that was carried out during spring 2015. The pilot study was successful as the teachers were able to introduce activities related to programming and computational thinking in their subjects after only two half day workshops. The next step was to scale this up to include all the schools in the city. As expected, this turned out to be a larger challenge. More than 70 teachers were involved in the second part of the project. Some of the lessons learned are that it is quite easy to provide teacher training, but harder to get teachers to actually change their teaching and even more challenging to get teachers to help their colleagues introduce programming or computational thinking in their teaching.

Based on our experience we draw some general conclusions and make suggestions for how to scale up the teaching of programming and computational thinking to all.

Conventional trajectory-based vehicular traffic analysis approaches work well in simple environments such as a single crossing but they do not scale to more structurally complex environments such as networks of interconnected crossings (e.g. urban road networks). Local trajectory models are necessary to cope with the multi-modality of such structures, which in turn introduces new challenges. These larger and more complex environments increase the occurrences of non-consistent lack of motion and self-overlaps in observed trajectories which impose further challenges. In this paper we consider the problem of motion pattern recognition in the setting of sequential local motion pattern models. That is, classifying sub-trajectories from observed trajectories in accordance with which motion pattern that best explains it. We introduce a Gaussian process (GP) based modeling approach which outperforms the state-of-the-art GP based motion pattern approaches at this task. We investigate the impact of varying local model overlap and the length of the observed trajectory trace on the classification quality. We further show that introducing a pre-processing step filtering out stops from the training data significantly improves the classification performance. The approach is evaluated using real GPS position data from city buses driving in urban areas for extended periods of time.

The formula progression procedure for Metric Temporal Logic (MTL), originally proposed by Bacchus and Kabanza, makes use of syntactic formula rewritings to incrementally evaluate MTL formulas against incrementally-available states. Progression however assumes complete state information, which can be problematic when not all state information is available or can be observed, such as in qualitative spatial reasoning tasks or in robot applications. Our main contribution is an extension of the progression procedure to handle partial state information. For each missing truth value, we efficiently consider all consistent hypotheses by branching progression for each such hypothesis. The resulting procedure is flexible, allowing a trade-off between faster but approximate and slower but precise partial-state progression.

In March 2017, the Swedish government decided to introduce digital competence - including programming - in primary school. As a consequence, every math and technology teacher in grades 1-9 in Sweden is expected to integrate programming in their teaching. Furthermore, the Swedish school law requires that teaching is based on scientific evidence and proven experience. In addition to professional development for teachers, it is hence crucial to also conduct research on different aspects of programming in the classroom. In this paper, we describe the process of developing a scientifically grounded instrument for assessing students' programming skills, as part of a longitudinal research project investigating how students in primary school learn programming. We also present the main findings related to the suitability of the instrument based on a pilot study conducted in spring 2019, collecting data from 310 students.

A key requirement of autonomous vehicles is the capability to safely navigate in their environment. However, outside of controlled environments, safe navigation is a very difficult problem. In particular, the real-world often contains both complex 3D structure, and dynamic obstacles such as people or other vehicles. Dynamic obstacles are particularly challenging, as a principled solution requires planning trajectories with regard to both vehicle dynamics, and the motion of the obstacles. Additionally, the real-time requirements imposed by obstacle motion, coupled with real-world computational limitations, make classical optimality and completeness guarantees difficult to satisfy. We present a unified optimization-based motion planning and control solution, that can navigate in the presence of both static and dynamic obstacles. By combining optimal and receding-horizon control, with temporal multi-resolution lattices, we can precompute optimal motion primitives, and allow real-time planning of physically-feasible trajectories in complex environments with dynamic obstacles. We demonstrate the framework by solving difficult indoor 3D quadcopter navigation scenarios, where it is necessary to plan in time. Including waiting on, and taking detours around, the motions of other people and quadcopters.

Groups of agents in multi-agent systems may have to cooperate to solve tasks efficiently, and coordinating such groups is an important problem in the field of artificial intelligence. In this paper, we consider the problem of forming disjoint coalitions and assigning them to independent tasks simultaneously, and present an anytime algorithm that efficiently solves the simultaneous coalition structure generation and task assignment problem. This NP-complete combinatorial optimization problem has many real-world applications, including forming cross-functional teams aimed at solving tasks. To evaluate the algorithm's performance, we extend established methods for synthetic problem set generation, and benchmark the algorithm using randomized data sets of varying distribution and complexity. Our results show that the presented algorithm efficiently finds optimal solutions, and generates high quality solutions when interrupted prior to finishing an exhaustive search. Additionally, we apply the algorithm to solve the problem of assigning agents to regions in a commercial computer-based strategy game, and empirically show that our algorithm can significantly improve the coordination and computational efficiency of agents in a real-time multi-agent system.

The role of computer science and IT in Swedish schools hasvaried throughout the years. In fall 2014, the Swedish government gavethe National Agency for Education (Skolverket) the task of preparing aproposal for K-9 education on how to better address the competencesrequired in a digitalized society. In June 2016, Skolverket handed overa proposal introducing digital competence and programming as interdisciplinarytraits, also providing explicit formulations in subjects such asmathematics (programming, algorithms and problem-solving), technology(controlling physical artifacts) and social sciences (fostering awareand critical citizens in a digital society). In March 2017, the governmentapproved the new curriculum, which needs to be implemented by fall 2018 at the latest. We present the new K-9 curriculum and put it ina historical context. We also describe and analyze the process of developingthe revised curriculum, and discuss some initiatives for how toimplement the changes.

This is the application for the RoboCup 2017 Standard Platform League from the Link¨oping Humanoids team

Linköping Humanoids participated in both RoboCup 2015 and 2016 with the intention of incrementally developing a good team by learning as much as  possible. We significantly improved from 2015 to 2016, even though we still didn’t perform very well. Our main challenge is that we are building our software from the ground up using the Robot Operating System (ROS) as the integration and development infrastructure. When the system became overloaded, the ROS infrastructure became very unpredictable. This made it very hard to debug during the contest, so we basically had to remove things until the load was constantly low. Our top priority has since been to make the system stable and more resource efficient. This will take  us to the next level.

From the start we have been clear that our goal is to have a competitive team by 2017 since we are developing our own software from scratch we are very well aware that we needed time to build up the competence and the software infrastructure. We believe we are making good progress towards this goal. The team of about 10 students has been very actively working during the fall with weekly workshops and bi-weekly one day hackathons.