It is often beneficial for an autonomous agent that operates in a complex environment to make use of different types of mathematical models to keep track of unobservable parts of the world or to perform prediction, planning and other types of reasoning. Since a model is always a simplification of something else, there always exists a tradeoff between the model’s accuracy and feasibility when it is used within a certain application due to the limited available computational resources. Currently, this tradeoff is to a large extent balanced by humans for model construction in general and for autonomous agents in particular. This thesis investigates different solutions where such agents are more responsible for balancing the tradeoff for models themselves in the context of interleaved task planning and plan execution. The necessary components for an autonomous agent that performs its abstractions and constructs planning models dynamically during task planning and execution are investigated and a method called DARE is developed that is a template for handling the possible situations that can occur such as the rise of unsuitable abstractions and need for dynamic construction of abstraction levels. Implementations of DARE are presented in two case studies where both a fully and partially observable stochastic domain are used, motivated by research with Unmanned Aircraft Systems. The case studies also demonstrate possible ways to perform dynamic abstraction and problem model construction in practice.

The problem of automatically selecting simulation models for autonomous agents depending on their current intentions and beliefs is considered in this paper. The intended use of the models is for prediction, filtering, planning and other types of reasoning that can be performed with Simulation models. The parameters and model fragments of the resulting model are selected by formulating and solving a hybrid constrained optimization problem that captures the intuition of the preferred model when relevance information about the elements of the world being modelled is taken into consideration. A specialized version of the original optimization problem is developed that makes it possible to solve the continuous subproblem analytically in linear time. A practical model selection problem is discussed where the aim is to select suitable parameters and models for tracking dynamic objects. Experiments with randomly generated problem instances indicate that a hillclimbing search approach might be both efficient and provides reasonably good solutions compared to simulated annealing and hillclimbing with random restarts.

One of the most successful methods for planning in large partially observable stochastic domains is depth-limited forward search from the current belief state together with a utility estimation. However, when the environment is continuous and the number of possible actions is practically infinite, then abstractions have to be made before any forward search planning can be performed. The paper presents a method to dynamically generate such planning problem abstractions for a domain that is inspired by our research with unmanned aerial vehicles (UAVs). The planning problems are created by first stating the selection of points to fly to as an optimization problem. When the points have been selected, a set of possible paths between them are then created with a pathplanner and then forward search in the belief state space is applied. The method has been implemented and tested in simulation and the experiments show the importance of modelling both the dynamics of the environment and the limited computational resources of the architecture when searching for suitable parameters in the planning problem formulation procedure.

Most of today’s autonomous problem solving agents perform their task with the help of problem domain specifications that keep their abstractions fixed. Those abstractions are often selected by human users. We think that the approach with fixed-abstraction domain specifications is very inflexible because it does not allow the agent to focus its limited computational resources on what may be most relevant at the moment. We would like to build agents that dynamically find suitable abstractions depending on relevance for their current task and situation. This idea of dynamic abstraction has recently been considered an important research problem within the area of hierarchical reinforcement learning [1].