The rapid advancements in artificial intelligence (AI) are creating both opportunities and challenges. As AI-based solutions become increasingly integrated into complex systems that humans have designed to serve a variety of societal functions, such as critical infrastructure, these systems are becoming more intelligent. This evolution introduces a new landscape for their engineering and management and opens up exciting opportunities for both researchers and practitioners to explore the emerging phenomenon of these complex and increasingly intelligent systems.

This thesis explores the combined demands of criticality and generativity in complex intelligent systems (CoIS). Many of these systems are critical, meaning they must meet stringent requirements for safety, reliability, robustness, and resilience. However, as AI and autonomous systems are integrated into these systems, the generative properties imply that systems evolve in ways that are difficult to predict or control. Generativity can pose challenges to the strict management of criticality, where control is often enforced to ensure the safe, reliable, or resilient functioning of these systems. This study explores these two seemingly contradictory dimensions—criticality and generativity—and their combined engineering and management implications in the emerging CoIS.

The findings of this thesis are based on a case study of a WASP research arena in public safety (WARA-PS), where various AI-based solutions and autonomous systems are being researched and integrated. These research activities aim not only to contribute to public safety systems and applications, but also to impact other fields. The thesis draws on interviews, observations, and archival data of WARA-PS, supplemented by a second study involving key informant interviews on the development of autonomous vehicles.

The contributions of this thesis are as follows. It provides engineering and management characterization of emerging CoIS, including their conceptualization, and implications for complexity and the changing role of system integrators, adding to the existing literature on complex systems. Furthermore, it addresses the engineering and management implications of combining generativity with criticality. The concept of “bounded generativity” is proposed as an approach to managing both criticality demands, and the unpredictable evolution introduced by generativity. Additionally, this thesis also provides insights into the impact of AI and autonomy on resilience, and the role of data in CoIS.