Data-driven fault diagnosis of dynamic systems is complicated by incomplete training data, unknown faults, and overlapping classes. Many existing machine learning models and data-driven classifiers are not expected to perform well if training data is not representative of all relevant fault realizations. In this work, a data-driven model, called a flexi-pipe model, is proposed to capture the variability of data in residual space from a few realizations of each fault class. A diagnosis system is developed as an open set classification algorithm that can handle both incomplete training data and overlapping fault classes. Data from different fault scenarios in an engine test bench is used to evaluate the performance of the proposed methods. Results show that the proposed fault class models generalize to new fault realizations when training data only contains a few realizations of each fault class.