There is no consensus on the supply chain management definition of resilience. To aid in evaluating the dynamic behaviour of such systems we need to establish clearly elucidated performance criteria that encapsulate the attributes of resilience. A literature review establishes the latter as readiness, responsiveness and recovery. We also identify robustness as a necessary condition that would complement resilience. We find that the Integral of the Time Absolute Error (ITAE) is an appropriate control engineering measure of resilience when it is applied to inventory levels and shipment rates. We use the ITAE to evaluate an often used benchmark model of make-to-stock supply chains consisting of three decision parameters. We use both linear and nonlinear forms of the model in our evaluation. Our findings suggest that optimum solutions for resilience do not yield a system that is robust to uncertainties in lead-time. Hence supply chains will experience drastic changes in their resilience performance when lead-time changes.