Time-of-use tariffs, which reflect the cost of producing one extra unit of electricity, will be more common in the future. In Sweden the electricity unit price will be high during the winter and cheaper during the summer. A bivalent heating system, where an oil-fired boiler takes care of the peak load, when the electricity price is high, and a heat pump the base load, may decrease the cost of space heating substantially. However, insulation retrofits are also likely to reduce the peak space-heating load in a building. This paper shows how a bivalent heating system can be optimized while also considering the insulation measures. The optimization is elaborated by the use of a mixed integer programming model and the result is compared with a derivative optimization method used in the OPERA (optimal energy retrofit advisory) model. Both models use the life-cycle cost (LCC) as a ranking criterion, i.e. when the lowest LCC for the building is achieved, no better retrofit combination exists for the remaining life of the building.