Metal halide perovskites have emerged as one of the most promising materials for photovoltaics (PVs), with power conversion efficiency of over 22% already demonstrated. In order to compete with traditional crystalline silicon PV, cost and stability are equally important issues that need to be considered besides efficiency. Copper thiocyanate (CuSCN) is an interesting candidate to be used as an inexpensive, thermally stable p-type charge conducting material in perovskite solar cells. Here, we report 13% efficient perovskite solar cells employing CuSCN as the hole-transport material. We compare the stability of cells employing CuSCN with those employing the archetypical organic hole-transporter 2,2 ,7,7 -Tetrakis (N,N-di-p-methoxyphenyl-amine) 9,9-Spirobifluorene (Spiro-OMeTAD), under elevated temperature in ambient atmosphere. Surprisingly, we find that the devices employing CuSCN degrade faster under elevated temperatures than the devices employing SpiroOMeTAD. We discover that an interfacial degradation mechanism occurs at the heterojunction between the perovskite absorber and the CuSCN, even in a dry nitrogen atmosphere, identifying the presence of a critical instability. Interestingly, with the additional coating of the completed cells with a thin film of insulating poly(methyl methacrylate) (PMMA), functioning as a rudimentary "on-cell" encapsulation, we significantly alleviate this issue and deliver efficient perovskite solar cells which survive for more than 1000 hours at 85 degrees C in air with only 25% degradation in performance. Beyond identifying a critical area to address in order to enable CuSCN to be useful for long term operation in perovskite solar cells, our findings indicate that the role of the "encapsulant" is to both keep the environment out, and keep degradation products within the cell.