Binding energy (BE) referencing is critical to the reliability of chemical analysis performed by X-ray photoelectron spectroscopy. Although the procedure is straightforward for metallic samples, no universal solution is available for insulators, wherein a build-up of positive charge during photoemission results in an uncontrolled change in the BE of the core-level peaks. As these peaks are used to assess the chemical bonding, shifts caused by charging lead to problems with spectra interpretation and contribute to an unacceptably large spread in the BE values reported for the same chemical state. It is often unclear which referencing methods should be applied to which sample type and which referencing approaches should be rejected. In this Perspective, we review essential concepts and key experiments related to BE referencing. We discuss energy diagrams and appropriate reference levels for conducting and insulating samples with and without electrical contact with the spectrometer, and we define criteria for the ultimate charge-reference method, using them to evaluate common referencing techniques. Although no method is free of issues, the most popular one, based on the adventitious carbon (AdC), turns out to be the least reliable. In particular, because the vacuum level aligns at the AdC-sample interface, the BE of the reference C 1s peak from AdC is not constant but varies with the sample work function. To rectify the situation, we suggest easy-to-do control experiments that refute the notion that the C 1s peak has constant BE. We further use the framework of energy diagrams to explain the consequences of the vacuum level alignment at the AdC-sample interface for measurements performed in the most common experimental configurations. Finally, we suggest ideas for improving the reliability of chemical analysis to stimulate the development of new referencing standards. Binding energy referencing is critical for the reliability of chemical analysis performed by X-ray photoelectron spectroscopy. This Perspective discusses the essential concepts related to this procedure and formulates the criteria for the ultimate charge reference, using them to expose problems with the existing referencing methods with the goal of stimulating the development of new approaches.