The prospects of using a reconfigurable intelligent surface (RIS) to aid wireless communication systems have recently received much attention. Among the different use cases, the most popular one is where each element of the RIS scatters the incoming signal with a controllable phase-shift, without increasing its power. In prior literature, this setup has been analyzed by neglecting the electromagnetic interference, consisting of the inevitable incoming waves from external sources. In this letter, we provide a physically meaningful model for the electromagnetic interference that can be used as a baseline when evaluating RIS-aided communications. The model is used to show that electromagnetic interference has a non-negligible impact on communication performance, especially when the size of the RIS grows large. When the direct link is present (though with a relatively weak gain), the RIS can even reduce the communication performance. Importantly, it turns out that the SNR grows quadratically with the number of RIS elements only when the spatial correlation matrix of the electromagnetic interference is asymptotically orthogonal to that of the effective channel (including RIS phase-shifts) towards the intended receiver. Otherwise, the SNR only increases linearly.