Objective: Sounds in the daily environment may cause loudness and annoyance. The present study investigated the perception of loudness and annoyance for eight different sounds present in a daily sound environment and at nine different levels varying by +/- 20 dB around the recorded level. The outcomes were related to tests of participants auditory and cognitive abilities. Design: The participants undertook auditory and working memory (WM) tests prior to ratings of everyday sounds previously shown to be disturbing for persons with hearing impairment (hearing aid users). Study sample: Twenty-one participants aged between 24 and 71 years, with normal hearing threshold levels. Results: Both perceived loudness and annoyance were primarily driven by the sound level. Sounds emitted from paper were rated as having greater loudness and being more annoying than the other sound sources at the same sound level. Auditory and cognitive abilities did not influence the perception of loudness and annoyance. Conclusions: Loudness and annoyance ratings were mainly driven by sound level. Expectations of a sound seemed to influence the assessment of loudness and annoyance while auditory performance and WM capacity showed no influence on the ratings.