Affect fluctuates in a moment-to-moment fashion, reflecting the continuous relationship between the individual and the environment. Despite substantial research, there remain important open questions regarding how a stream of sensory input is dynamically represented in experienced affect. Here, approaching affect as a temporally dependent process, we show that momentary affect is shaped by a combination of the affective impact of stimuli (i.e., visual images for the current studies) and previously experienced affect. We also found that this temporal dependency is influenced by uncertainty of the affective context. Participants in each trial viewed sequentially presented images and subsequently reported their affective experience, which was modeled based on images normative affect ratings and participants previously reported affect. Study 1 showed that self-reported valence and arousal in a given trial is partly shaped by the affective impact of the given images and previously experienced affect. In Study 2, we manipulated context uncertainty by controlling occurrence probabilities for normatively pleasant and unpleasant images in separate blocks. Increasing context uncertainty (i.e., random occurrence of pleasant and unpleasant images) was associated with increased negative affect. In addition, the relative contribution of the most recent image to experienced pleasantness increased with increasing context uncertainty. Taken together, these findings provide clear behavioral evidence that momentary affect is a temporally dependent and continuous process, which reflects the affective impact of recent input variables and the previous internal state, and that this process is sensitive to the affective context and its uncertainty.