Enhanced regulations, centralized dismantling processes, and sophisticated recycling technologies have been implemented in some e-waste dismantling areas in China with regard to environmental and economic aspects since 2005. In this study, rice grain samples were collected from 2006 to 2010 in an e-waste dismantling area to investigate the temporal trends and spatial distribution of As, Cd, Cu, and Pb. Geometric means of As, Cd, Cu, and Pb in rice samples from the e-waste dismantling area were 111, 217, 4676, and 237 ng g(-1), respectively. Levels of Pb showed a significant decreasing trend during the sampling period, whereas the other three elements remained relatively constant or even increased. Concentrations of Cd, Cu, and Pb in the e-waste dismantling area were significantly higher than those in the non-e-waste dismantling area (p < 0.05), which showed a close connection between e-waste dismantling activities and elevated Pb, Cu, and Cd contents. Risk assessment for human via rice consumption indicated that over 60% of the hazard quotient of Cd exceeded 1 in the e-waste dismantling area. Our study implied that stricter implementation of regulatory measures might lead to positive effects in controlling the release of some heavy metals to the environment. However, environmental behaviors differed with geochemical characteristics of individual elements. Further remediation actions to reduce heavy metal pollution to the surrounding environment might still be needed.