This letter investigates an unmanned aerial vehicle (UAV)-assisted data collection strategy where the UAV trajectory is optimally designed to collect status update from several Internet of Things (IoT) nodes, so as to minimize the average Age of Information (AoI). We consider a practical three-dimensional (3D) urban environment, and design the UAV's trajectory by considering the data collection, flight, and energy constraints. Motivated by the critical safety requirements for the UAV, i.e., the energy constraint during the data collection, we exploit the twin delayed deep deterministic policy gradient (TD3) approach by enforcing the safety constraint throughout the training, and propose a Safe-TD3 based trajectory design for average AoI minimization. By evaluating the long-term safety constraint via the integrated cost network, we illustrate the superiority of the proposed Safe-TD3 based trajectory design algorithm over the benchmarks in reducing the safety constraint violations during the training process while achieving a lower average AoI.