Time resolved photoluminescence spectroscopy is employed to monitor the effect of N incorporation on the band structure of GaNP alloys. Abrupt shortening in radiative lifetime of near-band gap emissions, arising from excitonic radiative recombination within N-related centers, is found to occur at very low N compositions of around 0.5%, i.e., within the same range as the appearance of the direct-band gap-like transitions in the photomodulated transmission spectra of GaNP reported previously. The effect has been attributed to an enhancement in oscillator strength of optical transitions due to band crossover from indirect to direct-band gap of the alloy.