The character of the hole states in a pyramidal GaAsAlGaAs quantum dot-in-dot (DiD) heterostructure is shown to be controllable by tailoring the confinement potential shape. The change in ground valence band state from heavy hole like to light hole like is demonstrated by side-view polarization resolved photoluminescence measurements. The experimental findings are supported by three-dimensional numerical model calculations. The results are applicable for polarization control in quantum dot photonic devices. © 2007 American Institute of Physics.