In this paper, we present a new result on robust adaptive dynamic programming for the Linear Quadratic Regulation (LQR) problem, where the linear system is subject to unmatched uncertainty. We assume that the states of the linear system are fully measurable and the matched uncertainty models unmeasurable states with an unspecified dimension. We use the small-gain theorem to give a sufficient condition such that the generated policies in each iteration of on-policy and off-policy routines guarantee robust stability of the overall uncertain system. The sufficient condition can be used to design the weighting matrices in the LQR problem. We use a simulation example to demonstrate the result.