Asymptotic theory is one of the core subjects in system identification theory and often used to assess properties of model estimators. In this paper, we focus on the asymptotic theory for the kernel-based regularized system identification and study the convergence in distribution of the generalized cross validation (GCV) based hyper-parameter estimator. It is shown that the difference between the GCV based hyper-parameter estimator and the optimal hyper-parameter estimator that minimizes the mean square error scaled by 1/root N converges in distribution to a zero mean Gaussian distribution, where N is the sample size and an expression of covariance matrix is obtained. In particular, for the ridge regression case, a closed-form expression of the variance is obtained and shows the influence of the limit of the regression matrix on the asymptotic distribution. For illustration, Monte Carlo numerical simulations are run to test our theoretical results.