Result: Kernel-based Regularized Estimators: Theoretical Insights and New Estimators with Improved Accuracy ⋆

This paper considers finite impulse response models and focuses on kernel-based regularized estimators. Although regularized estimators often achieve a better bias-variance trade-off than the maximum likelihood estimator and have drawn increasing attention, their finite-sample statistical properties, e.g., the mean squared error (MSE), are analytically intractable. To bypass this issue, we shift our focus to large-sample scenarios, employing the excess MSE, a high-order asymptotic quality measure, in the analysis. Based on the explicit expressions for the excess MSE of three commonly used regularized estimators, we discern two factors influencing performance: the alignment between the true parameter vector and the kernel matrix, and the number of hyper-parameters. We also analyze their quantitative influence to provide new theoretical insights. Moreover, we propose new estimators: a generalized Bayes estimator and three regularized estimators using scaled hyper-parameter estimators, which asymptotically dominate existing regularized estimators. Numerical results are provided to show the improved accuracy of these new estimators.