The Direct Weight Optimization (DWO) approach to estimating a regression function and its application to nonlinear system identification has been proposed and developed during the last few years by the authors. Computationally, the approach is typically reduced to a quadratic or conic programming and can be effectively realized. The obtained estimates demonstrate optimality or sub-optimality in a minimax sense w.r.t. the mean-square error criterion under weak design conditions. Here we describe the main ideas of the approach and represent an overview of the obtained results.