Advanced Algorithm for Parameters Estimation of Negative Binomial Distribution with High Dimensional Sparse Group Structure

Abstract

Negative binomial regression is a powerful technique for modeling count data, particularly when dealing with overdispersion. However, estimating the parameters for large-dimensional sparse models is challenging due to the complexity of optimizing the mean and dispersion parameter of the negative binomial distribution. To address this issue, the authors propose a novel approach that employs two iterations of the majorize-minimize (MM) algorithm, one for estimating the dispersion parameter and the other for estimating the mean parameters. These approaches improve the convergence speed and stability of the algorithm. The authors also use group penalty for variable selection, which enhances the accuracy and efficiency of the algorithm. The proposed method provides an explicit solution, simplifies the iteration process, and maintains good stability while ensuring algorithm convergence. Furthermore, the authors apply the proposed algorithm to the zero-inflated model and demonstrate its promising predictive performance on specific data sets. The research has important implications for count data modeling and analysis in various fields, such as data mining, machine learning, and bioinformatics.