Single-Cell RNA Sequencing (scRNA-seq) technology has enabled the biological research community to explore gene expression at a single-cell resolution. By studying differences in gene expression, it is possible to differentiate cell clusters and types within tissues. One of the major challenges in a scRNA-seq study is feature selection from high dimensional data. There are several statistical and machine learning methods available to solve this problem but their performances across data sets lack systematic comparison. In this research, we benchmark different penalized regression methods which are suitable for scRNA-seq data. Results on four different scRNA-seq datasets show that Sparse Group Lasso (SGL) implemented by the SGL R pack-age performs better than other methods in terms of area under the receiver operating curve (AUC). The computation time for different methods varies between data sets with SGL having the least average computation time. Based on our findings, we propose a new method for scRNA-seq clustering which applied SGL on a preselected subset of genes. These selected genes are the union of top important genes from the ridge, lasso, elastic net, and droplasso methods. The proposed method demonstrates an improvement in AUC compared to SGL and other methods as well.