Abstract
Recently, a filter supervised feature selection method, namely discriminative feature selection (DFS), was proposed, which combines linear discriminant analysis (LDA) and sparsity regularization effectively. However, DFS is computationally expensive due to the use of eigenvalue decomposition on large matrix. In this paper, we propose a two-stage DFS method, namely TSDFS, to improve the efficiency and keep the accuracy of classification. A direct LDA based feature selection is firstly performed to achieve dimension reduction preprocessing of the data. Then, a DFS procedure is performed efficiently on the reduced data in the second stage. The high efficiency of the whole TSDFS is credited with the high efficiency of dimension reduction preprocessing. In addition, we employ nested cross-validation technology to achieve automatic parameter selection for TSDFS. Extensive experimental results based on several benchmark data sets validate the effectiveness of TSDFS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)
Eroglu, D.Y., Kilic, K.: A novel Hybrid Genetic Local Search Algorithm for feature selection and weighting with an application in strategic decision making in innovation management. Inf. Sci. 405, 18–32 (2017)
He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Proceedings of the 18th International Conference on Neural Information Processing Systems, NIPS 2005, pp. 507–517 (2005)
Masaeli, M., Fung, G., Dy, J.G.: From transformation-based dimensionality reduction to feature selection. In: Proceedings of 27th International Conference on Machine Learning, ICML 2010, pp. 751–758. Omnipress, Haifa, Israel (2010)
Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: Proceedings of 23rd AAAI 2008 Conference on Artificial Intelligence, vol. 2, pp. 671–676. AAAI Press, Chicago, Illinois (2008)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)
Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53(1–2), 23–69 (2003)
Statnikov, A., Tsamardinos, I., Dosbayev, Y., Aliferis, C.F.: GEMS: a system for automated cancer diagnosis and biomarker discovery from microarray gene expression data. Int. J. Med. Inform. 74(7–8), 491–503 (2005)
Tao, H., Hou, C., Nie, F., Jiao, Y., Yi, D.: Effective discriminative feature selection with nontrivial solution. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 796–808 (2016)
Wang, L., Chu, F., Xie, W.: Accurate cancer classification using expressions of very few genes. IEEE/ACM Trans. Comput. Biol. Bioinform. 4(1), 40–53 (2007)
Wang, L., Wang, Y., Chang, Q.: Feature selection methods for big data bioinformatics: a survey from the search perspective. Methods 111, 21–31 (2016)
Wang, L., Zhou, N., Chu, F.: A general wrapper approach to selection of class-dependent features. IEEE Trans. Neural Netw. 19(7), 1267–1278 (2008)
Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)
Ye, J., Li, Q.: A two-stage linear discriminant analysis via QR-decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 929–941 (2005)
Zhao, M., Lin, M., Chiu, B., Zhang, Z., Tang, X.: Trace ratio criterion based discriminative feature selection via \(l_{2, p}\)-norm regularization for supervised learning. Neurocomputing 321, 1–16 (2018)
Acknowledgements
This work is supported by the National Science Foundation of China (Grant nos. 61671377, 61102095, 61571361 and 11401045), and the Science Plan Foundation of the Education Bureau of Shanxi Province (No. 18JK0719), and New Star Team of Xi’ an University of Posts and Telecommunications (Grant no. xyt2016-01).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhi, X., Wu, S. (2020). Two-Stage Discriminative Feature Selection. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1075. Springer, Cham. https://doi.org/10.1007/978-3-030-32591-6_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-32591-6_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32590-9
Online ISBN: 978-3-030-32591-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)