Skip to main content
SpringerLink
Log in

Non-negative matrix factorization based modeling and training algorithm for multi-label learning

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Multi-label learning is more complicated than single-label learning since the semantics of the instances are usually overlapped and not identical. The effectiveness of many algorithms often fails when the correlations in the feature and label space are not fully exploited. To this end, we propose a novel non-negative matrix factorization (NMF) based modeling and training algorithm that learns from both the adjacencies of the instances and the labels of the training set. In the modeling process, a set of generators are constructed, and the associations among generators, instances, and labels are set up, with which the label prediction is conducted. In the training process, the parameters involved in the process of modeling are determined. Specifically, an NMF based algorithm is proposed to determine the associations between generators and instances, and a non-negative least square optimization algorithm is applied to determine the associations between generators and labels. The proposed algorithm fully takes the advantage of smoothness assumption, so that the labels are properly propagated. The experiments were carried out on six set of benchmarks. The results demonstrate the effectiveness of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang M L, Zhou Z H. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(8): 1819–1837

    Article  Google Scholar 

  2. Zhang M L, Zhou Z H. Milti-label neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 2006, 18(10): 1338–1351

    Article  Google Scholar 

  3. Lo H, Wang J, Wang H, Lin S. Cost sensitive multi-label learning for audio tag annotation and retrieval. IEEE Transactions on Multimedia, 2011, 13(3): 518–529

    Article  Google Scholar 

  4. Sanden C, Zhang J. Enhancing multi-label music genre classification through ensemble techniques. In: Proceedings of the 34th International ACMSIGIR Conference on Research and Development in Information Retrieval. 2011, 705–714

    Google Scholar 

  5. Tang L, Rajan S, Narayanan V. Large scale multi-label classification via metalabeler. In: Proceedings of the 19th International Conference on World Wide Web. 2009, 211–220

    Chapter  Google Scholar 

  6. Gopal S, Yang Y. Multi-label classification with meta-level features. In: Proceedings of the 33rd International ACM SIGIR Conference on Research & Development in Information Retrieval. 2010, 315–322

    Google Scholar 

  7. Zhu X, Ghahramani Z. Learning from labeled and unlabeled data with label propagation. Technical Report, 2002

    Google Scholar 

  8. Read J, Martino L, Olmos P M, Luengo D. Scalable multi-output label prediction: from classifier chains to classifier trellises. Pattern Recognition, 2015, 48(6): 2096–2109

    Article  MATH  Google Scholar 

  9. Madjarov G, Gjorgjevikj D, Dzeroski S. Two stage architecture for multi-label learning. Pattern Recognition, 2012, 45(3): 1019–1034

    Article  Google Scholar 

  10. Teisseyre P. Feature ranking for multi-label classification usingMarkov networks. Neurocomputing, 2015, 205: 439–454

    Article  Google Scholar 

  11. Lee J, Kim D W. Memetic feature selection algorithm for multi-label classification. Information Sciences, 2015, 293: 80–96

    Article  Google Scholar 

  12. Zhu X, Lafferty J, Rosenfeld R. Semi-supervised learning with graphs. Carnegie Mellon University, Doctor Thesis, 2005

    Google Scholar 

  13. Chapelle O, Schlkopf B, Zien A. Semi-Supervised Learning. MA: The MIT Press, 2006

    Book  Google Scholar 

  14. Hou P, Geng X, Zhang M L. Multi-label manifold learning. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016, 1680–1686

    Google Scholar 

  15. Gao N N, Huang S J, Chen S C. Multi-label active learning by model guided distribution matching. Frontiers of Computer Science, 2016, 10(5): 845–855

    Article  Google Scholar 

  16. Kong X, Ng M K, Zhou Z H. Transductive multi-label learning via label set propagation. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(3): 704–719

    Article  Google Scholar 

  17. Huang S J, Yu Y, Zhou Z H. Multi-label hypothesis reuse. In: Proceedings of the 18th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2012, 525–533

    Google Scholar 

  18. Huang S J, Zhou Z H. Multi-label learning by exploiting label correlations locally. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence. 2012, 949–955

    Google Scholar 

  19. Lo H, Lin S,Wang H. Generalized k-label sets ensemble for multi-label and cost-sensitive classification. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1679–1691

    Article  Google Scholar 

  20. Lee J, Kim K, Kim N, Lee J H. An approach for multi-label classification by directed acyclic graph with label correlation maximization. Information Sciences, 2016, 351: 101–114

    Article  Google Scholar 

  21. Li P, Li H, Wu M. Multi-label ensemble based on variable pairwise constraint projection. Information Sciences, 2013, 222: 269–281

    Article  MathSciNet  Google Scholar 

  22. Wang B, Tsotsos J. Dynamic label propagation for semi-supervised multi-class multi-label classification. Pattern Recognition, 2016, 52: 75–84

    Article  Google Scholar 

  23. Wang S, Wang J, Wang Z, Ji Q. Enhancing multi-label classification by modeling dependencies among labels. Pattern Recognition, 2014, 47(10): 3405–3413

    Article  Google Scholar 

  24. Pillai I, Fumera G, Roli F. Multi-label classification with a reject option. Pattern Recognition, 2013, 46(8): 2256–2266

    Article  Google Scholar 

  25. Sun F, Tang J, Li H, Qi G, Huang T S. Multi-label image categorization with sparse factor representation. IEEE Transactions on Image Processing, 2014, 23(3): 1028–1037

    Article  MATH  MathSciNet  Google Scholar 

  26. Zhang Y, Zhou Z H. Multi-label dimensionality reduction via dependence maximization. ACM Transactions on Knowledge Discovery from Data, 2010, 4(3): 14

    Article  Google Scholar 

  27. Zhang M L, Zhang K. Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010, 999–1007

    Chapter  Google Scholar 

  28. Zhang M L, Wu L. Lift: multi-label learning with label specific features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 107–120

    Article  Google Scholar 

  29. Triguero I, Vens C. Labelling strategies for hierarchical multi-label classification techniques. Pattern Recognition, 2016, 56: 170–183

    Article  Google Scholar 

  30. Quevedo J R, Luaces O, Bahamonde A. Multilabel classifiers with a probabilistic thresholding strategy. Pattern Recognition, 2012, 45(2): 876–883

    MATH  Google Scholar 

  31. Zhang N, Ding S, Zhang J. Multi layer ELM-RBF for multi-label learning. Applied Soft Computing, 2016, 43: 535–545

    Article  Google Scholar 

  32. Huang Y, Wang W, Wang L. Unconstrained multimodal multi-label learning. IEEE Transactions on Multimedia, 2015, 17(11): 1923–1935

    Article  Google Scholar 

  33. Xu J, Jagadeesh V, Manjunath B S. Multi-label learning with fused multimodal Bi-relational graph. IEEE Transactions on Multimedia, 2014, 16(2): 403–412

    Article  Google Scholar 

  34. Gao W, Zhou Z. On the consistency of multi-label learning. Artificial Intelligence, 2013, 199(1): 22–44

    Article  MATH  MathSciNet  Google Scholar 

  35. Madjarov G, Kocev D, Gjorgjevikj D, Dzeroski S. An extensive experimental comparison of methods for multi-label learning. Pattern Recognition, 2012, 45(9): 3084–3104

    Article  Google Scholar 

  36. Lee D D, Seung H S. Learning the parts of objects by non-negative matrix factorization. Nature, 1999, 401(6755): 788–791

    Article  MATH  Google Scholar 

  37. Lawson C L, Hanson R J. Solving Least Squares Problems. New Jersey: Prentice-Hall, Inc., 1974

    MATH  Google Scholar 

  38. Joachims T. Transductive inference for text classification using support vector machines. In: Proceedings of the 16th International Conference on Machine Learning. 1999, 200–209

    Google Scholar 

  39. Vapnik V N. Statistical Learning Theory. New York: Wiley, 1998

    MATH  Google Scholar 

  40. Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from examples. Journal of Machine Learning Research, 2006, 7: 2399–2434

    MATH  MathSciNet  Google Scholar 

  41. Xu M, Jin R, Zhou Z H. Speedup matrix completion with side information: application to multi-label learning. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2013, 2301–2309

    Google Scholar 

  42. Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multilabel classification. Machine Learning, 2011, 85(3): 333–359

    Article  MathSciNet  Google Scholar 

  43. Wang J, Zhao Y, Wu X, Hua X S. A transductive multi-label learning approach for video concept detection. Pattern Recognition, 2011, 44(10–11): 2274–2286

    Article  MATH  Google Scholar 

  44. Furnkranz J, Hullermeier E, Mencia E L, Brinker K. Multilabel classification via calibrated label ranking. Machine Learning, 2008, 73(2): 133–153

    Article  Google Scholar 

  45. Mencia E L, Park S H, Furnkranz J. Efficient voting prediction for pairwise multi-label classification. Neurocomputing, 2010, 73(7–9): 1164–1176

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the support of the National Natural Science Foundation of China (Grant Nos. 61402076, 61572104, 61103146), the Fundamental Research Funds for the Central Universities (DUT17JC04), and the Project of the Key Laboratory of Symbolic Computation and Knowledge Engineering ofMinistry of Education, Jilin University (93K172017K03).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China

    Liang Sun, Hongwei Ge & Wenjing Kang

Authors
  1. Liang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongwei Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjing Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongwei Ge.

Additional information

Liang Sun received the BE degree in computer science and technology from Xidian University, China in 2003, the MS degree and PhD degree in computer application technology from Jilin University, China in 2006 and 2012, respectively. He is currently with the College of Computer Science and Technology, Dalian university of technology, China. His main research interests lie in machine learning and deep learning.

Hongwei Ge received BS and MS degrees in mathematics from Jilin University, China, and the PhD degree in computer application technology from Jilin University, China in 2006. He is currently a professor and a vice dean in the College of Computer Science and Technology, Dalian University of Technology, China. His research interests are machine learning, computational intelligence, optimization and modeling, computer vision, and deep learning.

Wenjing Kang received the BS degree from Northeast University, China in 2016. She is currently pursuing a master degree in the College of Computer Science and Technology, Dalian University of Technology, China. Her main research interests are deep learning, machine learning applications such as computer vision.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Ge, H. & Kang, W. Non-negative matrix factorization based modeling and training algorithm for multi-label learning. Front. Comput. Sci. 13, 1243–1254 (2019). https://doi.org/10.1007/s11704-018-7452-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-018-7452-y

Keywords

Search

Navigation