Skip to main content
SpringerLink
Log in

Nature Inspired Meta-heuristic Algorithms for Deep Learning: Recent Progress and Novel Perspective

  • Conference paper
  • First Online:
Advances in Computer Vision (CVC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 943))

Included in the following conference series:

Abstract

Deep learning is presently attracting extra ordinary attention from both the industry and the academia. The application of deep learning in computer vision has recently gain popularity. The optimization of deep learning models through nature inspired algorithms is a subject of debate in computer science. The application areas of the hybrid of natured inspired algorithms and deep learning architecture includes: machine vision and learning, image processing, data science, autonomous vehicles, medical image analysis, biometrics, etc. In this paper, we present recent progress on the application of nature inspired algorithms in deep learning. The survey pointed out recent development issues, strengths, weaknesses and prospects for future research. A new taxonomy is created based on natured inspired algorithms for deep learning. The trend of the publications in this domain is depicted; it shows the research area is growing but slowly. The deep learning architectures not exploit by the nature inspired algorithms for optimization are unveiled. We believed that the survey can facilitate synergy between the nature inspired algorithms and deep learning research communities. As such, massive attention can be expected in a near future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943)

    Article  MathSciNet  Google Scholar 

  2. Holland, J.: Adaptation in natural and artificial systems: an introductory analysis with application to biology, Control and artificial intelligence (1975)

    Google Scholar 

  3. Yang, X.-S., Deb, S.: Cuckoo search via lévy flights. In: 2009 World Congress on Nature and Biologically Inspired Computing, NaBIC 2009, pp. 210–214 (2009)

    Google Scholar 

  4. Karaboga, D.: An idea based on honey bee swarm for numerical optimization, Technical report-tr06, Erciyes university, engineering faculty, computer engineering department (2005)

    Google Scholar 

  5. Yang, X.-S., Deb, S., Fong, S., He, X., Zhao, Y.: Swarm intelligence: today and tomorrow. In: 2016 3rd International Conference on Soft Computing and Machine Intelligence (ISCMI), pp. 219–223 (2016)

    Google Scholar 

  6. Chiroma, H., Abdul-kareem, S., Ibrahim, U., Ahmad, I.G., Garba, A., Abubakar, A., et al.: Malaria severity classification through Jordan-Elman neural network based on features extracted from thick blood smear. Neural Netw. World 25, 565 (2015)

    Article  Google Scholar 

  7. Chaoui, H., Ibe-Ekeocha, C.C.: State of charge and state of health estimation for lithium batteries using recurrent neural networks. IEEE Trans. Veh. Technol. 66, 8773–8783 (2017)

    Article  Google Scholar 

  8. Dolezel, P., Skrabanek, P., Gago, L.: Pattern recognition neural network as a tool for pest birds detection. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–6 (2016)

    Google Scholar 

  9. Nie, L., Guan, J., Lu, C., Zheng, H., Yin, Z.: Longitudinal speed control of autonomous vehicle based on a self-adaptive PID of radial basis function neural network. IET Intel. Transp. Syst. 12(6), 485–494 (2018)

    Article  Google Scholar 

  10. Bahrammirzaee, A.: A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems. Neural Comput. Appl. 19, 1165–1195 (2010)

    Article  Google Scholar 

  11. Xu, Y., Cheng, J., Wang, L., Xia, H., Liu, F., Tao, D.: Ensemble one-dimensional convolution neural networks for skeleton-based action recognition. IEEE Sig. Process. Lett. 25(7), 1044–1048 (2018)

    Article  Google Scholar 

  12. Lam, H., Ling, S., Leung, F.H., Tam, P.K.-S.: Tuning of the structure and parameters of neural network using an improved genetic algorithm. In: 2001 The 27th Annual Conference of the IEEE Industrial Electronics Society, IECON 2001, pp. 25–30 (2001)

    Google Scholar 

  13. Chiroma, H., Abdulkareem, S., Abubakar, A., Herawan, T.: Neural networks optimization through genetic algorithm searches: a review. Appl. Math. 11, 1543–1564 (2017)

    Google Scholar 

  14. Karaboga, D., Akay, B., Ozturk, C.: Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. In: International Conference on Modeling Decisions for Artificial Intelligence 2007, pp. 318–329 (2007)

    Google Scholar 

  15. Nawi, N.M., Khan, A., Rehman, M.Z.: A new back-propagation neural network optimized with cuckoo search algorithm. In: International Conference on Computational Science and Its Applications 2013, pp. 413–426 (2013)

    Google Scholar 

  16. Juang, C.-F.: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 34, 997–1006 (2004)

    Article  Google Scholar 

  17. Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)

    Article  MathSciNet  Google Scholar 

  18. Fong, S., Deb, S., Yang, X.-s.: How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics. In: Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, pp. 3–25. Springer (2018)

    Google Scholar 

  19. Papa, J.P., Rosa, G.H., Pereira, D.R., Yang, X.-S.: Quaternion-based deep belief networks fine-tuning. Appl. Soft Comput. 60, 328–335 (2017)

    Article  Google Scholar 

  20. Fister Jr., I., Yang, X.-S., Fister, I., Brest, J., Fister, D.: A brief review of nature-inspired algorithms for optimization, arXiv preprint arXiv:1307.4186 (2013)

  21. Xing, B., Gao, W.-J.: Innovative computational intelligence: a rough guide to 134 clever algorithms. Springer (2014)

    Google Scholar 

  22. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436 (2015)

    Article  Google Scholar 

  23. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intel. 35, 1798–1828 (2013)

    Article  Google Scholar 

  24. Zhang, C., Tan, K.C., Li, H., Hong, G.S.: A cost-sensitive deep belief network for imbalanced classification. IEEE Trans. Neural Netw. Learn. Syst. 28(99), 1–4 (2018)

    Google Scholar 

  25. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J.: A survey on deep learning techniques for image and video semantic segmentation. Appl. Soft. Comput. 70, 41–65 (2018)

    Article  Google Scholar 

  26. Liu, Y., Chen, X., Wang, Z., Wang, Z.J., Ward, R.K., Wang, X.: Deep learning for pixel-level image fusion: recent advances and future prospects. Inf. Fusion 42, 158–173 (2018)

    Article  Google Scholar 

  27. Yaseen, M.U., Anjum, A., Rana, O., Antonopoulos, N.: Deep learning hyper-parameter optimization for video analytics in clouds. IEEE Trans. Syst. Man Cybern: Syst 15(99), 1–12 (2018)

    Google Scholar 

  28. Neterer, J.R., Guzide, O.: Deep learning in natural language processing. Proc. West Va. Acad. Sci. 90(1) (2018)

    Google Scholar 

  29. Tang, M., Gao, H., Zhang, Y., Liu, Y., Zhang, P., Wang, P.: Research on deep learning techniques in breaking text-based captchas and designing image-based captcha. IEEE Trans. Inf. Forensics Secur. 13, 2522–2537 (2018)

    Article  Google Scholar 

  30. Pathak, A.R., Pandey, M., Rautaray, S.: Application of deep learning for object detection. Proc. Comput. Sci. 132, 1706–1717 (2018)

    Article  Google Scholar 

  31. Ji, Y., Liu, L., Wang, H., Liu, Z., Niu, Z., Denby, B.: Updating the Silent Speech Challenge benchmark with deep learning. Speech Commun. 98, 42–50 (2018)

    Article  Google Scholar 

  32. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  33. Papa, J.P., Rosa, G.H., Marana, A.N., Scheirer, W., Cox, D.D.: Model selection for discriminative restricted boltzmann machines through meta-heuristic techniques. J. Comput. Sci. 9, 14–18 (2015)

    Article  Google Scholar 

  34. Papa, J.P., Rosa, G.H., Costa, K.A., Marana, N.A., Scheirer, W., Cox, D.D.: On the model selection of bernoulli restricted Boltzmann machines through harmony search. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1449–1450 (2015)

    Google Scholar 

  35. Rosa, G., Papa, J., Costa, K., Passos, L., Pereira, C., Yang, X.-S.: Learning parameters in deep belief networks through firefly algorithm. In: IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pp. 138–149 (2016)

    Chapter  Google Scholar 

  36. Rodrigues, D., Yang, X.-S., Papa, J.: Fine-tuning deep belief networks using cuckoo search. In: Bio-Inspired Computation and Applications in Image Processing, pp. 47–59 (2017)

    Chapter  Google Scholar 

  37. Ma, M., Sun, C., Chen, X.: Discriminative deep belief networks with ant colony optimization for health status assessment of machine. IEEE Trans. Instrum. Measur. 66, 3115–3125 (2017)

    Article  Google Scholar 

  38. Kuremoto, T., Kimura, S., Kobayashi, K., Obayashi, M.: Time series forecasting using restricted Boltzmann machine. In: International Conference on Intelligent Computing, pp. 17–22 (2012)

    Google Scholar 

  39. Baldominos, A., Saez, Y., Isasi, P.: Evolutionary convolutional neural networks: An application to handwriting recognition. Neurocomputing 283, 38–52 (2018)

    Article  Google Scholar 

  40. Liu, K., Zhang, L.M., Sun, Y.W.: Deep Boltzmann machines aided design based on genetic algorithms. In: Applied Mechanics and Materials, pp. 848–851 (2014)

    Google Scholar 

  41. Levy, E., David, O.E., Netanyahu, N.S.: Genetic algorithms and deep learning for automatic painter classification. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1143–1150 (2014)

    Google Scholar 

  42. Rere, L.R., Fanany, M.I., Arymurthy, A.M.: Simulated annealing algorithm for deep learning. Proc. Comput. Sci. 72, 137–144 (2015)

    Article  Google Scholar 

  43. Fedorovici, L.-O., Precup, R.-E., Dragan, F., David, R.-C., Purcaru, C.: Embedding gravitational search algorithms in convolutional neural networks for OCR applications. In: 7th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI) 2012, pp. 125–130 (2012)

    Google Scholar 

  44. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  45. Chaturvedi, I., Ong, Y.-S., Tsang, I.W., Welsch, R.E., Cambria, E.: Learning word dependencies in text by means of a deep recurrent belief network. Knowl.-Based Syst. 108, 144–154 (2016)

    Article  Google Scholar 

  46. Mannepalli, K., Sastry, P.N., Suman, M.: A novel adaptive fractional deep belief networks for speaker emotion recognition. Alexandria Eng. J. 56(4), 485–497 (2016)

    Article  Google Scholar 

  47. Qiao, J., Wang, G., Li, X., Li, W.: A self-organizing deep belief network for nonlinear system modeling. Appl. Soft Comput. 65, 170–183 (2018)

    Article  Google Scholar 

  48. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Federal College of Education (Technical), Gombe, Nigeria

    Haruna Chiroma

  2. Department of Mathematical Sciences, Abubakar Tafawa Balewa University, Bauchi, Nigeria

    Abdulsalam Ya’u Gital

  3. College of Computer Science and Information Systems, Jazan University, Jazan, Kingdom of Saudi Arabia

    Nadim Rana

  4. Department of Cyber Security Science, Federal University of Technology, Minna, Nigeria

    Shafi’i M. Abdulhamid

  5. Department of Mathematics, Gombe State University, Gombe, Nigeria

    Amina N. Muhammad & Aishatu Yahaya Umar

  6. Department of Computer Science, International Islamic University Malaysia, Gombak, Malaysia

    Adamu I. Abubakar

Authors
  1. Haruna Chiroma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdulsalam Ya’u Gital
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadim Rana
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shafi’i M. Abdulhamid
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amina N. Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aishatu Yahaya Umar
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Adamu I. Abubakar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Haruna Chiroma or Adamu I. Abubakar .

Editor information

Editors and Affiliations

  1. Saga University, Saga, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chiroma, H. et al. (2020). Nature Inspired Meta-heuristic Algorithms for Deep Learning: Recent Progress and Novel Perspective. In: Arai, K., Kapoor, S. (eds) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, vol 943. Springer, Cham. https://doi.org/10.1007/978-3-030-17795-9_5

Download citation

Publish with us

Policies and ethics

Search

Navigation