Skip to main content
SpringerLink
Log in

Development of a Neural Network Algorithm to Detect Soldier Load from Environmental Speech

  • Conference paper
  • First Online:
Advances in Simulation and Digital Human Modeling (AHFE 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 264))

Included in the following conference series:

  • 1142 Accesses

Abstract

The objective was to develop a model based on speech input that can identify when team members need adaptive autonomous assistance. Human teams often adjust their behavior to work cohesively and effectively as a team. Similarly, it is beneficial for autonomous agents to be able to adaptively adjust to team needs. We constructed a convolutional recurrent neural network model based on those developed for the recognition of emotion from speech. Audio recordings from a recent field exercise were used to train and validate the model. These data were labeled according to whether the speech occurred during an engagement (engaged, neutral, or no-speech). The model classified more than 99% of the training, validation, and test sets correctly. This information will allow us to design systems in which autonomous agents can prioritize, assist with, and take autonomous control of tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. South, T.: From unmanned teaming to new capabilities, the Army’s Next Generation Combat Vehicle will be a leap forward - Reader Mode. Army Times (2019)

    Google Scholar 

  2. Elfenbein, H.A.: Team emotional intelligence: what it can mean and how it can affect performance. In: Linking Emotional Intelligence and Performance at Work: Current Research Evidence with Individuals and Groups, pp. 165–184. Taylor and Francis (2013). https://doi.org/10.4324/9780203763896

  3. Waldinger, R.J., Hauser, S.T., Schulz, M.S., Allen, J.P., Crowell, J.A.: Reading Others’ Emotions: The Role of Intuitive Judgments in Predicting Marital Satisfaction, Quality, and Stability (2004). https://doi.org/10.1037/0893-3200.18.1.58

  4. Neubauer, C., Chollet, M., Mozgai, S., Dennison, M., Khooshabeh, P., Scherer, S.: The relationship between task-induced stress, vocal changes, and physiological state during a dyadic team task. In: ICMI 2017 - Proceedings of the 19th ACM International Conference on Multimodal Interaction (2017). https://doi.org/10.1145/3136755.3136804

  5. Lingard, L., et al.: Communication failures in the operating room: an observational classification of recurrent types and effects. Qual. Saf. Heal. Care. 13, 330–334 (2004). https://doi.org/10.1136/qshc.2003.008425

    Article  Google Scholar 

  6. Casale, S., Russo, A., Scebba, G., Serrano, S.: Speech emotion classification using machine learning algorithms. In: Proceedings - IEEE International Conference on Semantic Computing 2008, ICSC 2008 (2008). https://doi.org/10.1109/ICSC.2008.43

  7. Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., Schuller, B.: Deep neural networks for acoustic emotion recognition: raising the benchmarks. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (2011). https://doi.org/10.1109/ICASSP.2011.5947651

  8. Koolagudi, S.G., Rao, K.S.: Emotion recognition from speech: a review (2012). https://doi.org/10.1007/s10772-011-9125-1

  9. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  10. Ververidis, D., Kotropoulos, C.: Emotional speech recognition: resources, features, and methods. Speech Commun. (2006). https://doi.org/10.1016/j.specom.2006.04.003

    Article  MATH  Google Scholar 

  11. Schuller, B., Rigoll, G., Lang, M.: Hidden Markov model-based speech emotion recognition. In: Proceedings - IEEE International Conference on Multimedia and Expo (2003). https://doi.org/10.1109/ICME.2003.1220939

  12. El. Ayadi, M., Kamel, M.S., Karray, F.: Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognit. (2011). https://doi.org/10.1016/j.patcog.2010.09.020

    Article  MATH  Google Scholar 

  13. Badshah, A.M., Ahmad, J., Rahim, N., Baik, S.W.: Speech emotion recognition from spectrograms with deep convolutional neural network. In: 2017 International Conference on Platform Technology and Service, PlatCon 2017 - Proceedings (2017). https://doi.org/10.1109/PlatCon.2017.7883728

  14. Lim, W., Jang, D., Lee, T.: Speech emotion recognition using convolutional and recurrent neural networks. In: 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2016 (2017). https://doi.org/10.1109/APSIPA.2016.7820699

  15. Chen, M., He, X., Yang, J., Zhang, H.: 3-D convolutional recurrent neural networks with attention model for speech emotion recognition. IEEE Signal Process. Lett. 25, 1440–1444 (2018). https://doi.org/10.1109/LSP.2018.2860246

    Article  Google Scholar 

  16. Huang, C.W., Narayanan, S.S.: Attention assisted discovery of sub-utterance structure in speech emotion recognition. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pp. 1387–1391. International Speech and Communication Association (2016). https://doi.org/10.21437/Interspeech.2016-448

  17. Lyons, J.: python_speech_features Documentation Release 0.1.0 (2017)

    Google Scholar 

  18. Abadi, M., et al.: TensorFlow: a system for large-scale machine learning (2016)

    Google Scholar 

  19. van Rossum, G., Drake, F.L.: Python 3 Reference Manual (2009)

    Google Scholar 

  20. Scharine, A.A., Schaefer, K.E.: Adapting a Model of Emotional State Recognition to Detect Stress in a High-Noise Environment (ARL-TR-9137), Aberdeen Proving Ground, MD (2021)

    Google Scholar 

  21. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2014)

    Google Scholar 

  22. Schaefer, K.E., et al.: US Army Wingman Joint Capability Technology Demonstration (JCTD): Initial Soldier and Marine Feedback on Manned-Unmanned Gunnery Operations (ARL-TR-8663). Aberdeen Proving Ground, MD (2018)

    Google Scholar 

  23. Brewer, R.W., Cerame, E., Pursel, E.R., Zimmermann, A., Schaefer, K.E.: Manned-unmanned teaming: US army robotic wingman vehicles. In: Advances in Intelligent Systems and Computing (2019). https://doi.org/10.1007/978-3-319-94223-0_9

  24. Baker, A.L., et al.: Team Trust in Human-Autonomy Teams: Analysis of Crew Communication during Manned-Unmanned Gunnery Operations (ARL-TR-8969). Aberdeen Proving Ground, MD (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. U.S. Army Research Laboratory, Human Research and Engineering Directorate, 7101 Mulberry Point Rd, Aberdeen Proving Ground, MD, 21005-5425, USA

    Angelique Scharine

Authors
  1. Angelique Scharine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angelique Scharine .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Orlando, FL, USA

    Julia L. Wright

  2. Soar Tech, Orlando, FL, USA

    Daniel Barber

  3. Department of Product Development, University of Antwerp, Antwerp, Belgium

    Sofia Scataglini

  4. Biomedical Research and Environmental, NASA Lyndon Johnson Space Center, Houston, TX, USA

    Sudhakar L. Rajulu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Scharine, A. (2021). Development of a Neural Network Algorithm to Detect Soldier Load from Environmental Speech. In: Wright, J.L., Barber, D., Scataglini, S., Rajulu, S.L. (eds) Advances in Simulation and Digital Human Modeling. AHFE 2021. Lecture Notes in Networks and Systems, vol 264. Springer, Cham. https://doi.org/10.1007/978-3-030-79763-8_7

Download citation

Publish with us

Policies and ethics

Search

Navigation