Abstract
Deep learning is revolutionizing all areas of data science, including human factors research. Much of human factors data, however, have fundamental idiosyncrasies that make applying deep learning challenging. Further, the complexity of deep learning can make finding errors challenging and deducing what was actually learned by the model near impossible. This paper provides two case-studies in which our research group faced and overcame such challenges. It examines the root causes of each issue and discusses how they may lead to common challenges. We describe how we discovered problems and describe how we rectified them. It is our hope, that by sharing our experiences with likely common challenges, we can help other researchers in avoiding similar pitfalls.
The views, opinions and/or findings expressed are those of the authors and should not be inter-preted as representing the official views or policies of the Department of Defense or the U.S. Government.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
As reported by Google Scholar on January 15, 2021.
References
Bernard, M.: 27 incredible examples of AI and machine learning in practice, Forbes, 04 (2018). https://www.forbes.com/sites/bernardmarr/2018/04/30/27-incredible-examples-of-ai-and-machine-learning-in-practice. Accessed 01 2021
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, MIT Press, Cambridge (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)
Stisen, A., et al.: Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recog. In: ACM Embedded Networked Sensor System (2015)
Vaizman, Y., et al.: Recognizing detailed human context in-the-wild from smartphones and smartwatches. IEEE Pervasive Comput. 16(4), 62–74 (2017)
Ehatisham-Ul-Haq, M., Azam, M.A., Amin, Y., Naeem, U.: C2FHAR: Coarse-to-fine human activity recognition with behavioral context modeling using smart inertial sensors. IEEE Access 8, 7731–7747 (2020)
Ehatisham-ul-Haq, M., et al.: Using smartphone accelerometer for human physical activity and context recognition in-the-wild. Proc. Comp. Sci. 177, 24–31 (2020)
Fazli, M., Kowsari, K., Gharavi, E., Barnes, L., Doryab, A.: HHAR-net: hierarchical human activity recognition using neural networks, arXiv:2010.16052 [cs, eess, stat] (2020)
Imani, M., et al.: A framework for collaborative learning in secure high-dimensional space. In: IEEE 12th International Conference on Cloud Computing (CLOUD) (2019)
Nguyen, K.T., et al.: Dealing with imbalanced data sets for human activity recognition using mobile phone sensors. In: 3rd International Workshop on Smart Sensing Systems (2018)
Reyes-Ortiz, J.-L., Oneto, L., Samá, A., Parra, X., Anguita, D.: Transition-aware human activity recognition using smartphones. Neurocomputing 171, 754–767 (2016)
Yao, S., et al.: DeepSense: a unified deep learning framework for time-series mobile sensing data processing. In: 26th International Conference on World Wide Web, pp. 351–360 (2017)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, Curran Associates, Inc., pp. 2672–2680 (2014)
Chen, X., et al.: InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In: Neural Information Processing System, pp. 2180–2188 (2016)
Spurr, A., et al.: Guiding InfoGAN with Semi-supervision. In: Machine Learning and Knowledge Discovery in Databases, Springer International Publishing, pp. 119–134 (2017)
Baldwin, J., Burnham, R., Meyer, A., Dora, R., Wright, R.: Beyond speech: generalizing D-vectors for biometric verification. In: Proceedings of AAAI 2019, Honolulu, HI (2019)
Vaizman, Y., et al.: ExtraSensory app: data collection in-the-wild with rich user interface to self-report behavior. In: Conference on Human Factors in Computing System, pp. 1–12 (2018)
Mansoor, H., et al.: COMEX: identifying mislabeled human behavioral context data using visual analytics. In: IEEE Computer Software and Applications Conference (2019)
Mansoor, H. et al.: DELFI: mislabelled human context detection using multi-feature similarity linking. In IEEE Visualization in Data Science (VDS) (2019)
Adams, R., Marlin, B.: Learning time series segmentation models from temporally imprecise labels. In: Conference on Uncertainty in Artificial Intelligence (2018)
Imani, M., et al.: QuantHD: a quantization framework for hyperdimensional computing. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 39(10), 2268–2278 (2020)
Bosch, S., et al.: QubitHD: a stochastic acceleration method for HD computing-based machine learning, arXiv:1911.12446 [cs, stat] (2019)
Acknowledgments
We would like to acknowledge and thank DARPA for funding this research under contract #FA8750–18-C-0056. Further, we would like to thank our teammates at Charles River Analytics.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Weigel, B., Loar, K., Colón, A., Wright, R. (2021). What Did Our Model Just Learn? Hard Lessons in Applying Deep Learning to Human Factors Data. In: Ayaz, H., Asgher, U., Paletta, L. (eds) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2021. Lecture Notes in Networks and Systems, vol 259. Springer, Cham. https://doi.org/10.1007/978-3-030-80285-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-80285-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80284-4
Online ISBN: 978-3-030-80285-1
eBook Packages: EngineeringEngineering (R0)