Skip to main content
SpringerLink
Log in

Modeling Cognitive Load in Mobile Human Computer Interaction Using Eye Tracking Metrics

  • Conference paper
  • First Online:
Advances in Artificial Intelligence, Software and Systems Engineering (AHFE 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 271))

Included in the following conference series:

Abstract

Modeling cognitive load of user interaction based on ocular parameters have become a dominant method for exploring usability evaluation of interfaces for systems and applications. Growing importance of Artificial Intelligence in Human Computer Interaction (HCI) has proposed many approaches to understand users’ need and enhance human centric method for interface design. In particular, machine learning-based cognitive modeling, using eye tracking parameters have received more attention in the context of smart devices and applications. In this context, this paper aims to model the estimated cognitive load values for each user into different levels of cognition like very high, high, moderate, low, very low etc., while performing different tasks on a smart phone. The study focuses on the use behavioural measures, ocular parameters along with eight traditional machine learning classification algorithms like Decision Tree, Linear Discriminant Analysis, Random Forest, Support Vector Machine, Naïve Bayes, Neural Network, Fuzzy Rules with Weight Factor and K-Nearest Neighbor to model different levels of estimated cognitive load for each participant. The data set for modeling consisted of 250 records, 11 ocular parameters as prediction variables including age and type of task; and three types of classes (2-class, 3-class, 5-class) for classifying the estimated cognitive load for each participant. We noted that, Age, Fixation Count, Saccade Count, Saccade Rate, Average Pupil Dilation are the most important parameters contributing to modeling the estimated cognitive load levels. Further, we observed that, the Decision Tree algorithm achieved highest accuracy for classifying estimated cognitive load values into 2-class (86.8%), 3-class (74%) and 5-class (62.8%) respectively. Finally, from our study, it may be noted that, machine learning is an effective method for predicting 2-class-based (Low and High) cognitive load levels using ocular parameters. The outcome of the study also provides the fact that ageing affects users’ cognitive workload while performing tasks on smartphone.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Moray, N.: Mental Workload: Its Theory and Measurement. Plenum, New York (1979)

    Book  Google Scholar 

  2. Wickens, C.D., Hollands, J.G.: Engineering Psychology and Human Performance, 3rd edn. Prentice Hall, Upper Saddle River (2000)

    Google Scholar 

  3. Galy, E., Cariou, M., Mélan, C.: What is the relationship between mental workload factors and cognitive load types? Int. J. Psychophysiol. 83(3), 269–275 (2012)

    Article  Google Scholar 

  4. Salvucci, D.D., Lee, F.J.: Simple cognitive modeling in a complex cognitive architecture. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 265–272 (2003)

    Google Scholar 

  5. Abdul, A., von der Weth, C., Kankanhalli, M., Lim, B.Y.: COGAM: measuring and moderating cognitive load in machine learning model explanations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2020)

    Google Scholar 

  6. Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)

    Article  Google Scholar 

  7. Brunken, R., Plass, J.L., Leutner, D.: Direct measurement of cognitive load in multimedia learning. Educ. Psychol. 38(1), 53–61 (2003)

    Article  Google Scholar 

  8. Gevins, A., et al.: Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum. Factors 40(1), 79–91 (1998)

    Article  Google Scholar 

  9. Chen, S.: Cognitive load measurement from eye activity: acquisition, efficacy, and real-time system design. University of New South Wales (2014)

    Google Scholar 

  10. Klami, A., Saunders, C., de Campos, T.E., Kaski, S.: Can relevance of images be inferred from eye movements? In: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, pp. 134–140 (2008)

    Google Scholar 

  11. Eivazi, S., Bednarik, R.: Predicting problem-solving behavior and performance levels from visual attention data. In: Proceedings of Workshop on Eye Gaze in Intelligent Human Machine Interaction at IUI, pp. 9–16 (2011)

    Google Scholar 

  12. Li, X., Younes, R., Bairaktarova, D., Guo, Q.: Predicting spatial visualization problems’ difficulty level from eye-tracking data. Sensors 20(7), 1949 (2020)

    Article  Google Scholar 

  13. Behroozi, M., Parnin, C.: Can we predict stressful technical interview settings through eye-tracking? In: Proceedings of the Workshop on Eye Movements in Programming, pp. 1–5 (2018)

    Google Scholar 

  14. Appel, T., et al.: Predicting cognitive load in an emergency simulation based on behavioral and physiological measures. In: 2019 International Conference on Multimodal Interaction, pp. 154–163 (2019)

    Google Scholar 

  15. Joseph, A.W., DV, J.S., Saluja, K.P.S., Mukhopadhyay, A., Murugesh, R., Biswas, P.: Eye tracking to understand impact of aging on mobile phone applications (2021)

    Google Scholar 

  16. Mao, Y., He, Y., Liu, L., Chen, X.: Disease classification based on eye movement features with decision tree and random forest. Front. Neurosci. 14, 798 (2020)

    Article  Google Scholar 

  17. Salojärvi, J., Puolamäki, K., Simola, J., Kovanen, L., Kojo, I., Kaski, S.: Inferring relevance from eye movements: feature extraction. In: Workshop at NIPS 2005, in Whistler, BC, Canada, 10 December 2005, p. 45 (2005)

    Google Scholar 

  18. Breiman, L.: Random forests Leobreiman and Adele Cutler. Random Forests-Classification Description (2015)

    Google Scholar 

  19. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  20. Zhang, Z.: Naïve Bayes classification in R. Ann. Transl. Med. 4(12) (2016)

    Google Scholar 

  21. Wang, S.C.: Artificial neural network. In: Wang, S.C. (ed.) Interdisciplinary Computing in java Programming, pp. 81–100. Springer, Boston (2003). https://doi.org/10.1007/978-1-4615-0377-4_5

    Chapter  Google Scholar 

  22. Abadeh, M.S., Habibi, J., Soroush, E.: Induction of fuzzy classification systems via evolutionary ACO-based algorithms. Computer 35, 37 (2008)

    Google Scholar 

  23. Guan, F., Shi, J., Ma, X., Cui, W., Wu, J.: A method of false alarm recognition based on k-nearest neighbor. In: 2017 International Conference on Dependable Systems and Their Applications (DSA), pp. 8–12. IEEE (2017)

    Google Scholar 

  24. Behroozi, M., Parnin, C.: Can we predict stressful technical interview settings through eye-tracking? In: Proceedings of the Workshop on Eye Movements in Programming, pp. 1–5 (2018)

    Google Scholar 

  25. Krol, M., Krol, M.: A novel approach to studying strategic decisions with eye-tracking and machine learning. Judgm. Decis. Mak. 12(6), 596 (2017)

    Google Scholar 

  26. Richstone, L., Schwartz, M.J., Seideman, C., Cadeddu, J., Marshall, S., Kavoussi, L.R.: Eye metrics as an objective assessment of surgical skill. Ann. Surg. 252(1), 177–182 (2010)

    Article  Google Scholar 

  27. Cai, Y., Huang, H., Cai, H., Qi, Y.: A k-nearest neighbor locally search regression algorithm for short-term traffic flow forecasting. In: 2017 9th International Conference on Modelling, Identification and Control (ICMIC), pp. 624–629. IEEE (2017)

    Google Scholar 

  28. Pouw, W.T., Eielts, C., Van Gog, T., Zwaan, R.A., Paas, F.: Does (non-) meaningful sensori-motor engagement promote learning with animated physical systems? Mind Brain Educ. 10(2), 91 (2016)

    Google Scholar 

  29. Dubé, A.K., McEwen, R.N.: Do gestures matter? The implications of using touchscreen devices in mathematics instruction. Learn. Instr. 40, 89–98 (2015)

    Article  Google Scholar 

  30. Joseph, A.W., Murugesh, R.: Potential eye tracking metrics and indicators to measure cognitive load in human-computer interaction research. J. Sci. Res. 64(1) (2020)

    Google Scholar 

Download references

Acknowledgment

The authors are thankful to the Department of Computer Application, Madurai Kamaraj University, Madurai, India and National Institute of Design, Bengaluru Campus, India for their encouragement, motivation, and relentless support in carrying out our study.

Author information

Authors and Affiliations

  1. IT-Integrated Design, National Institute of Design, Bengaluru, Karnataka, India

    Antony William Joseph

  2. Department of Computer Application, Madurai Kamaraj University, Madurai, India

    J. Sharmila Vaiz & Ramaswami Murugesh

Authors
  1. Antony William Joseph
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Sharmila Vaiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramaswami Murugesh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antony William Joseph .

Editor information

Editors and Affiliations

  1. Institute for Advanced Systems Engineering, University of Central Florida, Orlando, FL, USA

    Tareq Z. Ahram

  2. University of Central Florida, Orlando, FL, USA

    Waldemar Karwowski

  3. Department of Pathology, College of Medicine, University of Saskatchewan Royal University Hospital, Saskatoon, SK, Canada

    Jay Kalra

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Joseph, A.W., Vaiz, J.S., Murugesh, R. (2021). Modeling Cognitive Load in Mobile Human Computer Interaction Using Eye Tracking Metrics. In: Ahram, T.Z., Karwowski, W., Kalra, J. (eds) Advances in Artificial Intelligence, Software and Systems Engineering. AHFE 2021. Lecture Notes in Networks and Systems, vol 271. Springer, Cham. https://doi.org/10.1007/978-3-030-80624-8_13

Download citation

Publish with us

Policies and ethics

Search

Navigation