Skip to main content
SpringerLink
Log in

Evaluating ANN Efficiency in Recognizing EEG and Eye-Tracking Evoked Potentials in Visual-Game-Events

  • Conference paper
  • First Online:
Advances in Neuroergonomics and Cognitive Engineering (AHFE 2017)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 586))

Included in the following conference series:

Abstract

EEG and Eye-tracking signals have customarily been analyzed and inspected visually in order to be correlated to the controlled stimuli. This process has proven to yield valid results as long as the stimuli of the experiment are under complete control (e.g.: the order of presentation). In this study, we have recorded the subject’s electroencephalogram and eye-tracking data while they were exposed to a 2D platform game. In the game we had control over the design of each level by choosing the diversity of actions (i.e. events) afforded to the player. However we had no control over the order in which these actions were undertaken. The psychophysiological signals were synchronized to these game events and used to train and test an artificial neural network in order to evaluate how efficiently such a tool can help us in establishing the correlation, and therefore differentiating among the different categories of events. The highest average accuracies were between 60.25%–72.07%, hinting that it is feasible to recognize reactions to complex uncontrolled stimuli, like game events, using artificial neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The ANN used in this study was created through the Encog Library [40].

References

  1. Koelstra, S., Mühl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3, 18–31 (2012). doi:10.1109/T-AFFC.2011.15

    Article  Google Scholar 

  2. Jain, A.K., Mao, J.: Artificial neural network: a tutorial. Communications 29, 31–44 (1996). doi:10.1109/2.485891

    Google Scholar 

  3. Basheer, I.A., Hajmeer, M.: Artificial neural networks: fundamentals, computing, design, and application. J. Microbiol. Methods 43, 3–31 (2000). doi:10.1016/S0167-7012(00)00201-3

    Article  Google Scholar 

  4. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst. 8, 373–389 (1995). doi:10.1016/0950-7051(96)81920-4

    Article  MATH  Google Scholar 

  5. Kubacki, A., Jakubowski, A., Sawicki, Ł.: Detection of artefacts from the motion of the eyelids created during eeg research using artificial neural network. Robot. Meas. Tech. Adv. Intell. Syst. Comput. 440, 447–457 (2016). doi:10.1007/978-3-319-29357-8_24

    Article  Google Scholar 

  6. Wu, F.Y., Slater, J.D., Honig, L.S., Ramsay, R.E.: A neural network design for event-related potential diagnosis. Comput. Biol. Med. 23(3), 251–264 (1993)

    Article  Google Scholar 

  7. Turnip, A., Hong, K.S.: Classifying mental activities from EEG-P300 signals using adaptive neural networks. Int. J. Innov. Comput. Inf. Control 8, 6429–6443 (2012)

    Google Scholar 

  8. Gupta, L., Molfese, D.L., Ravi, T.: An artificial neural-network approach to ERP classification. Brain Cogn. 27, 311–330 (1995)

    Article  Google Scholar 

  9. Wilson, G.F., Christopher, R.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors 45, 635–643 (2004). doi:10.1518/hfes.45.4.635.27088

    Article  Google Scholar 

  10. Wilson, G., Russell, C.: Operator functional state classification using multiple psychophysiological features in an air traffic control task. Hum. Factors 45, 381–389 (2003)

    Article  Google Scholar 

  11. Baldwin, C.L., Penaranda, B.N.: Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification. NeuroImage 59, 48–56 (2012). doi:10.1016/j.neuroimage.2011.07.047. Elsevier Inc.

    Article  Google Scholar 

  12. Robert, C., Gaudy, J.-F., Limoge, A.: Electroencephalogram processing using neural networks. Clin. Neurophysiol.: Off. J. Int. Fed. Clin. Neurophysiol. 113, 694–701 (2002). doi:10.1016/S1388-2457(02)00033-0

    Article  Google Scholar 

  13. Yannakakis, G.N., Hallam, J., Lund, H.H.: Entertainment capture through heart rate activity in physical interactive playgrounds. User Model. User-Adap. Interact. 18, 207–243 (2008). doi:10.1007/s11257-007-9036-7

    Article  Google Scholar 

  14. Perez Martínez, H., Garbarino, M., Yannakakis, G.N.: Generic physiological features as predictors of player experience. Affect. Comput. Intel. Interact. 267–276 (2011). doi:10.1007/978-3-642-24600-5_30

  15. Weber, R., Behr, K.M., Tamborini, R., Ritterfeld, U., Mathiak, K.: What do we really know about first-person-shooter games? An event-related, high-resolution content analysis. J. Comput.-Med. Commun. 14, 1016–1037 (2009). doi:10.1111/j.1083-6101.2009.01479.x

    Article  Google Scholar 

  16. Togelius, J., Karakovskiy, S., Shaker, N.: MarioAI (2012). http://www.marioai.org/LevelGeneration/source-code. Accessed 14 Mar 2016

  17. Shaker, N., Asteriadis, S., Yannakakis, G.N., Karpouzis, K.: A game-based corpus for analysing the interplay between game context and player experience. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, vol. 6975, pp. 547–556 (2011). doi:10.1007/978-3-642-24571-8_68

  18. Shaker, N., Asteriadis, S., Yannakakis, G.N., Karpouzis, K.: Fusing visual and behavioral cues for modeling user experience in games. IEEE Trans. Cybern. 43, 1519–1531 (2013). doi:10.1109/TCYB.2013.2271738

    Article  Google Scholar 

  19. Shaker, N., Shaker, M., Abou-zleikha, M.: Towards generic models of player experience. In: Proceedings, the Eleventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-2015), 191–197 (2015). http://www.aaai.org/ocs/index.php/AIIDE/AIIDE15/paper/download/11548/11378, ISBN 157735740X 9781577357407

  20. Yannakakis, G.N., Togelius, J.: Experience-driven procedural content generation. IEEE Trans. Affect. Comput. 2(3), 147–161 (2011). doi:10.1109/T-AFFC.2011.6

    Article  Google Scholar 

  21. Togelius, J., Shaker, N., Karakovskiy, S., Yannakakis, G.N.: The Mario AI championship 2009-2012. AI Mag. 34, 89–92 (2013). doi:10.1609/aimag.v34i3.2492

    Google Scholar 

  22. The_EyeTribe: The Eye Tribe Tracker (2015)

    Google Scholar 

  23. G.tec, medical engineering: Advanced Biosignal Acquisition, Processing and Analysis Products 2013–2014, pp. 1–96 (2013)

    Google Scholar 

  24. Kowalik, M.: Do-it-yourself eye tracker: impact of the viewing angle on the eye tracking accuracy. In: Proceedings of CESCG 2011: The 15th Central European Seminar on Computer Graphics, pp. 1–7 (2011). http://old.cescg.org/CESCG-2011/papers/Szczecin-Kowalik-Michal.pdf, ISBN 978-3-9502533-3-7

  25. Ooms, K.: Accuracy and precision of fixation locations recorded with the low-cost Eye Tribe tracker in different experimental set- ups 8 1–20 (2015). doi:10.16910/jemr.8.1.5

  26. Kosslyn, S.M.: Measuring the visual angle of the mind’s eye. Cogn. Psychol. 10, 356–389 (1978). doi:10.1016/0010-0285(78)90004-X

    Article  Google Scholar 

  27. Key, A.P.F., Dove, G.O., Maguire, M.J.: Linking brainwaves to the brain: an ERP primer. Dev. Neuropsychol. 27, 183–215 (2005). doi:10.1207/s15326942dn2702_1

    Article  Google Scholar 

  28. Başar, E., Başar-Eroglu, C., Karakaş, S., Schürmann, M.: Are cognitive processes manifested in event-related gamma, alpha, theta and delta oscillations in the EEG? Neurosci. Lett. 259, 165–168 (1999). doi:10.1016/S0304-3940(98)00934-3

    Article  Google Scholar 

  29. Sprague, N., Ballard, D., Robinson, A.: Modeling embodied visual behaviors. ACM Trans. Appl. Percept. 4, 11 (2007). doi:10.1145/1265957.1265960

    Article  Google Scholar 

  30. Reilly, R.: Triangulating the reading brain: eye movements, computational models, and EEG. In: Current Trends in Eye Tracking Research, pp. 131–139. Springer International Publishing (2014)

    Google Scholar 

  31. Laeng, B., Sirois, S., Gredeback, G.: Pupillometry: a window to the preconscious? Perspect. Psychol. Sci. 7, 18–27 (2012). doi:10.1177/1745691611427305

    Article  Google Scholar 

  32. Hyvärinen, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Netw. 13, 411–430 (2000). doi:10.1016/S0893-6080(00)00026-5

    Article  Google Scholar 

  33. Makeig, S., Bell, A.J., Jung, T.-P., Sejnowski, T.J.: Independent component analysis of electroencephalographic data. Adv. Neural. Inf. Process. Syst. 8, 145–151 (1996). doi:10.1109/ICOSP.2002.1180091

    Google Scholar 

  34. Shaker, N., Yannakakis, G.N., Togelius, J.: Towards automatic personalized content generation for platform games. In: AIIDE, October 2010. https://www.aaai.org/ocs/index.php/AIIDE/AIIDE10/%20paper/viewFile/2135/2546

  35. Nogueira, P.A., Aguiar, R., Rodrigues, R., Oliveira, E.: Designing players’ emotional reaction models: a generic method towards adaptive affective gaming. In: 2014 9th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6. IEEE. (2014). doi:10.1109/CISTI.2014.6877079

  36. D’Mello, S., Chipman, P., Graesser, A.: Posture as a predictor of learner’s affective engagement. In: Proceedings of the 29th Annual Meeting of the Cognitive Science Society, vol. 1 pp. 905–910 (2007)

    Google Scholar 

  37. MacDonald, J.H.: Kruskal-Wallis test. In: Biological Handbook of Statistics (2009)

    Google Scholar 

  38. Lindley, C.A., Sennersten., C.C.: Game play schemas: from player analysis to adaptive game mechanics. Int. J. Comput. Games Technol. 2008, 1–7 (2008). doi:10.1155/2008/216784

    Article  Google Scholar 

  39. Cecotti, H.: A time-frequency convolutional neural network for the offline classification of steady-state visual evoked potential responses. Pattern Recognit. Lett. 32 1145–1153 (2011). doi:10.1016/j.patrec.2011.02.022

  40. Heaton, J.: Encog: library of interchangeable machine learning models for Java and C#. J. Mach. Learn. Res. 16, 1243–1247 (2015)

    MathSciNet  MATH  Google Scholar 

  41. Kliegl, R., Dambacher, M., Dimigen, O., Sommer, W.: Oculomotor control, brain potentials, and timelines of word recognition during natural reading. In: Current Trends in Eye Tracking Research, pp. 141–155. Springer International Publishing (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Architecture, Design and Media Technology, Aalborg University Copenhagen, AC Meyervænge 15, 2450, Copenhagen, Denmark

    Andreas Wulff-Jensen & Luis Emilio Bruni

Authors
  1. Andreas Wulff-Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Emilio Bruni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Wulff-Jensen .

Editor information

Editors and Affiliations

  1. Department of Psychology, Human Factors and Applied Cognition, George Mason University, Fairfax, Virginia, USA

    Carryl Baldwin

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Cite this paper

Wulff-Jensen, A., Bruni, L.E. (2018). Evaluating ANN Efficiency in Recognizing EEG and Eye-Tracking Evoked Potentials in Visual-Game-Events. In: Baldwin, C. (eds) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2017. Advances in Intelligent Systems and Computing, vol 586. Springer, Cham. https://doi.org/10.1007/978-3-319-60642-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60642-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60641-5

  • Online ISBN: 978-3-319-60642-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation