Abstract
Multimodal interaction has been verified to improve user experience, but the challenge of resource competition and information integration in this human-centered interaction mode may negatively affect the efficiency and naturalness. This paper aims to figure out how efficiency and naturalness change in multimodal interaction in mobile navigation apps, which are mostly used as interactive systems for cars’ navigation tasks. The study was conducted based on the Amap app, with which participants should use the single touch-screen interaction and the gesture-combined-voice interaction, respectively, to complete the same route setting task in the driving environment. With the records of the operation time and participants’ subjective scores of two interaction modes, we analyzed the differences and changes in efficiency and naturalness. The results showed that multimodal interaction was more efficient than single-modality interaction; however, the naturalness of multimodal interaction did not improve significantly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhou, Z.M., Ji, A.H.: Touch human-machine interface engineering design and application. China Electric Power Press, Beijing (2013). (in Chinese)
Fang, Z.G., Wang, J.: The new trend of human-machine interaction: multimedia and multimodal. Chin. J. Ergon. 4(2), 34–38 (1998). (in Chinese)
Ma, W.J., Fang, Z.G.: Human-computer interaction style and its evolution. Aeronaut. Comput. Technol. 29(3), 16–20 (1999). (in Chinese)
Yin, C.: The design and application of natural interaction based on event prototype derivation. Hunan University, Changsha (2014). (in Chinese)
Dong, S.H., Wang, J., Dai, G.Z.: Human-computer interaction and multimodal user interface. Science Press, Beijing (1999). (in Chinese)
Cun, W.Z.: Research of human cognition and behavior based on multi-model. Guizhou University, Guiyang (2018). (in Chinese)
Dong, S.H.: Progress and challenge of human-computer interaction. J. Comput.-Aid. Des. Comput. Graph. 16(1), 1–13 (2004). (in Chinese)
Bunt, H., Beun, R.-J., Borghuis, T.: Multimodal human-computer communication systems, techniques, and experiments. Springer, Berlin (1998)
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: gesture and speech. ACM Trans. Comput.-Human Interact. 9(3), 171–193 (2002)
Obrenovic, Z., Starcevic, D.: Modeling multimodal human-computer interaction. Computer. 37(9), 65–72 (2004)
Kumar, P., Agrawal, A., Prasad, S.: Multimodal interface for temporal pattern based interactive large volumetric visualization. In: TENCON 2017–2017 IEEE Region 10 Conference, pp. 1239–1244. IEEE Press, New York (2017)
Ji, T.Q.: Multi-channel human-machine intelligent interaction technology. Nanjing University, Nanjing (2018). (in Chinese)
Xiao, B., Girand, C., Oviatt, S.: Multimodal integration patterns in children. In: Proceedings of 7th International Conference on Spoken Language Processing, pp. 629–632. Causal Productions Pty Ltd., Adelaide (2002)
Xiao, B., Lunsford, R., Coulston, R., Wesson, M., Oviatt, S.: Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences. In: Proceedings of the 5th International Conference on Multimodal Interfaces, pp. 265–272. ACM, New York (2003)
Oviatt, S., Lunsford, R., Coulston, R.: Individual differences in multimodal integration patterns: what are they and why do they exist? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 241–249. ACM, New York (2005)
Bohus, D., Horvitz, E.: Facilitating multiparty dialog with gaze, gesture, and speech. In: Proceedings of International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, pp. 1–8. ACM, New York (2010)
Turk, M.: Multimodal interaction: a review. Pattern Recogn. Lett. 36, 189–195 (2014)
Yang, M.H., Tao, J.H.: Intelligence methods of multi-modal information fusion in human-computer interaction. Sci. Sin. Inform. 48(4), 433–448 (2018). (in Chinese)
Zhang, C., Zhao, J.H.: Multimodal interaction design of automobile navigation. Pack. Eng. 36(22), 67–70 (2015). (in Chinese)
Diaper, D.: Scenarios and task analysis. Interact. Comput. 14(4), 379–395 (2002)
Most Drivers Who Own Cars with Built-in GPS Systems Use Phones for Directions. https://money.cnn.com/2016/10/10/autos/car-navigation-frustration/
Why Do Most People Use Mobile Phones Instead of Car Navigation? https://new.qq.com/omn/20191007/20191007A0D0SP00. (in Chinese)
2018 China Mobile Map Industry Annual Report. https://www.useit.com.cn/thread-22437-1-1.html. (in Chinese)
Yang, Y.: Research on multi-channel interaction of handheld mobile device based on voice interaction. Beijing University of Posts and Telecommunications, Beijing (2017). (in Chinese)
Tunnell, G.B.: Three dimensions of naturalness: an expanded definition of field research. Psychol. Bull. 84(3), 426–437 (1977)
Dall, R., Yamagishi, J., King, S.: Rating naturalness in speech synthesis: the effect of style and expectation. In: Proceedings of the 7th International Conference on Speech Prosody, pp. 1012–1016. Trinity College, Dublin (2014)
Wang, R., Dong, S.Y., Xiao, J.H.: Research on human-machine natural interaction of intelligent vehicle interface design. J. Mach. Desig. 36(2), 132–136 (2019). (in Chinese)
Qiu, H.Z.: Quantitative research and statistical analysis: case study of SPSS (PASW) data analysis. Chongqing University Press, Chongqing (2013). (in Chinese)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ling, J., Peng, Z., Yin, L., Yuan, X. (2020). How Efficiency and Naturalness Change in Multimodal Interaction in Mobile Navigation Apps. In: Ahram, T., Falcão, C. (eds) Advances in Usability, User Experience, Wearable and Assistive Technology. AHFE 2020. Advances in Intelligent Systems and Computing, vol 1217. Springer, Cham. https://doi.org/10.1007/978-3-030-51828-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-51828-8_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51827-1
Online ISBN: 978-3-030-51828-8
eBook Packages: EngineeringEngineering (R0)