Abstract
Due to the rapid advancement in Internet of Things (IoT), myriad systems generate data of massive volume, variety and velocity which traditional databases are unable to manage effectively. Many organizations need to deal with these massive datasets that encounter different types of data (e.g., IoT streaming data, static data) in different formats coming from multiple sources. Different data integration mechanisms are designed to process mostly static data. Unfortunately, these techniques are not adequate to integrate IoT streaming data from multiple sources. In this paper, we identify the challenges of IoT streaming data integration (ISDI). A generic window-based ISDI approach is proposed to deal with IoT data in different formats and subsequently introduced the algorithms to integrate IoT streaming data obtained from multiple sources. In particular, we extend the basic windowing algorithm for real-time data integration and to deal with the timing alignment issue. We also introduce a de-duplication algorithm to deal with data redundancies and to demonstrate the useful fragments of the integrated data. We conduct several sets of experiments and quantify the performance of our proposed window-based ISDI approach. The experimental results, performed on several IoT datasets, show the efficiency of our proposed ISDI solution in terms of processing time.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Herland, M., Khoshgoftaar, T.M., Bauder, R.A.: Big data fraud detection using multiple medicare data sources. J. Big Data 5(1), 29 (2018)
Chen, J., Chen, Y., Du, X., Li, C., Lu, J., Zhao, S., Zhou, X.: Big data challenge: a data management perspective. Front. Comput. Sci. 7(2), 157–164 (2013)
Bellahsène, Z., Bonifati, A., Rahm, E.: Schema Matching and Mapping. Springer, Berlin (2011)
Sagi, T., Gal, A., Barkol, O., Bergman, R., Avram, A.: Multi-source uncertain entity resolution: transforming holocaust victim reports into people. Inf. Syst. 65, 124–136 (2017)
Calbimonte, J.P., Corcho, O., Gray, A.J.: Enabling ontology-based access to streaming data sources. In: International Semantic Web Conference, pp. 96–111. Springer (2010)
Daraio, C., Lenzerini, M., Leporelli, C., Naggar, P., Bonaccorsi, A., Bartolucci, A.: The advantages of an ontology-based data management approach: openness, interoperability and data quality. Scientometrics 108(1), 441–455 (2016)
Bifet, A., Gavalda, R.: Learning from time-changing data with adaptive windowing. In: Proceedings of the 2007 SIAM International Conference on Data Mining, SIAM, pp. 443–448 (2007)
Gama, J., Sebastião, R., Rodrigues, P.P.: On evaluating stream learning algorithms. Mach. Learn. 90(3), 317–346 (2013)
Pareek, A., Khaladkar, B., Sen, R., Onat, B., Nadimpalli, V., Lakshminarayanan, M.: Real-time ETL in Striim. In: Proceedings of the International Workshop on Real-Time Business Intelligence and Analytics, p. 3. ACM (2018)
Ahad, M.A., Biswas, R.: Dynamic merging based small file storage (DM-SFS) architecture for efficiently storing small size files in hadoop. Procedia Comput. Sci. 132, 1626–1635 (2018)
Kayes, A., Han, J., Rahayu, W., Dillon, T., Islam, S., Colman, A.: A policy model and framework for context-aware access control to information resources. Comput. J. (2018) https://doi.org/10.1093/comjnl/bxy065
Kayes, A., Rahayu, W., Dillon, T., Chang, E.: Accessing data from multiple sources through context-aware access control. In: TrustCom, pp. 551–559. IEEE (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Tu, D.Q., Kayes, A.S.M., Rahayu, W., Nguyen, K. (2020). ISDI: A New Window-Based Framework for Integrating IoT Streaming Data from Multiple Sources. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds) Advanced Information Networking and Applications. AINA 2019. Advances in Intelligent Systems and Computing, vol 926. Springer, Cham. https://doi.org/10.1007/978-3-030-15032-7_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-15032-7_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15031-0
Online ISBN: 978-3-030-15032-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)