An Internal Threat Detection Model Based on Denoising Autoencoders

  • Zhaoyang ZhangEmail author
  • Shen Wang
  • Guang Lu
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 157)


Internal user threat detection is an important research problem in the field of system security. Recently, the analysis of abnormal behaviors of users is divided into supervised learning method (SID) and unsupervised learning method (AD). However, supervised learning method relies on domain knowledge and user background, which means it cannot detect previously unknown attacks and is not suitable for multi-detection domain scenarios. Most existing AD methods use the clustering algorithm directly. But for threat detection on internal users’ behavior, mostly for high-dimensional cross-domain log files, as far as we know, there are few methods of multi-domain audit log data with effective feature extraction. An effective feature extraction method which can not only reduce testing cost greatly, but also detect the abnormal behavior of users more accurately. We propose a new unsupervised log abnormal behavior detection method which is based on the denoising autoencoders to encode the user log file, and adopts the integrated method to detect the abnormal data after encoding. Compared with the traditional detection method, it can analyze the abnormal information in the user behavior more effectively, thus playing a preventive role against internal threats. In addition, the method is completely data driven and does not rely on relevant domain knowledge and user’s background attributes. Experimental results verify the effectiveness of the integrated anomaly detection method under the multi-domain detection scenario of user log files.


Internal threat User cross-domain behavior analysis Denoising autoencoder Gaussian mixture model Machine learning 


  1. 1.
    Mayhew, M., Atighetchi, M., Adler, A., et al.: Use of machine learning in big data analytics for insider threat detection. In: Military Communications Conference. IEEE (2015)Google Scholar
  2. 2.
    Gheyas, I.A., Abdallah, A.E.: Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis. Big Data Anal. 1(1), 6 (2016)CrossRefGoogle Scholar
  3. 3.
    Evolving insider threat detection stream mining perspective. Int. J. Artif. Intell. Tools 22(05), 1360013 (2013)CrossRefGoogle Scholar
  4. 4.
    Chen, C.-M., Huang, Y., Wang, E.K., Wu, T.-Y.: Improvement of a mutual authentication protocol with anonymity for roaming service in wireless communications. Data Sci. Pattern Recogn. 2(1), 15–24 (2018)Google Scholar
  5. 5.
    Chen, C.-M., Xu, L., Wu, T.-Y., Li, C.-R.: On the security of a chaotic maps-based three-party authenticated key agreement protocol. J. Netw. Intell. 1(2), 61–66 (2016)Google Scholar
  6. 6.
    Chen, Y., Nyemba, S., Malin, B.: Detecting anomalous insiders in collaborative information systems. IEEE Trans. Dependable Secure Comput. 9(3), 332–344 (2012)CrossRefGoogle Scholar
  7. 7.
    Young, W.T., Goldberg, H.G., Memory, A., et al.: Use of domain knowledge to detect insider threats in computer activities (2013)Google Scholar
  8. 8.
    Rousseeuw, P.J., Driessen, K.V.: A fast algorithm for the minimum covariance determinant estimator. Technometrics 41(3), 212–223 (1999)CrossRefGoogle Scholar
  9. 9.
    Liu, F.T., Kai, M.T., Zhou, Z.H.: Isolation forest. In: Eighth IEEE International Conference on Data Mining (2009)Google Scholar
  10. 10.
    Manevitz, L.M., Yousef, M.: One-class SVMs for document classification. J. Mach. Learn. Res. 2(1), 139–154 (2002)zbMATHGoogle Scholar
  11. 11.
    Lee, J., Kang, B., Kang, S.H.: Integrating independent component analysis and local outlier factor for plant-wide process monitoring. J. Process Control 21(7), 1011–1021 (2011)CrossRefGoogle Scholar
  12. 12.
    Li, D., Chen, D., Goh, J., et al.: Anomaly detection with generative adversarial networks for multivariate time series (2018)Google Scholar
  13. 13.
    Dilokthanakul, N., Mediano, P.A.M., Garnelo, M., et al.: Deep unsupervised clustering with Gaussian mixture variational autoencoders (2016)Google Scholar
  14. 14.
    Glasser, J., Lindauer, B.: Bridging the gap: a pragmatic approach to generating insider threat data. In: 2013 IEEE Security and Privacy Workshops (SPW). IEEE (2013)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Department of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina

Personalised recommendations