Traffic Outlier Detection by Density-Based Bounded Local Outlier Factors

Jialing Tang, Henry Y. T. Ngan

Abstract


Outlier detection (OD) is widely used in many fields, such as finance, information and medicine, in cleaning up datasets and keeping the useful information. In a traffic system, it alerts the transport department and drivers with abnormal traffic situations such as congestion and traffic accident. This paper presents a density-based bounded LOF (BLOF) method for large-scale traffic video data in Hong Kong. A dimension reduction by principal component analysis (PCA) was accomplished on the spatial-temporal traffic signals. Previously, a density-based local outlier factor (LOF) method on a two-dimensional (2D) PCA-proceeded spatial plane was performed. In this paper, a three-dimensional (3D) PCA-proceeded spatial space for the classical density-based OD is firstly compared with the results from the 2D counterpart. In our experiments, the classical density-based LOF OD has been applied to the 3D PCA-proceeded data domain, which is new in literature, and compared to the previous 2D domain. The average DSRs has increased about 2% in the PM sessions: 91% (2D) and 93% (3D). Also, comparing the classical density-based LOF and the new BLOF OD methods, the average DSRs in the supervised approach has increased from 94% (LOF) to 96% (BLOF) for the AM sessions and from 93% (LOF) to 95% (BLOF) for the PM sessions.

Keywords


Outlier; Density-Based; Local Outlier Factor; Supervised Approach; Traffic Data

References


C. C. Aggarwal, Outlier Analysis, Springer, 2013.

C. O'Reilly, A. Gluhak, M. A. Imran, S. Rajasegarar, "Anomaly detection in wireless sensor networks in a non-stationary environment," IEEE Trans. Communications Surveys & Tutorals, pp. 1413-1432, 2014.

V. Chandola, A. Banerjee, and V. Kumar. "Anomaly detection: a survey," ACM Computing Surveys (CSUR), vol. 41, issue 3, no. 15, 2009.

L. Clifton, D. A. Clifton, Y. Zhang, P. Watkinson, L. Tarassenko, and H. Yin, "Probabilitic novelty detection with support vector machines," IEEE Trans. Reliability, vol. 63, no. 2, pp. 455-467, 2014.

H. Lee, J. Song, and D. Park, "Intrusion detection system based on multi-class SVM," RSFDGrC, LNAI 3642, pp. 511-519, 2005.

H.-P. Kriegel, P, Kroge, and A, Zimek, "Outlier detection techniques," SIAM Int'l Conf. Data Mining, 2010.

C. S. C. Tsang, H. Y. T. Ngan, and G. K. H. Pang, "Fabric inspection based on the ELO rating method," Pattern Recognition, 51, pp. 378-394, 2016.

H. Y. T. Ngan and G. K. H. Pang, "Robust defect detection in plain and twill fabric using directional Bollinger bands," Optical Engineering, vol. 54, no. 7, 073106, 2015.

M. K. Ng, H. Y. T. Ngan, X. Yuan, and W. Zhang, "Patterned fabric inspection and visualization by the method of image decomposition," IEEE Trans. Automation Science & Engineering, vol. 11, no. 3, pp. 943-947, 2014.

S. Chen, W. Wang, and H. van Zuylen, "A comparison of outlier detection algorithms for ITS data," Expert Systems with Applications, vol. 37, no. 2, pp. 1169-1178, 2010.

E. Muller, P. I. Sanchez, Y. Mulle, and K. Bohm, "Ranking outlier nodes in subspaces of attributed graphs," IEEE 29th Int'l Conf. Data Engineering Workshops (ICDEW), pp. 216-222, 2013.

A. D. Pawar, P. N. Kalavadekar, and S. N. Tambe, "A survey on outlier detection techniques for credit card fraud detection," IOSR Journal of Computer Engineering, vol. 16, no .2, pp. 44-48, 2014.

H. Y. T. Ngan, N. H. C. Yung, and A. G. O. Yeh, "A comparative study of outlier detection for large-scale traffic data by one-class SVM and kernel density estimation," IS&T/SPIE Electronic Imaging, 94050I-94050I-10, 2015 .

D. M. J. Tax and R. P. W. Duin, "Outlier detection using classifier instability," Proc. Joint IAPR Int'l Workshops on Advances in Pattern Recognition, pp. 593-601, 1998.

D.J. Hill, B.S. Minsker and E. Amir, "Real-time Bayesian Anomaly Detection for Environmental Senor Data," Water Resources Reserrch, vol. 45, no. 4, 2009.

P. Gil, A. Santos, and A. Cardoso, "Dealing with outliers in wireless sensor networks: an oil refinery application," IEEE Trans. Control Systems Technology, vol. 22, no. 4, pp. 1589-1596, 2013.

H. Y. T. Ngan, N. H. C. Yung, and A. G. O. Yeh, "Outlier detection in traffic data based on the Dirichlet process mixture model," IET Intelligent Transport Systems, vol. 9, no. 7, pp. 773-781, 2015.

C. H. M. Wong, H. Y. T. Ngan and N. H. C. Yung, "Modulo-k clustering based outlier detection for large-scale traffic data," Proc. Int'l Conf. IEEE Information Technology and Application (ICITA), 2016.

T. T. Dang, H. Y. T. Ngan, and W.Liu, "Distance-based k-nearest neighbors outlier detection method in large-scale traffic data," IEEE Int'l Conf. Digital Signal Processing (DSP), pp. 507-710, 2015.

M. Amer, M. Goldstein and S. Abdennadher, "Enhancing one-class support vector machines unsupervised anomaly detection," Proc. ACM ODD, pp. 8-15, 2013.

M. Davy and S. Godsill, "Detection of abrupt spectral changes using support vector machines: an application to audio signal segmentation," Proc. IEEE ICASSP, vol. 2, pp. II-1313-II-1316, 2002.

N. A. Shrivastava, A. Khosravi, and B. K. Panigrahi, "Prediction interval estimation for wind farm power generation forecasts using support vector machines," Proc. IEEE Int'l Joint Conf. Neural Networks, pp.1-7, 2015.

J. Ma and S. Perkins, "Online novelty detection on temporal sequences," Proc. 9th ACM SIGKDD, pp. 613-618, 2003.

P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection, John Wiley & Sons Inc., 2005.

L. J. Latecki, A. Lazarevic, and D. Pokrajac, "Outlier detection with kernel density functions," Proc. 5th Int'l Conf. Machine Learning & Data Mining in Pattern Recognition, pp. 61-75, 2007.

E. M. Knorr, R. T. Ng, and V. Tucakov, "Distance-based outliers: algorithms and applications," The VLDB Journal, vol. 8, no. 3-4, pp. 237-253, 2000.

K. Cao, L. Shi, G. Wang, D. Han, and M. Bai, "Density-based local outlier detection on uncertain data," WAIM, LNCS, vol. 8485, pp. 67-71, 2014.

M. X. Ma, H. Y. T. Ngan, and W. Liu, "Density based outlier detection by local outlier factor on large-scale traffic data," IS&T Int'l Sym. Electronic Imaging, pp. 1-4, 2016.

A. Arning, R. Agrawal, and P. Raghavan, "A linear method for deviation detection in large databases," Proc. KDD, pp. 164-169, 1996.

F. T. Liu, K. M. Ting and Z. H. Zhou, "Isolation-based Anomaly Detection," ACM Trans. Knowledge Discovery from Data, vol. 6, no.1, 3:1-3:38, 2012.

M. M. Breunig, H. P. Kriegel, R. T. Ng, and J. Sander, "LOF: identifying density-based local outliers," Proc. ACM SIGMOD, pp. 93-104, 2000.

J. A. Hanley and B. J. McNeil, "The meaning and use of the area under a receiver operating characteristic (ROC) curve," Radiology, vol. 143, no. 1, pp. 29-36, 1982.


Full Text: PDF

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.

IT in Innovation IT in Business IT in Engineering IT in Health IT in Science IT in Design IT in Fashion

IT in Industry http://www.it-in-industry.org ISSN (Online): 2203-1731; ISSN (Print): 2204-0595