Detecting anomalies from roadside video streams

Abstract

The interconnected nature of road networks implies that anomalies on narrow residential roads can ripple through the entire traffic system, particularly in high鈥� traffic areas as common for the Maltese Islands. Detecting anomalies in such en鈥� vironments using roadside cameras is challenging due to the multitude of normal and anomalous events, changes in illumination, obstructions, complex anomalies, and difficult viewing angles. This thesis investigates anomaly detection methods tailored to the realistic road and data limitations typical of Maltese urban roads. Classical anomaly detection, which identifies anomalies from structured data, and deep learning鈥恇ased techniques, which detect anomalies directly from video input, were evaluated. The literature review revealed limited evaluations on realistic datasets for both methods. The classical method was developed to filter out ID switch artifacts and identify specific anomalies using a combination of filtering, DBSCAN clustering, masking, and rule鈥恇ased techniques. For the deep learning method, an AE model with the STAE [1] architecture was chosen for its ability to capture temporal rep鈥� resentation. Both methods were evaluated on video datasets collected in Malta and a relabeled Street Scene [2] dataset. The classical method demonstrated high reliability in detecting anomalies in structured data, achieving an 82% true positive rate and a 3% false positive rate for a local dataset. However, the data acquisition method did not accurately record all anomalies, reducing the true positive rate for actual video anomalies. The deep learning method showed strong performance across all datasets, achiev鈥� ing an 83% AUC and a 25% EER for a dataset recorded in the same location. Per鈥� formance was slightly reduced for locations with heavy shadows, as shown on a second local dataset. Segmenting frames into tiles and augmenting datasets improved performance in shadow鈥恆ffected conditions, as did masking irrelevant regions. An event鈥恖evel comparison showed both methods performed similarly in detecting non鈥恡ypical vehicle paths. The classical method excelled at identifying non鈥恡ypical object locations and was more robust against changes in scene dynam鈥� ics, is more modular, and easier to debug. The deep learning method was better at detecting non鈥恡ypical slow鈥恗oving and non鈥恡ypical vehicles and was more resilient to variations in the data acquisition method within the Intelligent Traffic System (ITS). However, neither method effectively detected unforeseen anomalies. Over鈥� all, this thesis provides valuable insights and guidance for choosing the most ap鈥� propriate anomaly detection methods tailored to different types of anomalies in complex urban road environments.