Unusual Activity Detection with Alert System Using Vision Transformer

Abstract

Traditional surveillance systems are constrained by the limitations of manual monitoring, often resulting in delayed detection of anomalous activities. This research investigates an AI-driven surveillance system employing a Vision Transformer (ViT) architecture, pretrained on the UCF Crime Dataset, to automate anomaly detection by classifying video sequences as ‘normal’ or ‘unusual’. The dataset was preprocessed by extracting frames at 30 fps, followed by resizing, augmentation, and fed into a fine-tuned ViT model, which achieved a macro F1-score of 0.7858 and 78.95% test accuracy. The system built in this study features a backend infrastructure based on Django, Django Channels, and Redis to enable efficient session management and WebSocket communication, supporting a live streaming module for anomaly detection and broadcasting via WebRTC, alongside a surveillance module for remote feed access. When any abnormal event is detected, the system automatically captures relevant snapshots of corresponding image and confidence score, and provides alert through the user interface or third-party applications. This approach aims to enhance detection accuracy, operational efficiency and situational awareness in surveillance environments.