A smart campus project started in 2019 finally sees its first academic paper titled “Network as a sensor for smart crowd analysis and service improvement” appear in a Smart Communities special issue of IEEE Network Magazine. It was meant to be a pure engineering project to showcase the potential of campus WiFi data for service optimisation and automation but it quickly became a data science project too when we started to gather and process hundreds of millions of anonymised connectivity data. In summary, we monitor how connected devices switch between WiFi APs and use machine learning to model crowd behaviours for predictive analysis, anomaly detection, etc. Comparing with conventional crowd analysis solutions based on video cameras or WiFi probing. our solution is less intrusive and does not require the installation of additional equipment. Our SDN infrastructure is the icing on the cake as it offers a single point for data aggregation.

Abstract:

With the growing availability of data processing and machine learning infrastructures, crowd analysis is becoming an important tool to tackle economic, social, and environmental challenges in smart communities. The heterogeneous crowd movement data captured by IoT solutions can inform policy-making and quick responses to community events or incidents. However, conventional crowd-monitoring techniques using video cameras and facial recognition are intrusive to everyday life. This article introduces a novel non-intrusive crowd monitoring solution which uses 1,500+ software-defined networks (SDN) assisted WiFi access points as 24/7 sensors to monitor and analyze crowd information. Prototypes and crowd behavior models have been developed using over 900 million WiFi records captured on a university campus. We use a range of data visualization and time-series data analysis tools to uncover complex and dynamic patterns in large-scale crowd data. The results can greatly benefit organizations and individuals in smart communities for data-driven service improvement.

An associated dataset that includes over 300 million records of WiFi access data is available at: https://bit.ly/3Dmi6X1.