PanoSynth

"Vigilance decrement", first systematically studied by Norman Mackworth during World War II in the 1940s when he discovered radar and sonar operators' tendency to miss rare irregular signals toward the end of their shifts, refers to the decline in detection accuracy and response speed over time during sustained attention tasks. From the perspective of distributed cognition theory and cognitive offloading, as Kirsch & Maglio (1994) noted, "certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone," which principle can be applied to address this issue.

This research developed a cognitive offloading system that transforms traditional CCTV surveillance interfaces from cognitively demanding arrays of 2D perspectives to an intuitive 3D environment by integrating diverse camera feeds (smartphones, drones, CCTV cameras) in real-time to solve the cognitive burden and vigilance decrement problems faced by security personnel monitoring multiple 2D screens simultaneously. Using MASt3R-SLAM-based camera-agnostic 3D reconstruction technology, the system achieves real-time processing (approximately 1.8 FPS per camera) with standard RGB cameras alone without requiring specialized hardware or camera calibration, and a user study with 12 participants demonstrated statistically significant improvements in ease of use (p=0.027) and learnability (p=0.01) compared to traditional systems. This system maintains cognitive burden at existing levels while enhancing spatial understanding and eliminating the mental translation process of integrating multiple 2D views into 3D, potentially extending monitoring duration significantly, with future challenges including mobile surveillance asset integration, dynamic object rendering improvements, surveillance-specific data fine-tuning, and visual quality enhancement.

--------------------------------

This project is branched off from my previous research from Republic of Korea Army and I am sharing upto what I am allowed to share.

Poster accepted to ACM UIST 2025.

Contact me for the research link.