Researcher: Raj Rajkumar
We will create core technologies for building scalable, large-scale video surveillance networks to detect intrusion, object and people movement for enforcing physical security in critical infrastructures. Video surveillance networks deployed in many large cities concerned about terrorist and illegal activities are heavily centered on logging of raw data, and very heavy human labor to detect tracks of miscreants after an incident of interest. Our objective is to automate object recognition, rule violation detection, data collection and classification at each surveillance node. In this first generation of research, these annotated markings will be generated at each node and stored along with the raw data. In the second generation, information will also be communicated across nodes such that a moving person or object can automatically be tracked across different surveillance nodes.
With partial support from CyLab, we have developed the FireFly family of sensor networks.This low-cost, wireless sensor networking platform is capable of data acquisition, processing and multi-hop mesh communication. (Additional information about a real time, preemptive multi-tasking operating system deployed on FireFly is provided at nano-rk.) We have also recently develop DSPcam, a relatively low-cost camera device with very high-end processing capabilities. The hardware capabilities are in place, but no processing capabilities, mesh networking capabilities or automated object detection capabilities are yet in place.
Consider a security-sensitive location, such as the fenced boundary of a critical infrastructure location, the entrance/exit to a secure facility, or the boarding of a person to or from a moving object in a sensitive public area. One or more surveillance cameras should automatically detect the presence of an object, its attributes (such as color, size and thermal density) and most matched template (such as a person, animal, car or truck.) It must also be possible to define profiles of the background (such as travel lanes, pavements, fences, doors,etc.) under different lighting and weather conditions. The superimposition of objects or people on expected or known backgrounds will allow the backgrounds to be masked out; core attributes to be extracted and the objects to be detected, recognized and categorized whenever possible. The raw data annotated with time-stamps of the relevant segments and the extracted meta-information will then be archived. The meta-information and timestamps can then be searched rapidly in the future. Immediate computations to detect any violations of policies (such as exit/entry at a no access door or driving in the wrong direction on a one-way street) can also be triggered.
Next, while video frames have a large amount of information from which rich details can be extracted, they also consume massive quantities of storage and bandwidth. Our Intelligent Video Surveillance Network will compress information selectively with higher resolution images stored when interesting events occur, and disregarding repetitive data or storing on low-resolution data otherwise.
Project deliverables will include a testbed of 10 DSPcam node that can communicate with one another for smart handoff of tracking objects; tools to define the background of an environment being monitored and specification of appearances under different lighting and weather conditions, algorithms to extract meta-attributes of objects, given different backgrounds and algorithms to detect vehicles, pedestrians and bikes; archival facilities to store raw video data with meta-information; and basic query facilities to search archived storage for video data matching specified attributes.