An Advanced Video Fusion System for Security Investigations

ITSP project (Project Reference : GHP/060/11) with 5M HKD funding.
06/2012 - 09/2014


System overview.
(visualization for a steal sequence)

Project summary

Video surveillance is deployed everywhere in HK to enhance public security. Security investigation is still much relied on checking 2D videos from separate camera views.

2D video segments do NOT provide global situational understanding (with both space and time registration).

  • Each camera has a fixed viewpoint, a fixed orientation and a limited field of view.
  • Hard to imagine the relative spatial position of people covered by different cameras.
  • Hard to find out the temporal order of activities/events within a large time window.
    • Suspects may coordinate their activities by remotely communicating with each other with body languages or phones.
  • Can not closely check an action or event from all viewing directions.
This project aims at analyzing and fusing videos from multiple cameras to create an informative and easy-to-comprehend reenactment of a past event to assist investigations, providing a global understanding with both space and time registration for complex scenarios and enabling investigators to have a close-up check of actions from novel viewing directions.

Project deliverables

  1. An internal 4D representation of past events.
    • A simple pre-built 3D virtual model of an environment under surveillance.
    • Recorded videos from a sufficient number of cameras.
    • Recovered camera poses with respect to the 3D virtual model.
  2. On-demand object extraction and action reconstruction.
  3. Visualizing 3D individual or group actions over the 3D virtual model with a freely moving viewpoint.

Major components

Team

Principal investigator

Co-investigators

Project manager

  • Dr. Bin Chan

Development team

  • Yongtao Hu: Multi-target detection and tracking
  • Wei Shao: target segmentation
  • Guanbin Li: cross-camera tracking
  • Frank Shen: User Interface
  • Dr. Calvin Fong: 3D modeling & camera calibration, visualization
  • Darren Li: Body Pose Tracking
  • Dr. Forrest Qin: Body Pose Tracking

Downloads