New Methodology and Software for Improving Subtitle Presentation in Movies and Videos
07/2014 - 12/2015
ITSP project (Project Reference : ITS/226/13) with 1.4M HKD funding.
Sponsor: The Hong Kong Society for the Deaf (HKSFD).
  • project page
  • project summary
    Subtitles provide valuable aids for understanding conversational speeches in movies and videos. However, the traditional way of placing subtitles at the bottom of the video screen causes eyestrain for ordinary viewers because of the need to constantly move the eyeballs back and forth to follow the speaker expression and the subtitle. The traditional subtitle is also inadequate for people with hearing impairment to understand conversional dialogues in videos.
    
    A new technology will be developed in this project that improves upon the traditional subtitle presentation in movies and videos. In this improved presentation, subtitles associated with different speakers in a video will be placed right next to the associated speakers, so viewers can better understand what is spoken without having to move the eyeballs too much between the speaker and the subtitle, thus greatly reducing eyestrain. Furthermore, with the aid of the improved subtitle, viewers with hearing impairment can clearly associate the speaker with the spoken contents, therefore enhancing their video viewing experience.
    
    We shall apply advanced computer vision techniques to face detection and lip motion detection to identify speakers in a video scene. We shall also study the optimal placement of subtitles around an identified speaker. Finally, a prototype software system will be developed that takes a subtitled video in a standard format and produces a new video with the improved subtitle presentation. As indicated by our initial user study, the outcome of this project holds the promise of benefitting all the people viewing a subtitled video and in particular those with hearing difficulty.

 

An Advanced Video Fusion System for Security Investigations
06/2012 - 09/2014
ITSP project (Project Reference : GHP/060/11) with 5M HKD funding.
Sponsors: Microsoft HK and Cyberview.
  • project page
  • project summary
    Video surveillance is deployed everywhere in HK to enhance public security. Security investigation is still much relied on checking 2D videos from separate camera views.
    
    This project aims at analysing and fusing videos from multiple cameras to create an informative and easy-to-comprehend reenactment of a past event to assist investigations, providing a global understanding with both space and time registration for complex scenarios and enabling investigators to have a close-up check of actions from novel viewing directions.
  • media

 

3D Infrared Building Modeling
03/2011 - 06/2011
Work with HKU-ME.
"3D simulation for visualizing infrared information of buildings"