Supervised Learning Approaches for Automatic Structuring of Videos


Spécialité : Mathématiques et Informatique

22/07/2015 - 14:00 Mr Danila Potapov (INRIA Grenoble) Grand Amphi de l'INRIA Rhône-Alpes, Montbonnot

Mots clé :
  • video classification
  • video summarization
  • computer vision
  • machine learning
Automatic interpretation and understanding of videos still remains at the frontier of computer vision. The core challenge is to lift the expressive power of the current visual features (as well as features from other modalities, such as audio or text) to be able to automatically recognize typical video sections, with low temporal saliency yet high semantic expression. Examples of such long events include video sections where someone is fishing (TRECVID Multimedia Event Detection), or where the hero argues with a villain in a Hollywood action movie (Inria Action Movies). In this manuscript, we present several contributions towards this goal, focusing on three video analysis tasks: summarization, classification, localisation.
First, we propose an automatic video summarization method, yielding a short and highly informative video summary of potentially long videos, tailored for specified categories of videos. We also introduce a new dataset for evaluation of video summarization methods, called MED-Summaries, which contains complete importance-scorings annotations of the videos, along with a complete set of evaluation tools.
Second, we introduce a new dataset, called Inria Action Movies, consisting of long movies, and annotated with non-exclusive semantic categories (called beat-categories), whose definition is broad enough to cover most of the movie footage. Categories such as "pursuit" or "romance" in action movies are examples of beat-categories. We propose an approach for localizing beat-events based on classifying shots into beat-categories and learning the temporal constraints between shots.
Third, we overview the Inria event classification system developed within the TRECVID Multimedia Event Detection competition and highlight the contributions made during the work on this thesis from 2011 to 2014.


  • Mr Zaïd Harchaoui (CHARGE DE RECHERCHE - INRIA Grenoble )
  • Mme Cordelia Schmid (PROFESSEUR - INRIA Grenoble )


  • Mr Ivan Laptev (PROFESSEUR - INRIA Paris )
  • Mr Patrick Perez (PROFESSEUR - Technicolor Rennes )


  • Mr Florent Perronnin (PROFESSEUR - Facebook AL Research Paris )
  • Mr Matthijs Douze (CHARGE DE RECHERCHE - INRIA Grenoble )