Many computer vision applications require to go beyond classification (i.e. simply listing the objects that appear in an image or a video sequence), and to locate these objects. This is the object detection task.
Most of the existing object detection methods cast detection as a classification problem using a sliding window approach: many possible windows in the image are considered in turn and classified to decide if the window contains an object or not. This type of approach is successful but extremely costly as such a sequential system has to classify thousands of windows for a single input image.
Our group has proposed a totally different way of tackling object detection, that we refer to as data-driven detection. Detection is cast as a retrieval problem, and object detection is performed using a single global image descriptor, reducing significantly the cost of detection with this approach.
- Predicting an Object Location using a Global Image Representation José A. Rodriguez, Diane Larlus. ICCV 2013
- Data-Driven Detection of Prominent Objects José A. Rodriguez, Diane Larlus, Zhenwen Dai. TPAMI. 2014.
The task of semantic segmentation, also called scene parsing, is to semantically label each pixel of an image, dividing an image into semantic regions, such as sky, water, tree and building in the example above. It is related to the object detection task.
Our group has a long history of contributions in semantic segmentation. A simple and yet very efficient method was proposed [BMVC08] that won the PASCAL VOC segmentation challenge in 2008. Some improvements have been proposed over the years ([IJCV11] [ICVGIP12]). The group also reflected on how to evaluate segmentation in the fairest and most meaningful manner [BMVC13].
We also consider segmentation when images are enriched with additional information. For instance, near-infrared (NIR) information is available for free from most consumer digital cameras, and we showed that it could be successfully used to improve segmentation [ECCV12] (more information about this work can be found on the website of our open innovation partners at EPFL)
- What is a good evaluation measure for semantic segmentation? Gabriela Csurka, Diane Larlus, Florent Perronnin. BMVC 2013
- On the use of Regions for Semantic Image Segmentation Rui Hu, Diane Larlus, Gabriela Csurka. ICVGIP 2012.
- A Simple High Performance Approach to Semantic Segmentation Gabriela Csurka, Florent Perronnin. BMVC 2008
- Semantic Image Segmentation Using Visible and Near-Infrared Channels .Neda Salamati, Diane Larlus, Gabriela Csurka, Sabine Süsstrunk. Workshop on Color and Photometry in Computer Vision at ECCV 2012.
- Incorporating Near-Infrared Information into Semantic Image Segmentation Neda Salamati, Diane Larlus, Gabriela Csurka, Sabine Susstrunk. 2014
Beyond object detection
Our group also explores tasks that go beyond the detection of objects, and that get a better understanding of the objects themselves and their structure. In particular, we look at the task of object part detection, in scenario with little supervision. We have also studied how well object part detectors could transfer across object categories, and which level of supervision is required to insure building good detectors.
- I Have Seen Enough: Transferring Parts Across Categories David Novotny, Diane Larlus, Andrea Vedaldi, BMVC 2016
- Learning the semantic structure of objects from Web supervision David Novotny, Diane Larlus, Andrea Vedaldi. Workshop @ ECCV 2016