Propose a robust method for object extraction in RGBD images.




GraphCut as an image segmentation technique was originally introduced by Boykov and Jolly [1]. This segmentation technique assigns to the image a weighted graph structure where each pixel is represented by node, and there are two additional nodes, the source and the sink, that represents the labels Foreground (i.e., Object) and Background (not Object ):


The graph has two kind of weighted edges:

This weighted graph induces an energy function that looks for an optimal segmentation taking into account an Smoothness Term (given by the weight of the n-deges) and a Data Term (given by the weight of the t-edges). Implementing a MaxFlow-MinCut procedure[2], the energy function is minimized, and we obtain a pixel labelling which is globally optimal.

Based in the principles of GraphCut, Rother et al. proposed GrabCut [3]. In this new algorithm, Gasussian Mixture Models (GMM) are built from the color data of certain pixels (seeds) to represent the color distribution of Foreground and Background. From these color models the Data Term of the energy function is defined to be a likelihood maximization.

GrabCut has shown good results even in complex images (where the colour distribution of foreground and background overlaps), but in most cases it still requires posterior user interaction (i.e., brushing some zones of pixels incorrectly labelled, and running the algorithm again).

Initial Image and Rectangular SelectionAdditional user interactionFinal Result!

Images from [3]

[1] Y. Boykov and M-P. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. Proc. IEEE Int. Conf. on Computer Vision, 2001.

[2] Y. Boykov and V. Kolmogrov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. on Pattern Analysis and Machine Intelligence, volume 26, pages 1124–1137, 2004.

[3] C. Rother, V. Kolmogorov, and A. Blake. Grabcut — interactive foreground extraction using iterative graph cut. Proc. ACM Siggraph, 2004.

[4] C. Rother, V. Kolmogorov, Y. Boykov, and A. Blake. Interactive Foreground Extraction using graph cut. Microsoft Technical Report:MSR-TR-2011-46.