5

I am currently working on image segmentation for the purposes of computer vision. I have read many papers and a few books dealing with MRFs and CRFs for computer vision. All of them define an energy function based on the single pixel and neighbouring pixel cliques. However, I have failed to find any reference where the energy function is defined explicitly.

Furthermore, the image I am working on has had its pixels previously clustered as stixels and the images were acquired from a stereo camera system, so the disparity is also available. However, I believe that the adaptation from pixel and pixel neighbourhood to stixel and stixel neighbourhood is straightforward enough.

So I wanted to know what are some standard energy functions for image segmentation? And what would be the best algorithm for optimizing over said function?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
RCountZero
  • 103
  • 6

2 Answers2

4

One such energy function is,

$$ E(x) = \sum_{i} \theta_{i}(x_i) \hspace{0.5em} + \hspace{0.5em} \sum_{ij} \theta_{ij}(x_{i}, x_{j}) $$

where the unary term is $$\theta_{i}(x_{i}) = -log\hspace{0.2em} P(x_{i})$$

and the pairwise term (Potts model) is $$\theta_{ij}(x_{i}, x_{j}) = \mu(x_{i}, x_{j}) \sum_{m=1}^{K}w_{m}k^{m}(f_{i},f_{j})$$

where $\hspace{1em} \mu(x_{i}, x_{j}) = 1, \hspace{0.7em}$ if $x_{i} \ne x_{j} \hspace{2em}$ (think not Kronecker delta function)

$$\sum_{m=1}^{K} w_{m}k^{m}(f_{i},f_{j}) = w_{1} exp\bigg( -\frac{\left\lVert p_{i}-p_{j} \right\rVert^2}{2\sigma^2_{\alpha}} -\frac{\left\lVert I_{i}-I_{j}\right\rVert^2}{2\sigma^2_{\beta}} \bigg) + w_{2} exp\bigg( -\frac{\left\lVert p_{i}-p_{j} \right\rVert^2}{2\sigma^2_{\gamma}} \bigg)$$



Objective: Minimize the above stated energy function E(x)

kmario23
  • 180
  • 8
-1

As for optimizing over the functions, if you are interested in segmentation, two algorithms are particularly common. One is based on graph flow (minimum s-t cut algorithm, read about it here). The other method is to use variational bayes (a bit more difficult to understand, but worth it because it pops up elsewhere frequently). You can read about it here.

c. leather
  • 17
  • 3
  • 1
    Could you add a little more information to summarize the contents of those links? That way your answer is protected against "link rot" - it will still make sense if one of the links stops working. – Silverfish Jul 07 '16 at 20:42