8

I'm following a computer vision course and I have this exercise: write a program that, given an hand image, it can recognize if the hand is open, closed, in a punch, or holding an "ok" posture, using only the techniques provided till now (pixel 4/8 connected, connected region, contour finding, holes finding, blob property like centroid, area, perimeter, eccentricity, image moments, image transformation like invert/power/log/gamma correction/contrast stretching, histogram computation and equalization).

I have done it with some basic blob properties(closed hand has a low eccentricity, "ok" has a hole, open hand has a big difference between the area of the inscribed ellipse in the blob and the blob area itself with a low eccentricity).. It seems to work but the first image is a little bit problematic.

I think there could be something more to make a more robust algorithm. Maybe some kind of moment property? Could some blob axis/orientation/extreme points help?

PS test images:

ok2

ok

open

punch

Emre
  • 2,807
  • 15
  • 21
nkint
  • 914
  • 8
  • 24
  • Are you allowed to use [skeletonization](http://en.wikipedia.org/wiki/Topological_skeleton)? See [the relevant Wikipedia article](http://en.wikipedia.org/wiki/Gesture_recognition#Algorithms) for more ideas. – Emre Jan 09 '13 at 23:03
  • no is not allowed because it is the content of a future classes! – nkint Jan 11 '13 at 21:46
  • You could try [applying Symbolic Aggregate approXimation](http://www.cs.ucr.edu/~eamonn/Shape_Discords.pdf) to the outer contour of each object. Basically reducing a complex shape to a time-serie and then cluster these in some way. – Ivo Flipse Jan 12 '13 at 13:55
  • i think i don't have to use any kind of machine learning, just a smart way to combine isses written in the question.. – nkint Jan 12 '13 at 16:00
  • maybe some morphological feature? – nkint Jan 12 '13 at 16:45

3 Answers3

2

You can try looking at Hu invariant moments. They can be constructed from basic moments, and are rotation, scale, reflection, and translation invariant.

Calculate them for a set of training contours first, and then apply them to the test contour.

There are implementations in Matlab and OpenCV, as far as I remember.

Paul R
  • 3,272
  • 17
  • 32
C.B.
  • 61
  • 4
2

According to the book Programming computer vision with Python an interesting approach is to use dense SIFT (a.k.a. HoG) features on your images, and feed these features to a classifier.

I didn't try it myself, but it seems quite sound as an approach. Furthermore, the inventor of the HoG feature proposes the Flutter app that worked quite well in my tests, and it would be very weird if the inventor didn't use his own features or a close derivative.

sansuiso
  • 3,869
  • 16
  • 26
2

One of my friends did this for his undergraduate thesis. What he basically did was encode properties of each gesture. For example,in the first figure, take a rectangular mask over theportions of the hand. The parts where the skin meets the rectangular mask edge should be noted and marked. Then the relative positions of the larger edge and the smaller edge can be compared.

So for pointing upwards, you will have a smaller edge upwards and larger edge at the wrist.

For pointing sideways, you have a smaller edge at one side and larger edge at the bottom.

The directions at the very least can be covered in this way.

Naresh
  • 698
  • 4
  • 20