Gesture Recognition Project - step 02 - i.MX53 processor

In this second step follows:

1 - pre-processing (color conversions, mathematical morphology and filters)
2 - skin color detection (segmentation)
3 - convexity hull for fingers detection (not used anymore - besides it is still being calculated on the video)
4 - Artificial Neural Network for Pattern Classification (gesture recognition)

Procedure Used for the recognition module:

Artificial Neural Network (Multilayer Perceptron Architecture) trained with the Backpropagation Algorithm. The input data is a downsized binary image of the detected contours on the pre-processed image. The sample and test data to train the ANN was acquired previously using the same application, saving the pixels values (binary image) in a TXT file.

Some issues appeared during the training, since the values of the pixels were 0 for black and 1 for white. And as this ANN is relatively big (50x50 image + limiar value = 2501 inputs) the neurons were getting saturated on the very first training epoch. With the help of my PhD supervisor, Professor Ivan Nunes da Silva (University of São Paulo - EESC) the values of the pixels were normalized to smaller values, -0.1 for black pixels and 0.1 for white pixels. With this approach the ANN was able to converge and learn the different patterns.

The ANN was trained for this second step with only 2 gestures, opened hand and closed hand. And for a quick test, I let only 30 neurons on the hidden layer and it took only 5 minutes to get the ANN trained.

here is the Result:




The images displayed on the LCD are: HSV segmented image, Grayscale segmented Image, Convexity hull (not used for the recognition), the ANN input data (50x50 binary image) and the result of detection.

UPDATE
------

The saturation of the neurons can be also avoided using lower values for the net weights. Before the training, random values must be generated to the net weights (initial random values), make sure that these values are small.

EOF!