Computer Vision on Embedded Systems: February 2011

Thursday, February 24, 2011

Gesture Recognition Project - step 04 - i.MX53 processor

New achievement ! the ANN was trained to recognizes 8 gestures, but only 7 are showed in this post (the 8th one was censored =D).

Here is the Mean square error evolution during the ANN training:

(MSE value x Training Epochs)

The main problem on this step is the patterns used to train the ANN, they are too similar and sometimes the ANN can get confused or understand an input data as a corrupted data and then it does the approximation erroneously. See the results below:

From this point, some investigations will be done on:

1) improve the hand detection, using glove ?!?! covering the arms ?!?!?
2) use new gestures to apply in an useful use-case
3) detect movements and recognizes motion gestures

EOF!

Wednesday, February 23, 2011

Gesture Recognition Project - step 03 - i.MX53 processor

Alright, one more video regarding one more step on this project.
So, What is new ?

In this step some modifications on the ANN were done, as follows:

1 - changed the output method, more neurons in the output layer
2 - changed the ativation function to tangent hyperbolic
3 - added the MOMENTUM to the Backpropagation algorithm

The ANN was trained now with three different patterns, the result can see below:

Notice that the little delay on the video is due to a lot of images being displayed at the same time and the dirty code wrote during the development.

For the next steps I will clean up the code (if I got time =)).

EOF!

Monday, February 21, 2011

Gesture Recognition Project - step 02 - i.MX53 processor

In this second step follows:

1 - pre-processing (color conversions, mathematical morphology and filters)
2 - skin color detection (segmentation)
3 - convexity hull for fingers detection (not used anymore - besides it is still being calculated on the video)
4 - Artificial Neural Network for Pattern Classification (gesture recognition)

Procedure Used for the recognition module:

Artificial Neural Network (Multilayer Perceptron Architecture) trained with the Backpropagation Algorithm. The input data is a downsized binary image of the detected contours on the pre-processed image. The sample and test data to train the ANN was acquired previously using the same application, saving the pixels values (binary image) in a TXT file.

Some issues appeared during the training, since the values of the pixels were 0 for black and 1 for white. And as this ANN is relatively big (50x50 image + limiar value = 2501 inputs) the neurons were getting saturated on the very first training epoch. With the help of my PhD supervisor, Professor Ivan Nunes da Silva (University of São Paulo - EESC) the values of the pixels were normalized to smaller values, -0.1 for black pixels and 0.1 for white pixels. With this approach the ANN was able to converge and learn the different patterns.

The ANN was trained for this second step with only 2 gestures, opened hand and closed hand. And for a quick test, I let only 30 neurons on the hidden layer and it took only 5 minutes to get the ANN trained.

here is the Result:

The images displayed on the LCD are: HSV segmented image, Grayscale segmented Image, Convexity hull (not used for the recognition), the ANN input data (50x50 binary image) and the result of detection.

UPDATE
------

The saturation of the neurons can be also avoided using lower values for the net weights. Before the training, random values must be generated to the net weights (initial random values), make sure that these values are small.

EOF!