Matlab Code for Automatic Gesture Controlled Robot

%---------- int camera ---------------------


hwInfo = imaqhwinfo('winvideo');
device1 = hwInfo.DeviceInfo(1);
device1.SupportedFormats;

vid=videoinput('winvideo',1,'YUY2_320x240');
set(vid,'ReturnedColorSpace','rgb');

triggerconfig(vid,'manual');
    set(vid,'FramesPerTrigger',1);
    set(vid,'TriggerRepeat', 100000);
    start(vid);

  %------- pp db 25 int commands ------------

dio1=digitalio('parallel','lpt1');
outreg=addline(dio1,0:7,0,'out');
%-------- taking values -------------

trigger(vid);
image1=getdata(vid,1);
imshow(image1);
%-------- main loop -----------------

for i=1:1500

trigger(vid);
image1=getdata(vid,1);

image1_r=image1(:,:,1);
image1_g=image1(:,:,2);
image1_b=image1(:,:,3);
image1_r1=im2bw(image1_r,140/255);
image1_g1=im2bw(image1_g,130/255);
image1_b1=im2bw(image1_b,130/255);

img=(image1_r1-image1_g1)-image1_b1;
img=img*255;
img=uint8(img);



img=bwareaopen(img,30);
img1=imfill(img,'holes');

[x1 y1]=bwlabel(img1);

if (y1 >0)
        [x y]=find(img1==1);
        mean_x=fix(mean(x));
        mean_y=fix(mean(y));

        if (mean_x > 60 && mean_x < 180 && mean_y > 80 && mean_y < 240)
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),0);
 
        else if (mean_x > 0 && mean_x < 60 && mean_y > 80 && mean_y < 240)
                putvalue(dio1.line(1),1);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),0);
        else if (mean_x > 180 && mean_x < 240 && mean_y > 80 && mean_y < 240)
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),1);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),0);
 
        else if (mean_x > 0 && mean_x < 240 && mean_y > 0 && mean_y < 80)
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),1);
                putvalue(dio1.line(4),0);
 
        else if (mean_x > 0 && mean_x < 240 && mean_y > 240 && mean_y < 320)
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),1);
 
        else
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),0);
 
            end
            end
            end
            end
        end
 
                string=strcat(num2str(mean_x),'and',num2str(mean_y));

                image2(:,:,1)=double(image1(:,:,1)) .* double(img1);
                image2(:,:,2)=double(image1(:,:,2)) .* double(img1);
                image2(:,:,3)=double(image1(:,:,3)) .* double(img1);

                image2=uint8(image2);

                subplot(1,2,1)
                imshow(image1)
                title('orignal image')

                subplot(1,2,2)
                imshow(image2);
                title(string);
else
                subplot(1,2,1)
                imshow(image1)
                title('no object detected')
             
                subplot(1,2,2)
                imshow(image1)
                title('no object detected')
             
                putvalue(dio1.line(1),0);
                putvalue(dio1.line(2),0);
                putvalue(dio1.line(3),0);
                putvalue(dio1.line(4),0);
end

end

Poster Presentation

Our group participated in the National Symposium On Interdisciplinary Sciences held at GGM Science College, Jammu and presented a Poster there.. Check out the poster.

Control Music Using Gesture in MATLAB

I recently develop a coding in matlab for controling music through ur finger motion.

In this program when ever you bring a red colour in front of the webcam then only your song will play otherwise it will stop.

This program is in the developing state and im modying it to be more realistic.

Here is the code in matlab:

%-----------int music file and audio player--------

[y f] = wavread('kh.wav');

p1=audioplayer(y,f);

%---------- int camera ---------------------

hwInfo = imaqhwinfo('winvideo');

device1 = hwInfo.DeviceInfo(1);

device1.SupportedFormats;

vid=videoinput('winvideo',1,'YUY2_320x240');

set(vid,'ReturnedColorSpace','rgb');

triggerconfig(vid,'manual');

    set(vid,'FramesPerTrigger',1);

    set(vid,'TriggerRepeat', 100000);

    start(vid);

    

%-------- taking values -------------

trigger(vid);

image1=getdata(vid,1);

imshow(image1);

for i=1:1500

trigger(vid);

image1=getdata(vid,1);

image1_r=image1(:,:,1);

image1_g=image1(:,:,2);

image1_b=image1(:,:,3);

image1_r1=im2bw(image1_r,140/255);

image1_g1=im2bw(image1_g,130/255);

image1_b1=im2bw(image1_b,130/255);

img=(image1_r1-image1_g1)-image1_b1;

img=img*255;

img=uint8(img);

img=bwareaopen(img,35);

img1=imfill(img,'holes');

[x1 y1]=bwlabel(img1);

if (y1 >0)

      [x y]=find(img1==1);

        mean_x=fix(mean(x));

        mean_y=fix(mean(y));

        if (mean_x > 60 && mean_x < 180 && mean_y > 80 && mean_y < 240)

            play(p1);

       

            string=strcat(num2str(mean_x),'and',num2str(mean_y));

                image2(:,:,1)=double(image1(:,:,1)) .* double(img1);

                image2(:,:,2)=double(image1(:,:,2)) .* double(img1);

                image2(:,:,3)=double(image1(:,:,3)) .* double(img1);

                image2=uint8(image2);

                subplot(1,2,1)

                imshow(image1)

                title('orignal image')

                subplot(1,2,2)

                imshow(image2);

                title(string);

        else

            pause(p1);

             subplot(1,2,1)

                imshow(image1)

                title('no object detected')

                

                subplot(1,2,2)

                imshow(image1)

                title('no object detected')

        end

end

end    

--------------------------------------------------------------------

Change the kh.wav file in wavread command with any wave file present in your computer.. and add that file to the current directory.

Download the m-file from the below link:                  

http://www.mediafire.com/?6xf8ni8xff4ppxb

Pic

Applications of Gesture Controlled Devices

Working

The most important tools for the working of AUTOMATIC GESTURE CONTROLLED ROBOT are the human gestures and image processing of these gestures. These gestures are used to direct robot according to our need. Image Processing has huge computational requirements, and it is not possible to run an image processing code directly on a small microcontroller. Hence, for our purpose, the simplest approach would be to run the code on a computer, which has a webcam connected to it to take the images, and the robot is controlled by the computer via parallel port. The code is written in software that provides the tools for acquiring images, analysing the content in the images and deriving conclusions. MATLAB is one of the much such software available which provide the platform for performing these tasks. An image (or any other data like sound, etc.) can be converted to a matrix and then various operations can be performed on it to get the desired results and values. Image processing is quite a vast field to deal with. We can identify colours, intensity, edges, texture or pattern in an image. In this project we would be restricting ourselves to detecting colours (using RGB values) only.

                       The output of a computer i.e. the image processing results of MATLAB is fed to the DB-25 parallel port which interfaces the computer with our gesture controlled robot. DB-25 is 25 pin port, divided in ratio of 13:12, 4 pins out of these pins receive input from PC and transfer that input to four 4N35-optocouplers as shown in Fig. 17. These optocouplers act as isolator and voltage regulator between DB-25 parallel port and microcontroller. The optocoupler application or function in the circuit is to:

Ø  Monitor high voltage

Ø  Output voltage sampling for regulation

Ø  System control micro for power on/off

 If the optocoupler IC breakdown, it will cause the equipment to have low power, blink, no power, erratic power and even power shut down once switch on the equipment.

                         Output from the four optocouplers goes to port B of microcontroller ATMEGA-8. It is a 28 pin controller that controls the movement of robot. Port D of the controller carries output from the controller to L293D H-Bridge Motor controller. The L293D is designed to provide bidirectional drive currents of up to 600-mA at voltages from 4.5 V to 36 V. The output of microcontroller is generally low and motor operates on high voltage and current values. L293D provides necessary voltage and current for running of D.C motor running at 60 rpm. Finally output of L293D motor controller goes to two D.C motor that moves the robot in four directions- left, right, clockwise, and anticlockwise.

ROBOT MOVEMENT	LEFT D.C. MOTOR MOVEMENT	RIGHT D.C. MOTOR MOVEMENT
Forward	Clockwise	Clockwise
Reverse	Anti- Clockwise	Anti- Clockwise
Left Turn	Anti- Clockwise	Clockwise
Right Turn	Clockwise	Anti- Clockwise
Stationary	OFF	OFF

Components list

Components Required:-

S.No.	Name of the Component	Qty.
01	Atmega8 AVR Microcontroller	01
02	IC L293D	01
03	IC 4N37	04
04	Voltage Regulator 7805	01
05	DC Motor- 60 rpm	02
06	Crystal Oscillator- 8 Mhz	01
07	8V Battery	01
08	Parallel Port DB-25	01
09	LED	05
10	Switch	01
11	Capacitor- 470 µF Capacitor- 22 pF	01 02
12	Resistor- 1K	04
13	Webcam	01
14	Connectors with Ribbon wire	02