Saturday, November 19, 2011

Arduino Google Voice Activated Servo Motor Controlling using Android

I was having some free time and wanted to put learn Android development. I found a Great Android Application called SL4A (Scripting Layer for Android) which is an intermediate Android app which sits between the Android OS native methods and popular scripting languages such as Shell Script, Python, Perl etc.

It opens up many possibilities to develop applications in Android really quickly with interpreted languages like Python. I always wanted to try out Door unlocking with an Android Google Voice API. For this system I wanted to connect to Wifi network and given the Voice command open or close a Door lock. So I came up with the following Circuit using Arduino;

Things required to build the Prototype.

  1. One Arduino (Any model)
  2. A Breadboard
  3. A Servo Motor
  4. Two LEDs (Green and Red)
  5. Two 1K resistors for LEDs
  6. Jumper Cables
Using the above mentioned equipment I put up the following prototype. 


The Green LED is driven using Pin 2, The Red LED is driven using Pin 3 and the Servo motor is attached to Pin 9 (Analog Output with PWM).

Android phone I have is a Samsung Galaxy ACE with Android OS 2.3.3 Gingerbread. I installed SL4A and  Python for Android to enable Python Scripting in Android.

#=======================================
#=          Shazin Sadakath            =
#=======================================

import android
import socket
# Android API Object SL4A Specific
droid = android.Android();

# Hostname or IP
hostname = "192.168.1.65";
# Port
port = 10000;
data = "";
# Creating a UDP Socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM);
# Connecting to Host (Usually not required since it is UDP)
s.connect((hostname,port));
print "Connected!";

exit = False;
while not exit:
  # Using Android API for Google Voice and Recognizing command
  data = droid.recognizeSpeech().result;
  # Command is open or close proceding it to the Server else Error message
  if data == "open" or data == "close":
    # Sending command
    s.sendto(data,(hostname,port));
    # Recieving an ACK since UDP is unreliable 
    data = s.recvfrom(1024);    
    print "Ack ", data[0];
    # If ACK is success ending the program
    if data[0] == "success":
      exit = True;
  else:
    print "Invalid Command";
   

The Middle Man in this application is a Java Thread based UDP Server which listens for UDP packets from a connected Android app and Does the Serial communication to the Arduino Microcontroller.

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.SocketException;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Shazin Sadakath
 */
public class SpeechDoor implements Runnable {
    private DatagramSocket socket;

    public SpeechDoor() {
        try {
            ArduinoBridge.init("COM27");
            socket = new DatagramSocket(10000);
            new Thread(this).start();
        } catch (SocketException ex) {
            Logger.getLogger(SpeechDoor.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    public void run() {
        byte[] data;
        boolean success;
        DatagramPacket recv, ack;
        while (true) {
            try {
                data = new byte[1024];
                success = false;
                recv = new DatagramPacket(data, data.length);
                System.out.printf("Waiting on Socket : %s:%d\n", socket.getLocalAddress().toString(), socket.getPort());
                socket.receive(recv);
                System.out.printf("Packet Recieved From : %s:%d\n", recv.getAddress().toString(), recv.getPort());
                String value = new String(data, recv.getOffset(), recv.getLength());
                System.out.printf("Value : %s\n", value);
                if("open".equals(value)) {
                    ArduinoBridge.writeToArduino((byte) 1);
                    success = true;
                } else if("close".equals(value)) {
                    ArduinoBridge.writeToArduino((byte) 2);
                    success = true;
                }
                if(success) {
                    data = "success".getBytes();                    
                } else {
                    data = "fail".getBytes();
                }
                ack = new DatagramPacket(data, data.length, recv.getAddress(), recv.getPort());
                socket.send(ack);
                success = false;
                Thread.sleep(1000);
            } catch (Exception ex) {
                Logger.getLogger(SpeechDoor.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
    }

    public static void main(String[] args) {
        new SpeechDoor();
    }
}


Arduino Serial Communication is done using the RXTX Library for Serial Communication.


import gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;
import java.io.OutputStream;
import java.util.Enumeration;

/**
 *
 * @author Shazin Sadakath
 */
public class ArduinoBridge {

    private static SerialPort port = null;
    private static CommPortIdentifier cpi = null;

    public static void init(String p) {
        Enumeration enums = CommPortIdentifier.getPortIdentifiers();

        while (enums.hasMoreElements()) {
            cpi = (CommPortIdentifier) enums.nextElement();
            if (p.equals(cpi.getName())) {
                break;
            }
        }

        if (cpi != null) {
            try {
                port = (SerialPort) cpi.open("ArduinoJavaBridge", 1000);
                if (port != null) {
                    port.setSerialPortParams(9600,
                            SerialPort.DATABITS_8,
                            SerialPort.STOPBITS_1,
                            SerialPort.PARITY_NONE);
                }

                System.out.println("Ready!");
                //new Thread(new ArduinoBridge()).start();

            } catch (Exception e) {
                e.printStackTrace();
            }


        }
    }

    public static void writeToArduino(byte value) {
        try {
            
            OutputStream os = null;

            os = port.getOutputStream();
            
            os.write((byte) 0xff);
            os.write((byte) (value));
            os.flush();        

            if (os != null) {
                os.close();
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

   
}


Finally the Arduino Sketch which is Loaded in to the Microcontroller


/* 
 * Shazin Sadakath
*/

#include <Servo.h>
#define MAX 150
#define MIN 0

int greenLedPin = 2;
int redLedPin = 3;
int servoPin = 9;
int command = 0;
boolean open = false;

Servo doorLock;

void setup() {
 Serial.begin(9600);
 doorLock.attach(9);
 pinMode(greenLedPin, OUTPUT);
 pinMode(redLedPin, OUTPUT);
 digitalWrite(greenLedPin, LOW);
 digitalWrite(redLedPin, HIGH);
}

void loop() {
 if(Serial.available() >= 2) {
  if(Serial.read() == 0xff) {
    command = Serial.read();
      if(command == 1) {
        if(!open) {
         doorLock.write(MAX); 
         digitalWrite(redLedPin, LOW);
         digitalWrite(greenLedPin, HIGH);
         open = true;
        }
      } else if(command == 2) {
        if(open) {
         doorLock.write(MIN);
         digitalWrite(greenLedPin, LOW);
         digitalWrite(redLedPin, HIGH);
         open = false;
      } 
     }     
  }  
 }  
 delay(10);
}

The video of the System is below. The accuracy completely depends on the way you pronounce the words "open" and "close", Surrounding noise and Google Voice API. In this video as you can see the The "open" word is not recognized first time. But the close word is.



Constructive Criticism is always welcome!

Trackbacks/Pings
  1. An Exercise in Servo Voice Control with Android - Hackaday 
  2. A Voice Activated Servo - SL4A Tutorials  

13 comments:

b1ackmai1er said...

Hi,

Great project. Are you able to add more information on how to install the software. I assume the first lot of code is python code. So that gets saved as text file on the phone with a .py extension and is run by the 4SLA script app? Obviously the Sketch gets loaded into the Arduino.
The two middle sections of code - is that C? Do I need visual C to compile this ? Regards Phil

s2_zat said...

Hi Blackmailer,

Yes you are correct. Python script just needs to saved in a .py extension file and can be run in SL4A application with Python for Android installed. And Sketch goes into Arduino. The Middle Man is a Java Thread based code. It just needs to compiled using javac and can be run in any OS with a JVM.

b1ackmai1er said...

Thanks. Will give it a try. I installed Eclipse, the Java development kit and the Android Dev elopement kit. Would Javac be part of the JavaDK? Can I compile the java code through the Eclipse development environment? I will see how far I get based on these assumptions. Thanks.

s2_zat said...

Hi Phil

yes it is part of jdk. Just create 2 classes with the exact name I have given and copy and paste and run the class with the main method. Of course you need an arduino to be connected and com port for ur arduino needs to specified in arduinobridge.init method. Let me know how you progress.

Vali said...

I am working in similar projects. I got the 4SLA and Python installed and the arduino code running.

I still don't get it with Javac. I have Eclipse and tested a couple of apps with arduino and android but I have no idea how to create a Java Thread based UDP Server :|


In your video I don't see any connections between arduino and the phone... cable, usb, wifi ?!

In the prototype picture are there led s plus connected to the ground (GND)?

s2_zat said...

Hi Vali,

Just create a New Java Project in Eclipse. Create 2 classes named SpeechDoor and ArduinoBridge.

Copy and Paste the Java Code for SpeechDoor and ArduinoBridge in those classes.

Download RXTX Library from here

http://rxtx.qbang.org/wiki/index.php/Main_Page

and follow the installation instructions. and set the project class path to point to RXTX.jar

Change the ArduinoBridge.init("COM27") to point to the comm port where your arduino board is connected. Right click on SpeechDoor class and Run as Java Application. The UDP Server is Up :).

The architecture is, From Android to UDP Server it connects through WiFi, From UDP Server to Arduino through USB Serial communication.

Of course the LEDs are grounded using the Bottom Horizontal connectors in the Bread board or else the LEDs won't illuminate.

Let me know how you progress.

Vali said...

thanks for the info

I've been digging into this idea and there are important resources to be taken into consideration (without witch the project is almost impossible to build :D )
1. http://www.arduino.cc/playground/Interfacing/Java
2. a book called "Pro Android Python with SL4A"

Thanks for the post!

Android app development said...

This is one of the Important post.I like your Talent.This is one of the well organized post.Android app developers

My Blog said...

Hi !
I am supposed to make an android-robot with 2 servos and a propellar in an Arduino board. If you can, please suggest me other list of necessary componets and which programming language to be used and also the circuitry of design. Reply waiting

Mark Ferran BSEE said...

It seems that three processors are required/programmed to make this control system function:
1) Android apps converting audio commands to digital bits, and controlling WiFi output of binary commands.
2) Host PC running Java to provide Wifi-to-USB communication of binary commands
3) Arduino board to convert binary commands (receive via USB) into stepped-analog/binary voltage signals to operate LEDs and servo motor.

I would want to simplify and generalize this solution to operate without the Host PC, so that it can be used to control a Remote-control toy car for example. Since my Android phone (Samsung Infuse) has BlueTooth, I think I would only need a bluetooth interface for the Arduino board. I would also want to provide a manual-entry/GUI to control the switches, not voice-commands, since interpretation of the computer voice commands is slow and uncertain. My fingers are faster than my tongue. Is there a "visual basic" app for the Android phone to create a GUI that can call upon a Bluetooth function of the phone? Is there a plug-and-play Bluetooth modem for the Arduino? Please respond also by sending your response to MR FERRAN >at> nycap DOT rr DOT com. (no spaces between capitalized letters in the username)

fatmi said...

very effective post and discusions, were very helpful.
what is arduino

Aung Thiha said...
This comment has been removed by the author.
Rolf said...

Google Bluetoothchat and Arduino. This is a setup I had with he armino project. I am moving to the btc code to remove the a amarino middleware app.

http://www.youtube.com/watch?v=BXAOf7vJYEU