Not Just Another Blog: Simple Twitter Sentiment Notification System with ESP8266 Wifi + Arduino

I am a huge fan of Cricket and die hard supporter of Sri Lankan Cricket Team. The World Cup was concluded recently with Australia emerging as champions. New Zealand were runner up and had a great tournament even though they lost the final. With cricket comes a lot of social media interaction. Trolls, Complements, Griefs, Celebrations, you name it all happens in social media platforms.

Out of this twitter stands out because in twitter you can directly interact with players involved and large audience of cricket followers. I wanted to know how twitter users are feeling and talking about cricket matches during the world cup so I developed a small, simple twitter sentiment notification system with Arduino and a recently bought ESP8266 Wifi.

Requirement

The requirement was to just illuminate a red or green light (out, not out in cricket :) ) based on the sentiment score of a particular tweet instead of reading the whole tweet. The idea was to keep everything as simple as possible because this is just a small project with limited usage.

Implementation Decision

For this I used Spring XD which is built for Extreme Data Processing by creating an engine to source , transform and sink from multiple data points. Furthermore I used a Redis list act as the sink from Spring XD and source for the arduino via ESP8266 over wifi.

The sentiment analysis is done by a custom written Processor module for Spring XD which just does negative word count and determines whether the tweet is negative or positive, simple but works for this simple project.

A small java TCP server act as the intermediate between arduino via ESP8266 over wifi and Redis

Architecture

Implementation

Hardware

Software

The source codes are available at https://github.com/shazin/tweetsentiments

Spring XD 1.1.0 was used with Redis 2.8.7, for this to work Spring XD must be started in Single Node mode with the following command

SPRING_XD_HOME/xd/bin/xd-singlenode

Redis server must be started with the following command

REDIS_HOME/src/redis-server

A shell must be opened to Spring XD to execute commands

SPRING_XD_HOME/xd/bin/xd-shell

in the shell following commands must be executed

This registers the custom sentiment processor module in Spring XD

module upload --file /Path/To/sent-processor-0.0.1-SNAPSHOT.jar --name sent-processor --type processor

This creates the stream which sources tweets from twitter, analysis and calculates their sentiment score and sinks in redis. twittersearch --query could be anything you want to search. In this case I have added spring to focus on the word spring.

stream create --name tweetsentiments --definition "twittersearch --query=Spring --consumerKey= --consumerSecret= | sent-processor | redis --queue=sentiments" --deploy

Finally the TCP server must be started and arduino sketch ESPWifi must be loaded into arduino.