I am a huge fan of Cricket and die hard supporter of Sri Lankan Cricket Team. The World Cup was concluded recently with Australia emerging as champions. New Zealand were runner up and had a great tournament even though they lost the final. With cricket comes a lot of social media interaction. Trolls, Complements, Griefs, Celebrations, you name it all happens in social media platforms.
Out of this twitter stands out because in twitter you can directly interact with players involved and large audience of cricket followers. I wanted to know how twitter users are feeling and talking about cricket matches during the world cup so I developed a small, simple twitter sentiment notification system with Arduino and a recently bought ESP8266 Wifi.
Requirement
The requirement was to just illuminate a red or green light (out, not out in cricket :) ) based on the sentiment score of a particular tweet instead of reading the whole tweet. The idea was to keep everything as simple as possible because this is just a small project with limited usage.
Implementation Decision
For this I used Spring XD which is built for Extreme Data Processing by creating an engine to source , transform and sink from multiple data points. Furthermore I used a Redis list act as the sink from Spring XD and source for the arduino via ESP8266 over wifi.
The sentiment analysis is done by a custom written Processor module for Spring XD which just does negative word count and determines whether the tweet is negative or positive, simple but works for this simple project.
A small java TCP server act as the intermediate between arduino via ESP8266 over wifi and Redis
Architecture
Implementation
Hardware
Software
The source codes are available at https://github.com/shazin/tweetsentiments
Spring XD 1.1.0 was used with Redis 2.8.7, for this to work Spring XD must be started in Single Node mode with the following command
Spring XD 1.1.0 was used with Redis 2.8.7, for this to work Spring XD must be started in Single Node mode with the following command
SPRING_XD_HOME/xd/bin/xd-singlenodeRedis server must be started with the following command
REDIS_HOME/src/redis-serverA shell must be opened to Spring XD to execute commands
SPRING_XD_HOME/xd/bin/xd-shellin the shell following commands must be executed
This registers the custom sentiment processor module in Spring XD
module upload --file /Path/To/sent-processor-0.0.1-SNAPSHOT.jar --name sent-processor --type processorThis creates the stream which sources tweets from twitter, analysis and calculates their sentiment score and sinks in redis. twittersearch --query could be anything you want to search. In this case I have added spring to focus on the word spring.
stream create --name tweetsentiments --definition "twittersearch --query=Spring --consumerKey= --consumerSecret= | sent-processor | redis --queue=sentiments" --deploy Finally the TCP server must be started and arduino sketch ESPWifi must be loaded into arduino.
java -cp jedis-2.6.2.jar:./ Server
Demonstration
References
- ESP8266 Arduino Library - https://github.com/itead/ITEADLIB_Arduino_WeeESP8266
- Spring XD - http://docs.spring.io/spring-xd/docs/current-SNAPSHOT/reference/html/
- Redis - http://redis.io/
