If you found the contents in this blog useful, then please make a donation to keep this blog running. You can make donations via Skrill with email address

Sunday, February 18, 2018

Using ThingsBoard, MQTT, Arduino to Monitor Plant Soil Moisture, Temperature and Humidity - Part 1


Recently I got a rose plant from one of my friends who got married. I have been having plants of all sorts at my home but as any busy guy I forget to water the plants. This is a problem I thought I need to address using technology.


After doing some research I got to know about a Platform called ThingsBoard which allows Devices to connect to a central server using HTTP, MQTT, CoAP protocols and send telemetry to visualize everything in a Dashboard. I quickly downloaded it, used my Ethernet Shield, DHT11 sensor and YL-69 Soil Moisture sensor which I had lying around to check the possibility of integrating everything together.


I just connected YL-69 Soil Moisture sensor to Arduno Analog Pin A1 to read the sensor data and Digital Pin 7 to power it if and when required. I connected Digital Pin 6 to read the sensor data from DHT11 sensor. Finally I followed the ThingsBoard Installation guide [1] and Getting Started guide [2] to setup ThingsBoard server in my laptop.

Finally wrote the following Arduino Sketch to use Ethernet Shield to connect to the network and ThingsBoard via MQTT protocol to send data.

Arduino sketch can be found below;

Data was sent in following format.
{"temperature":29.00, "humidity":30.00, "moisture":10, "active": false}

And was visualized in Dashboard Widgets like below;

You can see a Demo below where the Blue Color Soil Moisture sensor reading go up and down based on inserting and removal of the YL-69 sensor probe and Temperature and Humidity going up when I exhale hot air into the DHT11 sensor.

Future Work

Future work would be to add ESP8266 wireless connectivity to the entire thing and using ThingsBoard rule to fire and send an email when soil moisture level goes down below a threshold.



Tuesday, January 30, 2018

Git Commit Information in Spring MVC

At time there is a need for developers the need to know which Git Commit from which Git Branch was used to build a release so that they can keep track of the versions.
Spring Boot Actuator is tailor made for this if you are working on Spring Boot but if you are like me and not using Spring Boot for a specific project but only using Spring MVC then this guide will be helpful for you.
First of all you need to add the spring-boot-actuator
And then instead of relying on Spring Boot Autoconfiguration, in your Java Annotation configuration you can do the following;
And in order to generate the file you can use the Maven git-commit-id plugin and use it as below;
                 <name>validating project version</name>  
And when packaging you can use the following command
$ mvn clean package git-commit-id:revision 
Now when you access http://://info endpoint you will see the git commit id, branch name, commit time etc. and you can customize the pom.xml git-commit-id plugin configuration to have more information.

Tuesday, December 26, 2017

Java 9 Migration Tips

Java 9 was released this year and it is a much awaited release of Java with a lot of interesting and exciting features. But Upgrading is a concern for any CTO or Infrastructure Team as it may break the existing functionality or create unforeseen issues.

But in a recent presentation, Bernard Traversat who is the Vice President of Development for the Java SE Platform at Oracle provided some tips on successful migration to Java 9. Bernard manages the development team responsible for the Java Language, JVM, Core libraries, Java UI toolkits, and Java deployment stack for Oracle JDK product. So who else is fit enough provide advise on Java 9 Migration than him.

Why Upgrading is required;

  • Upgrades include Security fixes    
  • Upgrades include Regressions fixes    
  • New features with releases
  • More performance    
  • More robustness    
  • Because the top or bottom tier was upgraded (OS’s, Hardware)    
  • Reduce cost of operation via uniformity

Why Upgrading is delayed;

  • Fear of something is going to go wrong.
  • It will cost money if something goes wrong. According InformationWeek IT Downtime Costs $26.5 Billion in Revenue.
  • It requires extra work which costs time and money.
Out of Programming Languages Java, Golang, Python, C# .NET, As per Bernard Only Java is Backward compatible and upgradable.

There are binary, source and runtime compability explicit managed in the Java platform.
Most of the APIs (java.*, java.*) undergo millions of tests to ensure compatibility and have specifications managed under the JCP (Java Community Process).
In order to update a specification of an API, rigorous JCP process is required.


The following things remain the same from Java x to Java 9

  • The class path 
  • Class loading 
  • Not forced to migrate to modules
  • sun.misc.Unsafe works
  • Most existing code should work
  • The IDEs, Maven, etc. all have support for JDK 9 already

Things to look out for when migrating from Java x to Java 9

  • You will have to upgrade the libraries and tools that you use    
  • Build or deployment needs to be adjusted If you use any of the components that are shared between Java SE and Java EE 
  • A small number of supported APIs have been removed    
  • A number of non-API features and tools have been removed        
  • Libraries that you use may show some warnings

Incompatibility issues you will face during or after migration;

No 1 issue will be a Third Party library you use in your older Java code is not compatible with Java 9 anymore. This is addressed as many open source projects and vendors are already changing their code base to Java 9 but in case one hasn't done already until they do your migration will have issues.

Apart from that following can also affect you and you must be aware;

1. JEP-260 Encapsulate most internal APIs - This means Internal APIs will no longer be accessible to developers. If your code uses an internal API then it needs to change.

2. JEP-223 New Version String Scheme - JDK 1.9.0_25 will turn into JDK 9.1.3. If your code contains any version string checking then it has to change.

3. JEP-220 Modular Runtime Images - No more rt.jar and JDK directory structure will change. If your code has any JDK directory checking then it needs to change.

4. Most sun.misc.* and sun.reflect.* are removed - If your code uses any of these packages then it needs to change. Ex:- you can no longer use sun.misc.Base64 class.

5. JEP-214 Remove Java 8 Deprecated GC Combinations - Following GC Combinations will now produce errors. You need to change your Java start up scripts if you use any.

DefNew + CMS       : -XX:-UseParNewGC -XX:+UseConcMarkSweepGC
ParNew + SerialOld : -XX:+UseParNewGC
ParNew + iCMS      : -Xincgc
ParNew + iCMS      : -XX:+CMSIncrementalMode -XX:+UseConcMarkSweepGC
DefNew + iCMS      : -XX:+CMSIncrementalMode -XX:+UseConcMarkSweepGC -XX:-UseParNewGC
CMS foreground     : -XX:+UseCMSCompactAtFullCollection
CMS foreground     : -XX:+CMSFullGCsBeforeCompaction
CMS foreground     : -XX:+UseCMSCollectionPassing

6. Launching JVM with Unrecognized VM Options will result in failure to start

7. java.awt.peer and java.awt.dnd.peer packages will be hidden - If your code uses these it needs to be changed to use supporting APIs

8. Calling Thread.stop(Throwable) method will throw an Exception - This was deprecated for a long time now.

9. JEP-271 Unified GC Logging - Re-implemented GC Logging with the new JVM Logging framework JEP-158, If your Java start up scripts use any of the following they need to change.

-XX:+PrintGC will be changed to -Xlog:gc
-XX:+PrintGCDetails will be changed to -Xlog:gc*
-xloggc: will be changed to -Xlog:gc:

10. JEP-248 G1 is the new Garbage Collector - Serial GC is and will remain the default garbage collector for Windows 32 bit.


Migration to Java 9 is not as scary as it looks and will be easy with proper planning. Inorder to harness the full features it is advised to migrate
in small increments so that issues arising can also be mitigated easily.


Friday, December 8, 2017

Writing Reactive Repositories for Spring Data with Mongodb

1. Overview

Reactive Programming has been alive for sometime now. Programming frameworks like Akka, Reactive Streams, Reactor, RxJava etc are good examples. In simple terms reactive programming is about writing non blocking software that are asynchronous and event driven.
Reactive Programming requires a small number of threads to scale vertically (Scale up inside a single JVM) instead of horizontally (Scale out to different nodes by means of clustering).
With Spring 5.0 there is out of the box support for Reactive Programming and now Spring Data project also has Reactive support. Now we will be looking at those latest features here in detail:

2. Setup

In order to use Spring Data Reactive Repositories we need to include spring-boot-starter-data-mongodb-reactive, de.flapdoodle.embed.mongo (for testing), rxjava and rxjava-reactive-streams. Plus reactive mongo-db driver is needed to make full use of the reactive capabilities. The maven dependencies will look like below:

Complete pom.xml file can be found at the Github repository listed at the conclusion section.

3. Enabling Spring Data Reactive Repositories for Mongodb

As the title suggests we will have a look the Spring Data Reactive Repositories with Mongodb. The new @EnableReactiveMongoRepositories is introduced to enable Reactive Repository support for Mongodb. The following configuration enables Spring Data Reactive Repositories for Mongodb:

4. Reactive Repositories

Spring Data project uses the repositories programming model which is the most high-level abstraction to deal with data. They’re comprised of a set of CRUD methods defined in a Spring Data provided interface and domain-specific query methods.
Mainly by using an interface named CrudRepository which exposes methods like findOne, delete, save. With Reactive Programming support Spring Data project has now introduced two more interfaces named ReactiveCrudRepository and RxJava2CrudRepository (for RxJava project support)
A typical Spring Data Reactive Repository would look like below:

And a RxJava2 version of the same Repository would look like below:

Note that Spring 5.0 Reactor Project specific Flux is returned in ReactiveTaxiRepository and RxJava project specific Flowable is returned in RxJava2TaxiRepository.
These repositories are really identical to standard Spring Data Repositories except for the fact that now they can return and/or accept as parameters, reactive elements such as Flux, Mono and Flowable.  By default, reactive repositories use Project Reactor types but other reactive libraries can also be used such as RxJava2 as shown above.

5. Using Spring Data Reactive Repositories for Mongodb

When using the new Spring Data Reactive Repositories, We can use the full features of Reactive Programming provided by the entities Flux, Mono or Flowable (RxJava2).
And now we look at Reactor version would like below:

And now we look at the RxJava2 version:

The above codes will find Taxis by Number CAL-4259 and collect that Flux stream or Flowable stream into a List and will block until the collection is finished.

6. Streaming Data with Tailable Cursor

Spring Data Reactive Repositories provide a way to Stream Data as it arrives into Mongodb with a @Tailable annotation, sort of like an Event Source system. Sticking to our example of Taxis, we can Subscribe to a Tailable Stream and while being subscribed, insert Taxi entities into Mongodb.
This will enable to see newly added Taxi entities coming into system in real time in a streaming manner until the subscription is disposed of. Simulating Taxis entering into a City in real time:

7. Advantages

Compared to a Standard Spring Data Repository, A Reactive Repository provides all the features of Reactive Programming to Data Retrieval. Just by using Reactive Repositories, we can easily filter, process, aggregate data returned declaratively and use asynchronous capabilities provided out of the box in Reactive Programming.

8. Conclusion

Reactive Programming provides a lot of features such Functional, Declarative style of Coding which is being rapidly adopted by developers and enables to write scale-able, easy to understand code.
Now with Spring Data Reactive Repositories these features can be easily incorporated into the existing features of Spring Data project. The complete Source code for the project can be found at GitHub .

Tuesday, October 3, 2017

Java + Spring Boot implementation of Blockchain

Blockchain is the buzz word these days as it is technology used by Cryptocurrencies which is taking over the world like crazy. There are so many articles written on the Theory of Blockchain like this one but there are very little implementations. But recently I stumbled upon an article by Daniel van Flymen who happens to have written a Python based implementation of the Blockchain.

According to him and I am pretty sure many Software Engineers would agree that the best way to learn something is to actually implement it so that you learn more than you get to learn while reading theory documentation.

So this inspired me to write Java + Spring Boot based Implementation of Daniel's work so that I can learn the Blockchain workings. For everyone who is interested following is a basic version of how a Blockchain will look like in the wild.

So Daniel talks a lot about the details of the Blockchain which you can read but in short following are the Steps required to write a Blockchain implementation.

  1. Building a Blockchain
  2. Building an API to access the Blockchain
  3. Interacting with the Blockchain 
  4. Consensus
You can see all above Steps implemented in Java + Spring Boot at

Monday, September 18, 2017

Server Sent Event Processing in Spring MVC 4.2

In a previous post I spoke about the features Spring MVC 4.2 has from Streaming Request Processing. I have discussed how we can StreamingResponseBody to send large data asynchronously in a streaming manner. 

In this post I am planning to talk about SseEmitter which is another way to send semi structured data clients in an asynchronous streaming manner.

If you want to learn more about the definition and structure of a Server Sent Event you can read the Mozilla Documentation. But in short a Server Sent Event is a push event from a server which will be taking place inside of a single TCP/IP Socket connection from Client to Server. This event being push means it eliminates the need for constant polling of the Server for data thus reduces unwanted load on Server and the Client can receive the data in real time.

In Spring MVC following controller code can be written to send server sent events easily and in the client end Javascript can be used to read these events and present in the web page. For demo purpose I am showing a live Cricket match score and commentary coming down to a Web page client in real time. The server code will look like below; 

And Javascript client client code will look like below;

And as you can see from the image below there is only one request being initiated to /score end point but multiple commentary data being received through that long lived connection.

Saturday, September 9, 2017

Spring Web Flux - The Non Blocking Asynchronous Functional Reactive Web Framework - 2

Spring Web Flux Framework can be used with its latest Release Candidate Release 5.0.0.RC3 if you want to try it out. You just need to add the Spring Milestone Repository in your gradle or maven file.

I have written the following service which emulates a delay of maximum 1000 milliseconds in a service to test how the conventional Spring Web MVC vs Spring Web Flux work.

And the following controller implementations one with a Blocking conventional Spring Web MVC Controller method which returns a list to get people and a Non Blocking Spring Web Flux Controller method which returns a Fluxwhich is a defferred result and processed differently.

And Tested sending 1000 concurrent requests to both methods using a Gatling test. The Application was running inside of a Tomcat Container.

Spring Web MVC results were

And Spring Web Flux results were

If you compare the 99th Percentile, Max and Mean response times of the both results you can see Spring Web Flux is considerably faster without any code changes to improve performance. This is a promising sign. The numbers can be improved even more if we use the Netty Reactive Servers instead of conventional Tomcat servers in my understanding.