Skip to main content

Redis messaging using Spring and Kryo serialization

Redis is famous enterprise key/value data store which provides simple messaging using publisher/subscriber. I've decided to use the system to notify remote nodes (components) about a change of state between those components.

Spring among the others allows to use it's templating system to decouple business logic from messaging system used under the hood. The use of spring data for Redis guarantees a solution which does not utilize any redis command in the code.

Redis contains serialized content, either byte[] or string. So the last thing to reveal is domain model serialization. I've decided to fast binary serialization using Kryo framework as a winner of battle of serializators.

Maven

First of all, it's necessary to define all dependant components. Obviously, usual component like spring itself missing.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
<dependency>
 <groupId>com.esotericsoftware.kryo</groupId>
 <artifactId>kryo</artifactId>
 <version>2.22</version>
</dependency>
<dependency>
 <groupId>org.springframework.data</groupId>
 <artifactId>spring-data-redis</artifactId>
 <version>1.1.0.RELEASE</version>
</dependency>
<dependency>
 <groupId>redis.clients</groupId>
 <artifactId>jedis</artifactId>
 <version>2.1.0</version>
</dependency>

Kryo Serializator

I used domain model's entity which hold identifier of our internal service, this is just UUID. The point is to setup kryo serializator which get the entity and returns byte[]. The entity itself is not important.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
public class MyEntitySerializer extends Serializer<MyEntity> {

    @Override
    public void write(Kryo kryo, Output output, MyEntity object) {
  return ...;
    }

    @Override
    public MyEntity read(Kryo kryo, Input input, Class<MyEntity> type) {
        return ...;
    }
}

Redis Message Handler

The beautiful part is the complete definition of async message handling within xml configuration of redis message container.

Note that I used Jedis client as java redis client.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
  p:host-name="${redisHost}" p:use-pool="true"/>

<bean id="myEntitySerializer" class="MyEntitySerializer" />

<bean id="redisContainer" class="org.springframework.data.redis.listener.RedisMessageListenerContainer">
  <property name="connectionFactory" ref="jedisConnectionFactory"/>
  <property name="messageListeners">
    <!-- map of listeners and their associated topics (channels or/and patterns) -->
    <map>
      <entry>
        <key>
          <bean id="messageListener"
              class="org.springframework.data.redis.listener.adapter.MessageListenerAdapter">
            <constructor-arg>
              <bean class="MyEntityListener"
                  autowire="constructor"/>
            </constructor-arg>
            <property name="serializer" ref="myEntitySerializer"/>
          </bean>
        </key>
        <bean class="org.springframework.data.redis.listener.ChannelTopic">
          <constructor-arg value="${channelName}"/>
        </bean>
      </entry>
    </map>
  </property>
</bean>

What is going on?

Message container references both redis factory and given list of message listeners. It's a class (or set of classes) having registered method as a callback when a new message arrives.

The last property is channel with important channel name. The code uses two variables, first is redis host and already mentioned channel name.

Message Listener

The last thing to do is to define message listener containing callback method, MyEntityListener. The class instance is called always once new message arrive using channel topic.

Crucial point was to discover the signature of callback's method because spring's documentation is little bit sloopy. Quick look into org.springframework.data.redis.listener.adapter.MessageListenerAdapter's onMessage shows the correct way.

1
2
3
4
5
6
public class MyEntityListener {

    public void handleMessage(MyEntity entity, String channel) {
      // provide handling code
    }
}

Incoming message is automatically deserialized so the method accepts entity itself.

Conclusion

Look at previous code. Every redis related code is defined in spring's context and container class. No boilerplate code. Pretty nice.

Architectural note should explicitly show that the entity serves as a notification only. This is very important as such messaging is not reliable. Although the entity holds some information all the persistent data is defined within another place, e.g. as key/value pairs. Approaching message just notifies subscriber that new content is available to refresh and it's supposed to GET key(s).

Comments

Popular posts from this blog

Performance Battle of NoSQL blob storages #1: Cassandra

Preface We spend last five years on HP Service Virtualization using MsSQL database . Non-clustered server. Our app utilizes this system for all kinds of persistence. No polyglot so far. As we tuned the performance of the response time - we started at 700ms/call and we achieved couple milliseconds per call at the end when DB involved - we had to learn a lot of stuff. Transactions, lock escalation , isolation levels , clustered and non clustered indexes, buffered reading, index structure and it's persistence, GUID ids in clustered indexes , bulk importing , omit slow joins, sparse indexes, and so on. We also rewrite part of NHibernate to support multiple tables for one entity type which allows use scaling up without lock escalation. It was good time. The end also showed us that famous Oracle has half of our favorite features once we decided to support this database. Well, as I'm thinking about all issues which we encountered during the development, unpredictive behavio

NHibernate performance issues #3: slow inserts (stateless session)

The whole series of NHibernate performance issues isn't about simple use-cases. If you develop small app, such as simple website, you don't need to care about performance. But if you design and develop huge application and once you have decided to use NHibernate you'll solve various sort of issue. For today the use-case is obvious: how to insert many entities into the database as fast as possible? Why I'm taking about previous stuff? The are a lot of articles how the original NHibernate's purpose isn't to support batch operations , like inserts. Once you have decided to NHibernate, you have to solve this issue. Slow insertion The basic way how to insert mapped entity into database is: SessionFactory.GetCurrentSession().Save(object); But what happen when I try to insert many entities? Lets say, I want to persist 1000 libraries each library has 100 books = 100k of books each book has 5 rentals - there are 500k of rentals  It's really slow! The inser

Jenkins + git revision in all build names

Jenkins by default assigns version of a build using local counter within each type of a build. An example is better. When you look at this overview, you definitely do not know which code revision was used in Compile build and which in Integration Tests . I've followed nice article regarding real CI pipeline using jenkin s. It uses Build Name Setter Plugin. Unfortunately this article uses SVN revision number. So I said I'll just use git revision as git is my source control. But it's not so easy as how it could seem for first look. My Jenkins setup comprised of first compile build step which clones git server and performs an compilation. Second build steps clones the repository from first step and executes integration tests . The problem here is that the second step does not know which git revision compile step cloned. Here is list of steps how to do that. 1. You obviously need Git Plugin , Build Name Setter Plugin and Parameterized Trigger Plugin 2. Compile