Skip to main content

Redis messaging using Spring and Kryo serialization

Redis is famous enterprise key/value data store which provides simple messaging using publisher/subscriber. I've decided to use the system to notify remote nodes (components) about a change of state between those components.

Spring among the others allows to use it's templating system to decouple business logic from messaging system used under the hood. The use of spring data for Redis guarantees a solution which does not utilize any redis command in the code.

Redis contains serialized content, either byte[] or string. So the last thing to reveal is domain model serialization. I've decided to fast binary serialization using Kryo framework as a winner of battle of serializators.

Maven

First of all, it's necessary to define all dependant components. Obviously, usual component like spring itself missing.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
<dependency>
 <groupId>com.esotericsoftware.kryo</groupId>
 <artifactId>kryo</artifactId>
 <version>2.22</version>
</dependency>
<dependency>
 <groupId>org.springframework.data</groupId>
 <artifactId>spring-data-redis</artifactId>
 <version>1.1.0.RELEASE</version>
</dependency>
<dependency>
 <groupId>redis.clients</groupId>
 <artifactId>jedis</artifactId>
 <version>2.1.0</version>
</dependency>

Kryo Serializator

I used domain model's entity which hold identifier of our internal service, this is just UUID. The point is to setup kryo serializator which get the entity and returns byte[]. The entity itself is not important.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
public class MyEntitySerializer extends Serializer<MyEntity> {

    @Override
    public void write(Kryo kryo, Output output, MyEntity object) {
  return ...;
    }

    @Override
    public MyEntity read(Kryo kryo, Input input, Class<MyEntity> type) {
        return ...;
    }
}

Redis Message Handler

The beautiful part is the complete definition of async message handling within xml configuration of redis message container.

Note that I used Jedis client as java redis client.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
  p:host-name="${redisHost}" p:use-pool="true"/>

<bean id="myEntitySerializer" class="MyEntitySerializer" />

<bean id="redisContainer" class="org.springframework.data.redis.listener.RedisMessageListenerContainer">
  <property name="connectionFactory" ref="jedisConnectionFactory"/>
  <property name="messageListeners">
    <!-- map of listeners and their associated topics (channels or/and patterns) -->
    <map>
      <entry>
        <key>
          <bean id="messageListener"
              class="org.springframework.data.redis.listener.adapter.MessageListenerAdapter">
            <constructor-arg>
              <bean class="MyEntityListener"
                  autowire="constructor"/>
            </constructor-arg>
            <property name="serializer" ref="myEntitySerializer"/>
          </bean>
        </key>
        <bean class="org.springframework.data.redis.listener.ChannelTopic">
          <constructor-arg value="${channelName}"/>
        </bean>
      </entry>
    </map>
  </property>
</bean>

What is going on?

Message container references both redis factory and given list of message listeners. It's a class (or set of classes) having registered method as a callback when a new message arrives.

The last property is channel with important channel name. The code uses two variables, first is redis host and already mentioned channel name.

Message Listener

The last thing to do is to define message listener containing callback method, MyEntityListener. The class instance is called always once new message arrive using channel topic.

Crucial point was to discover the signature of callback's method because spring's documentation is little bit sloopy. Quick look into org.springframework.data.redis.listener.adapter.MessageListenerAdapter's onMessage shows the correct way.

1
2
3
4
5
6
public class MyEntityListener {

    public void handleMessage(MyEntity entity, String channel) {
      // provide handling code
    }
}

Incoming message is automatically deserialized so the method accepts entity itself.

Conclusion

Look at previous code. Every redis related code is defined in spring's context and container class. No boilerplate code. Pretty nice.

Architectural note should explicitly show that the entity serves as a notification only. This is very important as such messaging is not reliable. Although the entity holds some information all the persistent data is defined within another place, e.g. as key/value pairs. Approaching message just notifies subscriber that new content is available to refresh and it's supposed to GET key(s).
Post a Comment

Popular posts from this blog

Http and TCP Load Balancing with Kubernetes

Kubernetes allows to manage large clusters which are composed of docker containers. And where is large computation power there is large amount of data throughput. However, there is still a need to spread the data throughput so it can reach and utilize particular docker container(s). This is called load balancingKubernetes supports two basic forms of load balancing. Internal among docker containers themselves and external for clients which call you application from outside environment - just users.


Internal Load Balancing with Kubernetes Usual approach during the modeling of an application in kubernetes is to provide domain models for pods, replications controllers and services. Look at the great documentation if you are not aware of all these principles.

For simplification, pod is a set of docker containers which are always located on the same node. Replication controller allows and guarantees scalability. The last but not least is the service which hides all instances of pods - cre…

Validating nginx config file using docker approach

I try to setup nginx as a load balancer. The configuration is just a file with a lot of constrains so I need a validation. There is no online validation service, as e.g. CoreOS has, and I don't want to install nginx on my laptop as I work on a distributed app.

Docker is right approach for me. Let say I have following config:


In short, I'm going to pass nginx config to running nginx instance and look to the logs.

Put you nginx.config to the temp and start the docker image:

sudo docker run --name nginx -v /tmp/nginx.config:/etc/nginx/nginx.conf:ro -d nginx It uses volume mapping so the command just starts a new docker container and mounts a local /tmp/nginx.config to the given in-container path. You can obviously change the volume path to your personal path. Is it working or not? Look at logs.

sudo docker logs nginx If there is no entry, your file is fine. In the case of an error, you can see something like this:

2016/01/08 11:37:31 [emerg] 1#1: unexpected "}" in /etc/…