Friday, June 13, 2014

Kryonet: simple but super-fast TCP communication in Java + performance benchmarks

Martin Podval
We have designed cluster-aware and cloud-based solution of service virtualization for some time. Incredible work as distributed system is next dimension of thinking - and learning as well :-)

We was deciding to split atomic piece of our domain work to multiple nodes. Such step would hopefully provide us to point certain messages to certain nodes. We can than store data in main memory and avoid distributed locking above some remote store, probably Redis. This is usual approach how to achieve better response time of the whole solution to avoid database search for all calls. Obviously with well designed replication.

I was really worried about latency. We need really low response time in couple of milliseconds at most under heavy load. I'm talking about the whole solution which consumes CPU a lot because of the calculations of simulated response under the hood, so no space for any wasting.

What is the best latency of such transmission between two nodes? What is the throughput of one or multiple connections? Except theoretical analysis I also did our own proof of concept to verify our thoughts in the reality.

I like kryo serialization framework. It's very fast, very simple and the boot of anybody is in level of minutes. Hopefully I discovered that guys from Esoteric Software think about remote communication and they did very logic step. They developed framework allowing TCP communication using kryo serialization - kryonet.

Well, I believe that nobody is surprised that the solution is very fast and super easy as well. Like kryo itself.

There are some benchmarks how TCP communication can be fast, in level o microseconds, but what is the real response time, performance, when I need to transfer some domain object to another node in real environment using gigabit network. I mean no laboratory environment but just real one where our app will probably run in couple of months?

Well, in short it's hundred of microseconds.

Kryonet via Kryo

It is built above java nio, kryo serialization and asynchronous approach when the response is being received.
It's better to read the manual on github as coding around the sending and the receiving is very easy.

Kryo preserves the way that the serializators are separated classes, later registered to the system, so you do no have to affect your domain classes. But obviously, you have to open them to support injections of fields from outside. Or you can use the reflection too.

The response is received asynchronously to the processing of the original request. Kryonet exposes TCP port on the server side and uses TCP under the hood. But don't worry, you almost can't touch complexity of TCP messaging.

Number meaning and setup

What was my message? It contains identifier, uuid, and map of elements. The element contains the data but the important thing is the overall size of the message.

Response time is the whole round-trip:
  1. Serialization on the client side
  2. To send the request to the server
  3. Deserialization of the message on server side
  4. To respond the message identifier back to the client
  5. The deserialization of received message on the client size
Well, the response time is just the number you can expect when you want to push some message to remote component and wait for the acknowledgment - ACK.

Both client and server are HP Proliant, 24 and 32 cores, >1 gigabit network.

Performance Benchmarks

One TCP Connection

First of all, I used one TCP connection.

Message Size [bytes]/Number of elementsAverage RT [micro-seconds]Median RT [micro-seconds]Buffer sizeThroughput [msgs/s]Rough Load [msgs/s]
48 / 2190160256 B5400100k
48 / 220217016 kB5400100k
48 / 26721571kB183001M
48 / 241813016kB193004M
3216 / 20031631316 kB6501k
3216 / 200332331256 B-1k

Message size: size of serialized message into raw byte[].

Average vs. median: as my test keeps all response time value, it was very interesting to analyze them. Average value reflects all edge cases, mostly the divergence during the flushing of buffers etc., but median value does not. What does it mean in real? The difference between average and median value indicates growing number of slower responses.

Buffer: kryo uses serialization and object buffer, see the documentation.

Rough load: all calls are asynchronous which means that I had to limit incoming load generated by my generator. The approach I chose was to call standard java thread sleep after certain messages. The operation in milliseconds is not quite accurate but it was fine for rough limitation of input load. 4M means that the generator thread sleeps for 1 milliseconds once it generates four thousand of messages.

The discussion:
  • there is only two times slower response time for 100 times larger message
  • there is almost no different response time for various buffer sizes
  • increasing load means almost the same median value but highly increased average response time. Well, the response time is not stable at all which indicates that you can't rely on response time in the case of massive load

Multiple Client Connections to One TCP Server Port

The approach having multiple clients using one TCP port usually scales well, but not for kryonet as it uses limited buffer size, see following table. I used 10 concurrent clients for the measure.

Message Size [bytes]/Number of elementsAverage RT [micro-seconds]Median RT [micro-seconds]Buffer size (client/server)Throughput [msgs/s]Rough Load [msgs/s]
48 / 225625416k/16k90001k
48 / 258053516k/160k420005k
48 / 2330002300016k/300k8000010k

The discussion:
  • kryo itself is very fast but single threaded deserialization brings certains limits so huge throughput was achieved for the cost of really high latency
  • the increasing of the buffer size within the server allows to serve much more requests as they just fall into buffer and were not rejected as before for lower buffer size

Multiple Connections to Multiple Ports

The last includes multiple clients connected to the multiple server ports. It brought the best performance as expected.

Message Size [bytes]/Number of elementsConnectionsAverage RT [micro-seconds]Median RT [micro-seconds]Buffer size (client/server)Throughput [msgs/s]Rough Load [msgs/s]
48 / 210542486256/25654000100k
48 / 210500051816k/16k155000500k
48 / 2202008142316k/32k210000500k
48 / 250146040416k/32k182000100k
48 / 2502024378256k/256k220000100m
48 / 21003120404256k/256k231000100m
3216 / 200104053688k/32k210003k
3216 / 2001041535316k/32k210003k
3216 / 2002039935016k/32k323002k
3216 / 2005038032716k/32k437001k
3216 / 2005041034464k/128k780002k

The discussion:

  • incredible load for 100/50 clients per second in no bad time as I would originally expected
  • as I've already discuss the stuff above, there is huge different between median and average RT as well
  • there is no such difference between 20 and 100 clients in throughtput but in average/median RT

Resources

Well, usually the transport of a message from one to another node is not your business case. It's important that both client and server is able to do it's core logic - message processing of received throughput is always the core.

One client does not almost burden the server, see htop screenshot for the most approach having the best throughput:


On the other hand, 10 concurrent clients to one server side port is almost the same even if the number of transported messages is 5x time higher:


As you can see, 50 concurrent clients and ports is more likely theoretical stuff for this measure than usable solution as the server do not any other thing then deserialize incoming throughtput :-)


Conclusion

It's fast and easy. Almost 20k transmitted small messages for one client per second is incredible number for me. On the other hand, 150-200 micro seconds response time is surprise for me as well.

The only thing to consider is to carefully define the size of all buffers. There is no queuing and just the rejection of a message can happen if the connection is under heavy load.

2 komentářů:

About me

Powered by Blogger.