Skip to main content

Using CoreOS stack and Kubernetes #1: Introduction

We were lucky enough in December 2014 to join the group of teams who use CoreOS stack and Kubernetes on their way to become next generation of cloud infrastructure. It has been almost one year so I'd like to provide a article series about our experience with the whole stack.

The Motivation

You usually want to model your business domain, provide useful APIs, break your application into pieces, services, and so on. Well, it's your work.

The distributed computing is one of most challenging disciplines in the computer science. Why is that? Because of an asynchronicity in the form of remote calls among distributed components. There are no locks like in your favorite languages. However, there are remote calls with no guarantees of any response or in any time.

It's pretty challenging to provide high-available application, with no downtime during updates, crashes. The application which scales according to the needs. The application with guarantees any data consistency.

What are typical questions and considerations when you start to build such app?

  • how can I run exactly 3 instances of a service in my app? 
  • how can I detect that some instance failed? 
  • how can I run a new replica instead of dead one? 
  • what if there are more than 3 instances because the dead replica was not so much dead and it's now back in the cluster? 
  • what if there are two replicas - dead one and new one - which process same part of the data? 
  • how can I guarantee that all replicas can see the same configuration? 
  • where can service B discover a link to running service A? 
  • where can service B discover new instance of service A because the first one failed? 
  • how can I install all that mess to one operating system? 
I could write many and many questions like these above. CoreOS and Kubernates allows you to address many of these questions.

CoreOS stack and Kubernates provide well tested but tiny platform for your cluster/cloud infrastructure. You can focus on your business not on the infrastructure.

Components

Here is a diagram how all tools fits together:



  • CoreOS :It's just very simple linux distribution prepared for cluster/cloud deployment.
  • Fleet is responsible for running units (services) on remote nodes
  • Etcd is distributed key value store using Raft consensus. The purpose is to store the configuration in namespaces. I've already wrote some articles about etcd
  • Flannel allows to provide private networking among nodes - or docker container in this case
  • Kubernetes uses all tools together to provide cluster management. You can describe your application via kubernetes descriptors and use kubectl or REST API to run, scale or fail-over your app. Obviously in cooperation with the application. One can say that it's PaaS for dockerized application. And (s)he would be right.

What should I read to become more familiar with all these?

If you would have only 30 minutes check out this video:



What's next?

I'd like to write article series about our experience with CoreOS and Kubernetes. I'd like to deal with the installation in next article.

Here is a link to the whole series.

Comments

Popular posts from this blog

NHibernate performance issues #3: slow inserts (stateless session)

The whole series of NHibernate performance issues isn't about simple use-cases. If you develop small app, such as simple website, you don't need to care about performance. But if you design and develop huge application and once you have decided to use NHibernate you'll solve various sort of issue. For today the use-case is obvious: how to insert many entities into the database as fast as possible?

Why I'm taking about previous stuff? The are a lot of articles how the original NHibernate's purpose isn't to support batch operations, like inserts. Once you have decided to NHibernate, you have to solve this issue.

Slow insertion
The basic way how to insert mapped entity into database is:
SessionFactory.GetCurrentSession().Save(object);But what happen when I try to insert many entities? Lets say, I want to persist
1000 librarieseach library has 100 books = 100k of bookseach book has 5 rentals - there are 500k of rentals It's really slow! The insertion took exactly

Tomcat 7 remote deployment

I decided to provide automatic deployment of war packaged application using Jenkins and Deployment plugin. The target platform is Amazon with Tomcat 7, see nice set of articles to find out how to setup such environment for free.

Well, there is couple of tutorials but they missing some points so it pushed me to lost one hour of my work.

What I gotFresh installation of Tomcat 7 on remote machine with opened 8080 port on firewallPersonal war file supposed to be deployed
How to push it to tomcat?
1. First of all, there is simple configuration of tomcat users in file tomcat-users.xml - it was my pain in the ass :-) As original comprehensive documentation says, it's necessary to define user, but which one(s)?

Here is working example of tomcat-users.xml:

<tomcat-users>
<user username="manager-gui" password="changeit" roles="manager-gui"/>
<user username="manager-script" password="changeit" roles="manager-script"/>

Git on Windows: MSysGit

I have started to use Git today. I read a lot of discussions that there is no good tool for Windows platform. After forethought I have decided to used TortoiseGit. I also feared of difficult work related with Git as a lot of articles mentioned many instructions. As I already said, I have decided to use TortoiseGit, because I'm used to work with TortoiseSvn, but for start, MSysGit is enought. So this article is about MSysGit, next will be about TortoiseGit.

How to start with MSysgit on local machine?
Download and install Git for WindowsCreate source code directory for your git appRight click the directory at your favorite file browser. Menu should contain item "Git init here". It initializes chosen directory to be git-abled :-)It was your first usage of Git.

Commit data to local Git repository

Now, you can add any file, your first source code, to created directory. If you are prepared to commit any changes to your local git repository, follow next instructions.
Right-click th…