Technically Speaking

The Official Bigstep Blog

 

Kubernetes Benchmark: Bigstep vs AWS [part 1]

This two-part study compares the performance aspects of running a Kubernetes environment on two different cloud providers: Bigstep Metal Cloud versus Amazon Web Services. In this first part we discuss the key differences that were identified measuring network latency, throughput and disk throughput. Besides network, compute and storage performance we also cover aspects related to the complexity of setting up the environment.

Setup and Methodology

The purpose of this benchmark is to compare Kubernetes performance on two different cloud providers, one offering a bare metal IaaS while the other a virtualized IaaS. We will not get into details on micro-services architectures, benefits of container-based application and virtualization impact on performance, but we recommend these resources for research.

The entire benchmark and tests should be easily reproducible on any other cloud provider or on premise, and for this purpose, we uploaded all the materials needed for this experiment

Hardware specification:
 
Model
CPU
Memory
Storage
Network Performance
FMCI 20.64
20*
64 GB ECC
Bigstep BSA
4 x 10 gbps
m4.4xlarge
16 (vCPU)
64 GB
EBS-only**
high

*hyperthreaded
**Dedicated EBS Bandwidth: 2000(Mbps)

For Bigstep we used the UK data centre and the CPU is Intel(R) Xeon(R) CPU E5-2630 v4 (@2.20GHz) while for AWS we used the Ohio region and the CPU used is 2.4 GHz Intel Xeon E5-2676 v3 (@ 2.40GHz). Despite the fact that it’s not the same type of processor and we have 4 additional threads in Bigstep, the CPU frequency is higher in AWS instances. Even though the instances are not identical, the purpose is not to compare processor generation but rather equivalent instances in terms of resources and price.

Cluster deployment

Both providers offer Kubernetes as a managed service, taking away the burden of managing docker, kubernetes and operating systems. Here we measure the time it takes to create a container platform (we assume the account is already created, and in AWS a VPC and at least 2 subnets are also created):

 

 
Create Cluster
Create Cloud Formation Stack
Create Worker Nodes
Attach Workers
Total time
Container Platform
12 min
-
-
-
12 min
 
Elastic Container Service
11 min
3 min
2 min
2 min
18 min

 

The amount of time and skills required is greater for AWS setup. In Bigstep the setup implies to choose the instance type, set the docker image and hit deploy, all under one minute and the rest of the time is spent booting the actual servers and starting the OS.

Network Latency

Network latency is the most important aspect in a high-performance container environment. Without the hypervisor’s overhead, the bare metal environment should have better performance compared to virtual environments, the question is by how much.

We used the open source netperf utility for testing latency. It is maintained by the Kubernetes community and easy to deploy on a cluster.

It uses kubectl to start the containers on both AWS communication and Bigstep Bare Metal nodes of the cluster. 

This is a sample of the results:

  Container Platform

  Elastic Container Service

As we can see in the above charts there is significantly lower network latency (almost x6) and 5 times more network throughput on Bigstep bare metal container platform. Most applications depend on network throughput and latency to offer a good user experience: web apps, streaming platforms or distributed applications all depend on the nature of inter-container communication. From this perspective, one should really take into consideration running containers on bare metal servers as it offers up to 5 times more network performance.

Filesystem Write Performance

Another important aspect in a high-performance container platform is the filesystem read/write capacity. One simplistic approach to testing filesystem performance is using dd to write a bulk of zeros in chunks of 1M or more. We used simple-container-benchmarks container for the test and monitor it using dstat.

This is a sample of the results (2 runs for 1G and 2G):

  Container Platform

# dd bs=1M count=1024 if=/dev/zero of=~/simple-container-benchmarks-writetest conv=fdatasync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 5.38783 s, 199 MB/

 

# dd bs=2M count=2048 if=/dev/zero of=~/simple-container-benchmarks-writetest conv=fdatasync

2048+0 records in

2048+0 records out

2147483648 bytes (2.1 GB) copied, 14.0173 s, 153 MB/s

 

  Elastic Container Service

# dd bs=1M count=1024 if=/dev/zero of=~/simple-container-benchmarks-writetest conv=fdatasync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 7.33642 s, 146 MB/s

 

# dd bs=2M count=2048 if=/dev/zero of=~/simple-container-benchmarks-writetest conv=fdatasync

2048+0 records in

2048+0 records out

4294967296 bytes (4.3 GB) copied, 32.3296 s, 133 MB/s

 

As we can observe, the performance of containers running on bare metal is between 15% and 35% higher when we measure throughput in MB per second and twice as fast when measuring the time to write a 2GB file to disk.

Another relevant test for most applications is Sysbench, a multi-threaded benchmark tool used for database workloads.

Closing Note

The scope of these benchmarks remains to create an objective opinion on running containers on bare metal servers or virtualized servers with a focus on performance and cost. The scripts, containers and detailed explanations are available on Github and it should be easy to reproduce the tests on other environments.

As we gathered from the tests, running containers on bare metal servers offers up to 5 times more network performance; when we measure throughput in MB per second, the performance of containers running on bare metal is also between 15% and 35% higher.

In the second part we will cover sysbench tests on mysql database and complex workload that involve database and applications communication as well as provide a cost/month analysis.

Got a question? Need advice? We're just one click away.
Sharing is caring:TwitterFacebookGoogle+PinterestEmail

Readers also enjoyed:

IaaS Versus PaaS Versus SaaS: What are the Differences?

If you have contracted for cloud computing services, your service falls under one of these categories: Software as a Service, Platform as a Service, or…

Webinar: AWS DocumentDB vs. MongoDB on Bigstep Metal Cloud

At the beginning of January, Amazon Web Services announces the launch of a new database in their offering: DocumentDB. If you type Mongo in AWS search,…

Why Does Bare Metal + Cloud = Better?

Companies cope with exponentially more data than they had just a few years ago. The sheer number of devices that create and transmit data has proliferated…

Leave a Reply

Your email address will not be published.

* Required fields to post your comments.
Please review our Privacy Notice in order to understand how we process your personal data and what are your rights in this respect.