Benchmarking Amazon EC2 instance for a Web Server

For the last few days I’ve been testing Amazon web services (AWS). I’m looking for a reliable and scalable hosting provider to host Gloople e-commerce platform. Feature wise AWS is amazing. It offers literally everything an enterprise application might need. In fact, they offer so many services (and various configurations of those services) it might be confusing. I was trying to figure out which setup will be the most cost effective for our usage. It’s quite hard to tell is 4x M1 Small Instance better then 1x M1 Large instance just by reading specification (both setups cost the same). “Better” is of course subjective term and depends on a software although web applications have similar needs.

When you rent a server/instance to server websites you pay for broadband and computing power to process requests. It’s not about how impressive RAM/CPU figures are. The question is how many request per second can be handled and how long does it take.

The benchmark was performed with following software:
– WordPress – tested application.
– ab (apache benchmark) – measuring software
– APC (alternative php cache) – I wanted to benchmark computing power not hard drive I/O

WordPress site had 2 “Lorep Ipsum” posts (changing number of posts will change benchmark figures) with few categories, tags and an image. I benchmark each setup from the same remote host with 50 concurrent connections and performed 500 requests (ab -c 50 -n 500). Tested URL was homepage.

The most striking conclusion from this tests is that 4 small instances gives as much power as 2 medium or 1 large. They also cost the same. Some solutions might benefit from distributed setup so when one instance falls there are 3 more. On the other hand large instance has 4 time more RAM which might be more appropriate for some extensive caching strategy.
It gets little bit more interesting with requests per second divided by cost per hour.

The “High CPU” group visibly stands out. When you need multiple web frontends it looks like the best choice is the “High CPU – medium” instance. It gives the best value for the money. At the moment of writing this article the instance comes with following parameters:

1.7 GiB of memory
5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
350 GB of instance storage
32-bit or 64-bit platform
I/O Performance: Moderate
EBS-Optimized Available: No
API name: c1.medium

It’s worth to mention that “High CPU – High” offers High I/O performance. My WordPress benchmark couldn’t benefit from that because PHP code was cached in memory. I can imagine that for some type of applications it might be an important requirement.

Will Gloople move to AWS ecosystem? I don’t know. There are many other factors which need to be taken under consideration. Nevertheless it was fun to learn little bit more about Amazon and their pricing schema. EC2 instances look very similar and it’s tempting to skip testing and go for a bunch of small servers. This examples shows that sometimes it pays to do homework.