We’ve played with Amazon’s Elastic Compute Cloud service before – principally as a low-cost proof-of-concept to demonstrate Pixenate‘s ability to scale across multiple servers in a clustered environment (Some of our larger customers have several instances of Pixenate running on multiple servers).
Recently I took another look at EC2 – this time for a different reason…
Our Twitter Mosaic service can generate large mosaic images on the fly which can be printed on to T-Shirts, Mugs and Bags.
The service pulls 100’s of user avatar images from twitter and creates a large mosaic from the image collection. This turned out to be a much slower process than anticipated – not because of the image processing but because of how Twitter stores it’s images. Twitter uses Amazon’s S3 cloud storage service – a very low-cost effective storage system. The only problem with S3 right now is that access can be slow. Our service was pulling 100’s of images at a time from S3 – that’s 100’s of HTTP requests – most of which could take anywhere between 0.05 seconds and 5.00 seconds to complete. The result was that our service could take up to 2 whole minutes to create mosaic for use on a product. Clearly not a good shopping experience.
One of the overlooked advantages of EC2 is that if you run software on an EC2 server, S3 is blazing fast. If you’re writing software that does any heavy lifting with S3 data – do it on EC2. It’s like working in zero gravity.
We quickly changed the service so that a key part (the retrieval and composition of S3 images) moved to EC2 and a single HTTP request could be made to the EC2 server. By doing so, we reduced the time it took to create a mosaic from 2 minutes to a couple of seconds. A big win.