07.28
Had one of those “interesting” emails in my inbox this morning. It was in regards to ElasticDrive, a product that allows you to use Amazon’s S3 service to provide a backing store for a remote file system. Now, remote file systems are nothing new, they’ve been around for decades. What makes ElasticDrive interesting is it makes the remote storage appear as a block device, allowing you to format it with the file system of your choice or even use it in a software RAID configuration.
Now, iSCSI is a protocol that presents remote storage as a block device. It needs to be said that iSCSI was designed to do this. ElasticDrive works over HTTP to connect to a virtually unlimited amount of storage – just add more S3 buckets and you’ve increased your storage. HTTP was *not* designed to do this. ElasticDrive uses 64kb buffers behind the file system. So, if you use the ext3 file system, with its default 4kb buffers, ElasticDrive will not commit the data to S3 until the 64kb buffer is full. So while your system is happily throwing 4kb buffer chucks at it, you’re skating on thin ice.
Now, granted, I haven’t downloaded or used ElasticDrive, but I don’t have a whole lot of confidence in it, especially when Amazon’s S3 goes down for 8 hours. Also, Amazon has no guarantees on the performance of its S3 service. And since all traffic is going over the Internet, which has no performance guarantees, I wouldn’t want to put any real data on it – data critical to my business.
I could be wrong. ElasticDrive may be a great product. It may function excellently and without error. Unfortunately, it isn’t going to be a cheap option. The software itself costs $1 per gig of space. That’s on top of what Amazon is going to charge you. They charge per gig of storage, per gig of data transfer in and out of their S3 network and they charge for the HTTP requests to read and write data. All that can add up real quick.