Amazon service for hosting images
the journal of Michael Werneburg
twenty-seven years and one million words
My photos take up something like three gigs of disk space. Time to clean up.
The two problems are a) the disk space used and b) the amount of our bandwidth allotment that's used in sending images.
My business partner suggested the Amazon AWS/Cloudfront service. Looking into it, it seemed a fairly good deal. You pay for storage and bandwidth, but the rates are very low.
So I tried it.
So far, I've found that it's difficult to use well, but that it front-end performs very well indeed.
challenges
First, it's not easy to set up. You need a tool like S3Hub (beta software for the Mac) or S3Fox (a Firefox plugin) to even transfer files to the thing. I'm not sure why Amazon did things this way, but I suspect it's because there's a lot going on under the hood. No matter, just use the client tools. I explain it below.
But #2 issue: the speed. The upload speeds are terrible, and simply uploading files is not enough. You have to then set the access permissions on the files, and the clients seem to do this by individual transactions. In my case I uploaded 55,000 files, and then had to watch the client upgrade a counter to 55,000 as it set the permissions on the files. One by f'n one.
how to do it
The process that I recommend is this:
1. Sign up for the AWS and Cloudfront services.
2. Get a client tool like S3Hub or S3Fox.
3. Start up the client, using the "key" and "secret key" (great names, guys) that you'll receive in an email from Amazon.
4. Create a "bucket" via the client (don't do this through Amazon's admin interface, buckets created there don't turn up in the client software for some reason).
5. Make a copy of your directory structure with no files in it.
6. Upload your directories.
7. Set the permissions on the directories, and tick the check-box to ensure that each file inherits it's directory's permissions (this is a step I missed)
8. Upload your files
9. Create a "cloudfront" for your AWS bucket
10. Set a "CNAME" from your domain that can mask the weird "cloudfront" name (look this up on the 'net if you don't understand it)
As of now, all of the photos on my site are run on http://i.emuu.net, which is hosted by Amazon.
Three good things that happened today:
1. Sorted out the badly-documented process for hosting images on Amazon
2. Freed up a lot of drive space on our webserver
3. Improved the performance of my humble website