Use Amazon S3 & Firefox To Serve Static Files
Hosting services have gotten a lot better in recent times, more storage, more bandwidth, cheaper prices. But there is only so far that regular hosting will go. When I first launched PSDTUTS, despite being on a great webhost with some big specs, we ran into both bandwidth and file serving problems when our tutorials (each containing dozens of heavy images) would land on Digg. And then a friend recommended I try using Amazon’s Simple Storage Service (aka S3).
It turned out to be really easy to use S3 like a giant, unlimited host, and I’ve been using it ever since for high-access, or heavy files like podcasts for FreelanceSwitch, images for blogs, and even for sending files to clients. In this tutorial I’ll show you how with just Firefox you can turn S3 into your own personal content delivery network.
It’s super simple, even I could do it!
Step 1
The first thing you are going to need is an account. So do the following:
- Log in to your regular Amazon.com account (or if you don’t have one, create one!)
- Go to Amazon.com/AWS (AWS = Amazon Web Services)
- Using your Amazon account, Create an AWS account by following the prompts
- When you’ve created your AWS account (shown below) you’ll see a list of web services that you might be interested in. Choose "Amazon Simple Storage Service"
Step 2
Now before you can do anything with S3 you have to Sign Up For This Web Service. (I know there’s a lot of signing up to do!)
You’ll need to provide a credit card so that Amazon can bill you for bandwidth, storage and processing. The rates are low, something like $.15 p/gb transferred.
Although you might be thinking that your webhost provides bandwith at a much cheaper rate, this probably isn’t really the case. For example a company like Dreamhost will offer you literally terrabytes of bandwidth for just a few dollars a month. However if you try to hit their servers quickly to make use of this bandwidth (e.g. during a Digg, or when you just sent a newsletter out about a download file), you’re unlikely to get either a quick download, or even a download at all. In my experience webhosts count on the fact that there are a lot of factors that ensure it’s highly unlikely you’ll ever use that bandwidth.
In the time I’ve used S3 – about 8 months now – I’ve hit it with some pretty large spikes of traffic. The biggest was a day when we had 150,000 visitors in a single day on a tutorial with a total of 3mb of images. So that day alone it served up 450gb of data, and didn’t even bat an eyelash. I’m sure in the scheme of things there are much bigger users too.
So it’s pretty safe, and it’s pretty cheap. Meanwhiles Amazon should really be paying me money for all this promotion :-)
Step 3
So anyway once you’ve signed up, click on the "Your Web Services Account" button and a little menu appears. This is where you control your account, check on your activity (ie. bill) and so on.
Click on "AWS Access Identifiers"
Step 4
Now you’ll get two numbers, your access key ID and your secret access key. Needless to say you want to keep these very safe!
Copy the two numbers out into a text editor so we have them on hand, because we’ll need them in a second.
Step 5
Next we’re going to need our Firefox Extension. If you’re not using Firefox … well you’re on your own :-) I know for example you can use Transmit on a Mac to log in to your S3 account. Presumably on Windows there is something similar (if anyone knows … please do leave a comment).
So we’re going to use the S3Fox extension from RJonna
Visit the website and click on the link shown in the screenshot to download S3Fox.
Step 6
Once installed, your Firefox will restart and then you’ll see that you can now access S3Fox either from the bottom right of your browser window or from the Tools menu.
So go to Tools > S3 Organiser and you should have a tab appear showing your S3Fox window. It’s really similar to an FTP program and hence super easy to use. There are a couple of differences though.
So click on Manage Accounts and create an account with your Access Key and Secret Key from earlier. Click Save.
Step 7
Once you’ve created an account you’ll get a blank area on the right hand side. The first thing you need to do is create a Bucket. A Bucket is Amazon terminology for what amounts to a folder or directory of files. Bucket names have to be unique across the service, so for example you won’t be able to create a bucket called "Test" because I’m sure that was like the first bucket name to go.
Anyhow right click on the right hand side and choose "Create Directory", this will in fact create a Bucket because we’re in the root. Once you have a bucket created you can create directories inside. You can also have as many buckets as you wish.
You’ll be prompted for a name and the extension will show you a bunch of rules about bucket names. Weirdly the extension seems to interchange the words bucket, folder and directory.
Step 8
So I created a bucket named "nettuts" and then added two directories inside called "003_S3" and "s3demo". Then I have dragged an image over to upload to S3. The image is called s3image.jpg.
Now Right-Click on the file and choose Edit ACL. Before you can access your file you’re going to need to change its permissions, and that’s what this does.
You’ll be prompted to give Read/Write/Full Control by ticking or unticking some selections. You only want to give Read access here, so tick the boxes.
(Note on a Mac the extension has a weird bug that when you click on the ‘x’ it doesn’t immediately change to a tick, but when you click Save, all of a sudden it does. It’s confusing and hopefully will get fixed, but you get used to it)
Step 9
Now once you have set permissions, Right Click again and choose Copy URL to Clipboard. The URL For my image is:
http://d2o0t5hpnwv4c1.cloudfront.net/003_S3/s3demo/s3image.jpg
As you can see the first part of the URL is the bucket name, then the domain, then the folders and finally the file name.
That’s It!
And that’s it, here’s the image that I placed on S3. Right click and view its location to see! (Actually all the images in this tutorial are on S3 :-)
Of course if you want to do all this dynamically with PHP or Ruby, that’s a whole other kettle of fish, and hopefully one we’ll cover here in future. You can also get WordPress and other app plugins that connect up to S3.
Enjoyed this Post?
Subscribe to our RSS Feed, Follow us on Twitter or simply recommend us to friends and colleagues!
By Collis Ta'eed
Join our newsletter!
Create a Tutorial, Get Paid!
Have something to teach the world? Want to earn money doing it? Tutorials, screencasts, and articles published here on Nettuts+ are largely contributed by readers just like you! We'll pay you great money for good content. Find Out More
Nice tutorial, Collis.
I’d first heard about S3 from Paul Stamatiou, who was using it to serve all the images(sprites) for his lightweight design to make it even better. He’d said that spreading files across fast hosts would help optimization.
I’m glad you covered this in detail – very useful, and I’ll be using S3 for my theme image backgrounds too. But you could’ve made the images a bit more compressed – it takes some time to load. Ok, its not a big deal because I could open up something else, but hey….. ;)
Fun!
Very nice to know that sommething like that exists :)
Thanks for Sharing!!!!
A good FTP client for Windows is the freeware FileZilla. Not as good as Transmit, but a viable alternative for those stuck on Windows.
Just what I was needing considering that I’m looking into S3 for this exact purpose. Thanks for the excellent walk-thru! I’m looking forward to using Panic’s Transmit (my favorite FTP client by far) for the S3 service.
This site is quickly becoming one of my top favs. Great article!
so do you only host your big files with amazon or the whole site?
Collis, I actually started looking into S3 a while back after noticing that you were using it on PSDTUTS, so it’s great to see a post about it made it onto NETTUTS.
@Chris Filezilla is a great suggestion for a windows FTP client, but sadly it currently doesn’t support S3 and according to this post probably never will.
Wow, thanks for the S3 intro. I’ve been using 1&1 and have heard good and bad things about it, mostly bad. I’ll definitely look into this as my next hosting choice if anything were to go wrong :) Is there anything bad at all about S3?
Great tutorial Collis!
I’m in the middle of working on a website project were I’m expecting very heavy traffic, with each visitor consuming around 30-60MB per visit. I’ve been planning on using Amazon S3 to host the videos – your tutorial made my job even easier.
If you develop some sort of “Premium Membership”, I’ll definitely be interested.
@Danny – In regards to 1and1′s service. I’ve been using them for the past 2 years. I have their basic $4mo package have used it to host over 10 low-traffic sites. I’ve not experienced any problems with them. I have heard that their customer support is quite lacking due to the fact that they’re based in India. For small websites with out lot’s of traffic, you should be able to get away with 1and1. If you’re planning for larger bursts of traffic, then you might look into Media Temples Grid Service, or if you don’t mind the extra work, then use Amazon S3.
I always wondered why I saw so many links to amazon loading on your pages. Dealing with high traffic sites seems to be something that people only learn the tricks when they fall in the pits. Thanks for sharing your tips.
Nice intro to S3 – I’ve seen it around, but hadn’t looked into it. I need look no further :)
Thanks.
Thanks for sharing the tips,
but psdtuts.com cannot be access today. Don’t know why ?.
Thanks Collis!
I’ve been wanting to find out how S3 works.
S3 is a fantastic service, and it bewilders me why more people don’t take advantage of it, thanks for the guide!
Thanks, another usefull tutorial!
David Carreira
Thanks for this amazing information. Amazon is really great.
Don’t get me wrong, i apreciate all your effort with theese tutorials – but i would really like to see some usefull web design stuff. If it’s of any use here are some basic ideas. Maybe a review on ajax – i’m mostly interested in URL consequence and for now the only option is a “fake” url like #something ?
That’s just at the moment what came to my mind, the point is please write something for development :)
Nice tutorial!….away from that…when is the new design for FaveUp going to be released…you said a week a week ago..I’m eager to see man! :P
after i saw how psdtuts was running on S3 a while back i had a look into it my self and now i successfully use it with a new site i run qbrushes.com & qvectors.com
for people with wordpress there is a great plugin that allows you to directly upload from within the posting page which can save some time. here is the plugin site. http://tantannoodles.com/toolkit/wordpress-s3/
also a problem i ran into with trying the Firefox plugin is that none work with FF3 beta except this add-on https://addons.mozilla.org/en-US/firefox/addon/6955 . but its very lacking in features, so i wouldn’t recommend it.
a nice standalone program is S3safe.com http://www.techpowerup.com/downloads/951/S3Safe_1.0.html its shareware but i’ve been using it for a while now and it doesn’t expire.
p.s! sorry for stuff up :( please delete the 2 prevoius posts.. didn’t mean to spam like that.
A great ftp client that i actually cant live without is ‘FireFTP’ – its just a simple add on/plug in for firefox and can be used straight through the browser by just going to ‘tools’. It works fine on my mac, havent tried it on a machine running windows though. I imagine it will work tho. Give it a go its completely free and a life saver to boot!!
O and BTW, love this site and psd tuts.. keep up the great work guys!
I’ll second what Chris said above, FileZilla is a nice FTP program for Windows users.
Again, another good, eye opening tutorial. I had read a little bit about AWS, but it’s always nice to see things in use.
I look forward to more like this!
So do you only use S3 service to host your large files..i.e. images, videos and so on?
Nice tutorial Collis! Thanks!
One question, in step 2 why are you logged in as your wife? I will guess she opened the account or it’s her amazon account…lol. Anyway going off topic, nice eye-opening to S3. I’ve been looking around for different web hosts and this has added another to my list.
Thanks!!
This is definitely worth considering for high bandwidth cases – the BBC use Amazon’s S3 server for a lot of their static imagery.
Oh man! This is great! Thanks for this tutorial. Looks like this may get me a raise at work. ha ha ha. =D
I love S3. I have been using it for a while. On my redesigned new site using a new CMS, the some of images wouldn’t load every time. Other than that one blip on that one site, all is great. I use it to host my videos too. I just got an email from Amazon saying they are lowering there rates too!
Very interesting, might need to sign up to this. thanks collis
Its only going to get cheaper! ;)
Dont really see a need for this when you could use any other photo uploading service.
Nice to know anyways.
Great article. I have a question for the audience at large; Can you use s3 and have a lightboxed image gallery? I am building up a few sites that will have quite a few images and I know spreading things around speeds everything up a bit. Like the look of lightboxing but also like the features of s3, so basically can they be combined? Again fantastic site – when you have a membership area I will be paying.
Great overview of the Amazon S3. Six Revisions just started using Amazon S3, and I must say, I love it. It’s a great way to offload static media files and reduce bandwidth consumption.
I also use S3Fox to manage images.
I’d like to add to this tutorial in saying that you can make the links cleaner by setting up a subdomain for your s3 bucket. for example, on Six Revisions, I have it so that images.sixrevisions.com points to my image bucket on Amazon S3. The whole thing takes 5 minutes and is more than worth the time investment.
Great service! Thank you for the info!
Good walkthrough, I wonder why amazon couldn’t just use plain old ftp protocol? Just so you know amazon is a good service but doesn’t offer all the benefits of a content delivery network. Amazon stores your files on their data center which has huge bandwith but a traditional content delivery network disperes your files globally (or regionally) to make them closer to users. Wikipedia has a good article.
@Matt – very interesting that you say the BBC uses the service.
Flickr is an alternative hosting choice, but the bandwidth is far less than Amazon’s service provides. Of course, it only hosts images, too, but it’s worth using if you have more modest requirements.
I see your pics are loading from the S3 server.. but i noticed that you didnt create a CNAME Alias like in this tutorial: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1073&ref=featured … just wondering. Thought you might want to have included this in this tutorial.
@fisch79 – although that’s definitely a nice feature to implement to make the links to your images cleaner, it’s optional and outside the scope of the tutorial. Plus, creating CNAME records is different depending on your web host/domain registrar.
My two cents.
isnt this what you have done though? Just double checking that there isnt anoter better method?
Cheers
Hey
Very nice tutorial !
Guys..please clear something up for me. All these hosting services like Amazon S3 and MOSSO Cloud FS are for hosting only images, videos and other files right ? The main site still resides on our own separate web host account like Rackspace in which we put the links in our source code to these files stored on AS3 .. right ?
I’m bit stupid of S3 service.. Now I’m reading more about it.. But I would like to ask.. Should I host there e.g. 10 milion files iwth no problems?
@ Rooney , S3 service it best used for static files like images, files, videos etc, but MOSSO is different. its just like a normal hosting but using the same cloud technology as S3.
Interesting idea, I’ll consider it for the download of my WordPress themes.
Good stuff, thanks!
The CNAME part is fantastic.
1. Create a bucket with a DNS name (ie, images.YOURDOMAIN.com)
2. Point that domain to “images.YOURDOMAIN.com.s3.amazonaws.com” in your hosting package as a CNAME.
3. Once the DNS switches over, when you access images.YOURDOMAIN.com, you’ll get the contents of the S3 bucket.
This keeps everything looking clean in the code. More professional and people are never the wiser that you’re using S3.
Great article Collis. I will seriously consider using this for my next project!
Thanks for the useful article, Collis
I found your site from your guest post on ProBlogger. If you’re open to requests, I’d love to see a tutorial on how you use Amember to run your subscription service on PSDTuts. I’m planning a site that will have a similar kind of concept (though very different market) – free stuff, with extras for people who pay for a subscription
Cheers, Jon
LOVING the cname stuff someone posted. I’m using transmit on mac osx (ftp client) and the s3 option is great.
Great article. Thanks for the enlightment.
:-)
thanks so much! great great article!
ive just sign up and start uploading stuff!! thanks!
A very good tutorial. Thank you.
We have a client who’s site is VERY popular, and the forums are constantly under bombardment from the users. One of the issues faced is storage space for file attachments, so will look at this to see if it can work. The CNAME set up may well help here. If anyone has used this with phpBB for similar purposes, we’d love to hear about your experiences.
Thanks for this info..I’ve already started using it.
But I’m having a problem…when I try embedding video into my blog posts, it doesn’t play the video..Anyone have any suggestions for embedding video located on S3 into a wordpress blog?
Thanks
A useful post. Thanks for this, reading stuff on Amazon didn’t help me at all!