Hosting services have gotten a lot better in recent times, more storage, more bandwidth, cheaper prices. But there is only so far that regular hosting will go. When I first launched PSDTUTS, despite being on a great webhost with some big specs, we ran into both bandwidth and file serving problems when our tutorials (each containing dozens of heavy images) would land on Digg. And then a friend recommended I try using Amazon's Simple Storage Service (aka S3).
It turned out to be really easy to use S3 like a giant, unlimited host, and I've been using it ever since for high-access, or heavy files like podcasts for FreelanceSwitch, images for blogs, and even for sending files to clients. In this tutorial I'll show you how with just Firefox you can turn S3 into your own personal content delivery network. It's super simple, even I could do it!
Step 1
The first thing you are going to need is an account. So do the following:
- Log in to your regular Amazon.com account (or if you don't have one, create one!)
- Go to Amazon.com/AWS (AWS = Amazon Web Services)
- Using your Amazon account, Create an AWS account by following the prompts
- When you've created your AWS account (shown below) you'll see a list of web services that you might be interested in. Choose "Amazon Simple Storage Service"
Step 2
Now before you can do anything with S3 you have to Sign Up For This Web Service. (I know there's a lot of signing up to do!)
You'll need to provide a credit card so that Amazon can bill you for bandwidth, storage and processing. The rates are low, something like $.15 p/gb transferred.
Although you might be thinking that your webhost provides bandwith at a much cheaper rate, this probably isn't really the case. For example a company like Dreamhost will offer you literally terrabytes of bandwidth for just a few dollars a month. However if you try to hit their servers quickly to make use of this bandwidth (e.g. during a Digg, or when you just sent a newsletter out about a download file), you're unlikely to get either a quick download, or even a download at all. In my experience webhosts count on the fact that there are a lot of factors that ensure it's highly unlikely you'll ever use that bandwidth.
In the time I've used S3 - about 8 months now - I've hit it with some pretty large spikes of traffic. The biggest was a day when we had 150,000 visitors in a single day on a tutorial with a total of 3mb of images. So that day alone it served up 450gb of data, and didn't even bat an eyelash. I'm sure in the scheme of things there are much bigger users too.
So it's pretty safe, and it's pretty cheap. Meanwhiles Amazon should really be paying me money for all this promotion :-)
Step 3
So anyway once you've signed up, click on the "Your Web Services Account" button and a little menu appears. This is where you control your account, check on your activity (ie. bill) and so on.
Click on "AWS Access Identifiers"
Step 4
Now you'll get two numbers, your access key ID and your secret access key. Needless to say you want to keep these very safe!
Copy the two numbers out into a text editor so we have them on hand, because we'll need them in a second.
Step 5
Next we're going to need our Firefox Extension. If you're not using Firefox ... well you're on your own :-) I know for example you can use Transmit on a Mac to log in to your S3 account. Presumably on Windows there is something similar (if anyone knows ... please do leave a comment).
So we're going to use the S3Fox extension from RJonna
Visit the website and click on the link shown in the screenshot to download S3Fox.
Step 6
Once installed, your Firefox will restart and then you'll see that you can now access S3Fox either from the bottom right of your browser window or from the Tools menu.
So go to Tools > S3 Organiser and you should have a tab appear showing your S3Fox window. It's really similar to an FTP program and hence super easy to use. There are a couple of differences though.
So click on Manage Accounts and create an account with your Access Key and Secret Key from earlier. Click Save.
Step 7
Once you've created an account you'll get a blank area on the right hand side. The first thing you need to do is create a Bucket. A Bucket is Amazon terminology for what amounts to a folder or directory of files. Bucket names have to be unique across the service, so for example you won't be able to create a bucket called "Test" because I'm sure that was like the first bucket name to go.
Anyhow right click on the right hand side and choose "Create Directory", this will in fact create a Bucket because we're in the root. Once you have a bucket created you can create directories inside. You can also have as many buckets as you wish.
You'll be prompted for a name and the extension will show you a bunch of rules about bucket names. Weirdly the extension seems to interchange the words bucket, folder and directory.
Step 8
So I created a bucket named "nettuts" and then added two directories inside called "003_S3" and "s3demo". Then I have dragged an image over to upload to S3. The image is called s3image.jpg.
Now Right-Click on the file and choose Edit ACL. Before you can access your file you're going to need to change its permissions, and that's what this does.
You'll be prompted to give Read/Write/Full Control by ticking or unticking some selections. You only want to give Read access here, so tick the boxes.
(Note on a Mac the extension has a weird bug that when you click on the 'x' it doesn't immediately change to a tick, but when you click Save, all of a sudden it does. It's confusing and hopefully will get fixed, but you get used to it)
Step 9
Now once you have set permissions, Right Click again and choose Copy URL to Clipboard. The URL For my image is:
http://nettuts.s3.amazonaws.com/003_S3/s3demo/s3image.jpg
As you can see the first part of the URL is the bucket name, then the domain, then the folders and finally the file name.
That's It!
And that's it, here's the image that I placed on S3. Right click and view its location to see! (Actually all the images in this tutorial are on S3 :-)
Of course if you want to do all this dynamically with PHP or Ruby, that's a whole other kettle of fish, and hopefully one we'll cover here in future. You can also get Wordpress and other app plugins that connect up to S3.
Related Posts
Check out some more great tutorials and articles that you might like
Plus Members
Source Files, Bonus Tutorials and
More for $9 a month for all TUTS+
sites in one subscription.
User Comments
( ADD YOURS )Sumesh May 7th
Nice tutorial, Collis.
I’d first heard about S3 from Paul Stamatiou, who was using it to serve all the images(sprites) for his lightweight design to make it even better. He’d said that spreading files across fast hosts would help optimization.
I’m glad you covered this in detail – very useful, and I’ll be using S3 for my theme image backgrounds too. But you could’ve made the images a bit more compressed – it takes some time to load. Ok, its not a big deal because I could open up something else, but hey…..
( )pexe May 7th
Fun!
Very nice to know that sommething like that exists
Thanks for Sharing!!!!
( )Chris May 7th
A good FTP client for Windows is the freeware FileZilla. Not as good as Transmit, but a viable alternative for those stuck on Windows.
( )Erik Reagan May 7th
Just what I was needing considering that I’m looking into S3 for this exact purpose. Thanks for the excellent walk-thru! I’m looking forward to using Panic’s Transmit (my favorite FTP client by far) for the S3 service.
( )Robert May 7th
This site is quickly becoming one of my top favs. Great article!
( )mamjed May 7th
so do you only host your big files with amazon or the whole site?
( )Jeff May 7th
Collis, I actually started looking into S3 a while back after noticing that you were using it on PSDTUTS, so it’s great to see a post about it made it onto NETTUTS.
@Chris Filezilla is a great suggestion for a windows FTP client, but sadly it currently doesn’t support S3 and according to this post probably never will.
( )Danny May 7th
Wow, thanks for the S3 intro. I’ve been using 1&1 and have heard good and bad things about it, mostly bad. I’ll definitely look into this as my next hosting choice if anything were to go wrong
Is there anything bad at all about S3?
( )Erik Hansen May 7th
Great tutorial Collis!
I’m in the middle of working on a website project were I’m expecting very heavy traffic, with each visitor consuming around 30-60MB per visit. I’ve been planning on using Amazon S3 to host the videos – your tutorial made my job even easier.
If you develop some sort of “Premium Membership”, I’ll definitely be interested.
@Danny – In regards to 1and1’s service. I’ve been using them for the past 2 years. I have their basic $4mo package have used it to host over 10 low-traffic sites. I’ve not experienced any problems with them. I have heard that their customer support is quite lacking due to the fact that they’re based in India. For small websites with out lot’s of traffic, you should be able to get away with 1and1. If you’re planning for larger bursts of traffic, then you might look into Media Temples Grid Service, or if you don’t mind the extra work, then use Amazon S3.
( )Ignician Website Design May 7th
I always wondered why I saw so many links to amazon loading on your pages. Dealing with high traffic sites seems to be something that people only learn the tricks when they fall in the pits. Thanks for sharing your tips.
( )Shane May 8th
Nice intro to S3 – I’ve seen it around, but hadn’t looked into it. I need look no further
Thanks.
( )Helmy May 8th
Thanks for sharing the tips,
( )but psdtuts.com cannot be access today. Don’t know why ?.
Phillip May 8th
Thanks Collis!
( )I’ve been wanting to find out how S3 works.
Ben Griffiths May 8th
S3 is a fantastic service, and it bewilders me why more people don’t take advantage of it, thanks for the guide!
( )D. Carreira May 8th
Thanks, another usefull tutorial!
David Carreira
( )Joefrey Mahusay May 8th
Thanks for this amazing information. Amazon is really great.
( )Medium May 8th
Don’t get me wrong, i apreciate all your effort with theese tutorials – but i would really like to see some usefull web design stuff. If it’s of any use here are some basic ideas. Maybe a review on ajax – i’m mostly interested in URL consequence and for now the only option is a “fake” url like #something ?
That’s just at the moment what came to my mind, the point is please write something for development
( )Adam May 8th
Nice tutorial!….away from that…when is the new design for FaveUp going to be released…you said a week a week ago..I’m eager to see man!
( )qbrushes May 8th
after i saw how psdtuts was running on S3 a while back i had a look into it my self and now i successfully use it with a new site i run qbrushes.com & qvectors.com
for people with wordpress there is a great plugin that allows you to directly upload from within the posting page which can save some time. here is the plugin site. http://tantannoodles.com/toolkit/wordpress-s3/
also a problem i ran into with trying the Firefox plugin is that none work with FF3 beta except this add-on https://addons.mozilla.org/en-US/firefox/addon/6955 . but its very lacking in features, so i wouldn’t recommend it.
a nice standalone program is S3safe.com http://www.techpowerup.com/downloads/951/S3Safe_1.0.html its shareware but i’ve been using it for a while now and it doesn’t expire.
p.s! sorry for stuff up
please delete the 2 prevoius posts.. didn’t mean to spam like that.
( )Ben May 8th
A great ftp client that i actually cant live without is ‘FireFTP’ – its just a simple add on/plug in for firefox and can be used straight through the browser by just going to ‘tools’. It works fine on my mac, havent tried it on a machine running windows though. I imagine it will work tho. Give it a go its completely free and a life saver to boot!!
O and BTW, love this site and psd tuts.. keep up the great work guys!
( )Ryan May 8th
I’ll second what Chris said above, FileZilla is a nice FTP program for Windows users.
Again, another good, eye opening tutorial. I had read a little bit about AWS, but it’s always nice to see things in use.
I look forward to more like this!
( )Rich May 8th
So do you only use S3 service to host your large files..i.e. images, videos and so on?
( )Adam Griffiths May 8th
Nice tutorial Collis! Thanks!
One question, in step 2 why are you logged in as your wife? I will guess she opened the account or it’s her amazon account…lol. Anyway going off topic, nice eye-opening to S3. I’ve been looking around for different web hosts and this has added another to my list.
Thanks!!
( )Matt May 8th
This is definitely worth considering for high bandwidth cases – the BBC use Amazon’s S3 server for a lot of their static imagery.
( )Christian Mejia May 8th
Oh man! This is great! Thanks for this tutorial. Looks like this may get me a raise at work. ha ha ha. =D
( )Ana Lee May 8th
I love S3. I have been using it for a while. On my redesigned new site using a new CMS, the some of images wouldn’t load every time. Other than that one blip on that one site, all is great. I use it to host my videos too. I just got an email from Amazon saying they are lowering there rates too!
( )Ali May 8th
Very interesting, might need to sign up to this. thanks collis
( )fisch79 May 8th
Its only going to get cheaper!
( )Miles Johnson May 8th
Dont really see a need for this when you could use any other photo uploading service.
Nice to know anyways.
( )Marcus May 8th
Great article. I have a question for the audience at large; Can you use s3 and have a lightboxed image gallery? I am building up a few sites that will have quite a few images and I know spreading things around speeds everything up a bit. Like the look of lightboxing but also like the features of s3, so basically can they be combined? Again fantastic site – when you have a membership area I will be paying.
( )Jacob Gube May 8th
Great overview of the Amazon S3. Six Revisions just started using Amazon S3, and I must say, I love it. It’s a great way to offload static media files and reduce bandwidth consumption.
I also use S3Fox to manage images.
I’d like to add to this tutorial in saying that you can make the links cleaner by setting up a subdomain for your s3 bucket. for example, on Six Revisions, I have it so that images.sixrevisions.com points to my image bucket on Amazon S3. The whole thing takes 5 minutes and is more than worth the time investment.
( )Nico May 8th
Great service! Thank you for the info!
( )narf May 8th
Good walkthrough, I wonder why amazon couldn’t just use plain old ftp protocol? Just so you know amazon is a good service but doesn’t offer all the benefits of a content delivery network. Amazon stores your files on their data center which has huge bandwith but a traditional content delivery network disperes your files globally (or regionally) to make them closer to users. Wikipedia has a good article.
( )Shane May 8th
@Matt – very interesting that you say the BBC uses the service.
Flickr is an alternative hosting choice, but the bandwidth is far less than Amazon’s service provides. Of course, it only hosts images, too, but it’s worth using if you have more modest requirements.
( )fisch79 May 9th
I see your pics are loading from the S3 server.. but i noticed that you didnt create a CNAME Alias like in this tutorial: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1073&ref=featured … just wondering. Thought you might want to have included this in this tutorial.
( )Jacob Gube May 9th
@fisch79 – although that’s definitely a nice feature to implement to make the links to your images cleaner, it’s optional and outside the scope of the tutorial. Plus, creating CNAME records is different depending on your web host/domain registrar.
My two cents.
( )Chris May 2nd
isnt this what you have done though? Just double checking that there isnt anoter better method?
Cheers
( )Rooney May 10th
Hey
Very nice tutorial !
Guys..please clear something up for me. All these hosting services like Amazon S3 and MOSSO Cloud FS are for hosting only images, videos and other files right ? The main site still resides on our own separate web host account like Rackspace in which we put the links in our source code to these files stored on AS3 .. right ?
( )JAM3SoN May 11th
I’m bit stupid of S3 service.. Now I’m reading more about it.. But I would like to ask.. Should I host there e.g. 10 milion files iwth no problems?
( )Qbrushes May 12th
@ Rooney , S3 service it best used for static files like images, files, videos etc, but MOSSO is different. its just like a normal hosting but using the same cloud technology as S3.
( )Tom May 14th
Interesting idea, I’ll consider it for the download of my Wordpress themes.
( )Razvan May 17th
Good stuff, thanks!
( )JR May 22nd
The CNAME part is fantastic.
1. Create a bucket with a DNS name (ie, images.YOURDOMAIN.com)
2. Point that domain to “images.YOURDOMAIN.com.s3.amazonaws.com” in your hosting package as a CNAME.
3. Once the DNS switches over, when you access images.YOURDOMAIN.com, you’ll get the contents of the S3 bucket.
This keeps everything looking clean in the code. More professional and people are never the wiser that you’re using S3.
( )James May 23rd
Great article Collis. I will seriously consider using this for my next project!
( )Jon May 24th
Thanks for the useful article, Collis
I found your site from your guest post on ProBlogger. If you’re open to requests, I’d love to see a tutorial on how you use Amember to run your subscription service on PSDTuts. I’m planning a site that will have a similar kind of concept (though very different market) – free stuff, with extras for people who pay for a subscription
Cheers, Jon
( )Philip Campbell May 26th
LOVING the cname stuff someone posted. I’m using transmit on mac osx (ftp client) and the s3 option is great.
( )Hyun Lee June 5th
Great article. Thanks for the enlightment.
( )rob June 10th
thanks so much! great great article!
( )ive just sign up and start uploading stuff!! thanks!
Mothership June 15th
A very good tutorial. Thank you.
We have a client who’s site is VERY popular, and the forums are constantly under bombardment from the users. One of the issues faced is storage space for file attachments, so will look at this to see if it can work. The CNAME set up may well help here. If anyone has used this with phpBB for similar purposes, we’d love to hear about your experiences.
( )tracyseleana June 17th
Thanks for this info..I’ve already started using it.
But I’m having a problem…when I try embedding video into my blog posts, it doesn’t play the video..Anyone have any suggestions for embedding video located on S3 into a wordpress blog?
Thanks
( )devolute June 22nd
A useful post. Thanks for this, reading stuff on Amazon didn’t help me at all!
( )MD July 12th
Oh thnx! That’s what I was looking for!
( )Tim July 17th
SteadyOffload is another great service to consider. Rather than uploading your static content to a third-party host, it will automatically be mirrored, synchronized and delivered from globally scattered cache servers. Without any complicated setup stuff, it integrates very smoothly.
( )Martin Aranovitch July 27th
This is an excellent tutorial. I just opened up an account with the S3 service to host videos.
I have two questions about Amazon S3:
1) How do we / Can we embed the videos in our web pages? It seems that I can only get the direct link to where the video is hosted, not an embed code for pasting into HTML pages.
2) Can we make the files private/hidden from being found/listed in search engines? I can’t seem to work out how to do this in Amazon S3.
Thanks for any help and great tutorial!
( )Andrei September 3rd
Had a problem with firefox extension, but found this: “S3 Browser”.
( )Windows Themes September 4th
Thax ! Didn’t knew about S3
( )Simon September 15th
I’ve been following this tut closely, however after adding my profile (Key ID and secret access key) I can’t add folders or directories. I get something like an error: “The request signature we calculated does not match the signature you provided. Check your key and signing method.”
I checked the keys 20 times what could be the problem here? Any help is really appreciated.
( )Jim Leeds September 17th
Honestly, this is nuts – just go to smestorage.com and sign up for their OpenS3 programme if you want to use your own S3 keys. Once you have uploaded you files you get a unique url for any file as well as being able to share it access the files from your Iphone or mobile or facebook etc – much easier…….
( )Jesse Andrews September 20th
Another Firefox extension to manage your S3 bucket/files is s3:// – it was just approved by mozilla at https://addons.mozilla.org/en-US/firefox/addon/6955
I’m the creator of it and created it to do what you are describing in your article
( )Andrew Boyd October 17th
well, i successfully followed yr instructions all the way thru to creating permissions et al, but am stumped at the last step — copying URL to clipboard. i’m on a mac, and however many times i try to control-click (the right-click equivalent on a mac) the file, i can’t get a pop up menu. and there seems to be no other way to find or grab the URL.
any offer of help on this front is welcome.
( )state shirt November 6th
Thanks for making this easy. I was having a hard time figuring this out!
( )mary December 24th
I’ve been considering doing something like this with S3 for some of my image heavy websites, so this tutorial is super useful. Thanks!
( )b February 11th
nice one you’ve saved my ass
( )Manila Map February 19th
Thanks! My map site is always going down in the last few days of the month because of my web host’s bandwidth restrictions. Now I don’t have to worry about how many users my site gets.
( )Sandeep Jain May 7th
Hello,
If I set read only permission on my files on S3 then how can I read them. I have some files on S3 and have set the read only permission. Now I need a function or command to read them with php script.
Thanks
( )Andy May 17th
If you are on Windows try CloudBerry Explorer for Amazon S3. It makes managing files in S3 EASY http://cloudberrylab.com/ It supports most of the Amazon S3 and CloudFront features and It is a FREEWARE
( )Lisa June 1st
Hello and thanks for the post. When I select “read only” and then try to embed into a site, nothing happens. When I go back to ACL I notice that my “read” permissions changed back to “x” mark again. Anything I’m doing wrong?
( )keidi July 24th
Great way to serve static files. How can you find it.
( )Simon September 4th
Good article.
Also worth looking at how you can use Google App Engine as a pseudo CDN. It is even cheaper (i.e. free)
( )pierretasse September 15th
Your presentation is great and clear. I worked for one year on S3, EC2 and I know what I am telling. I think beginner will have some idea about how S3 worked and how easy it can be used.
( )But you need to know that it cost something as you pay for bandwith
Xicom Technologies Ltd. November 19th
Thank you for sharing this information … I’ve already started using it.
( )