We live in a world where people are increasingly expecting more and faster speeds. In fractions of a second, your website can lose valuable visitors and in turn, money. Although most people think CDNs are for the "big dogs", they're actually super cheap and incredibly easy to use these days.
In this tutorial I'll show you how to setup and use Amazon's Web Services S3 and CloudFront to decrease website load time as well as show the performance differences.
A CDN is a Content Delivery (or Distribution) Network. It is a network of computers with each system placed at different points with the same data on each. When someone accesses the network, they can access the file on the system nearest them or the one with less current load. This results in better lower latency and file download duration. To learn more about CDNs see "Content delivery network" at Wikipedia.
In the example image above, visitors access the server nearest them that will provide the best possible performance. The network of servers would be the CDN. A regular web host would have one central server in which all those visitors would have to access. That one server could be located only in the US or maybe Europe and would result in longer latency and load times for visitors farther away.
Using more than one server, even on just one continent, will make a difference in performance.
Why & The Proof
I've had quite a few people ask me why a CDN is important, even for smaller websites, and why they should bother paying for yet another web service. The simple answer is, the faster–the better. And why not offer your customers (visitors) the best you can?
The smaller the website, the less of an impact a CDN will make. Although, if your visitors translate into money for you then every little bit helps.
In 2006 Google's tests showed that increasing load time by 0.5 seconds resulted in a 20% drop in traffic.
In 2007 Amazon's tests showed that for every 100ms increase in load time, sales would decrease 1%.
This year (2009) Akamai (a CDN leader) revealed in a study that 2 seconds is the new threshold for eCommerce web page response times.
It's cheap. It's easy. And it can translate into more money in terms of customers and saving on your regular web host expenses.
Amazon Web Services (AWS)
Amazon provides a whole slew of fantastic web services. We'll be using Amazon's Simple Storage Service (S3) and CloudFront. S3 is a data storage solution in the cloud which can be tied to CloudFront, Amazon's CDN.
If you're looking for a slightly simpler, all-in-one solution, Rackspace Cloud Files is another great option. They've partnered with Limelight Network's CDN which has slightly better performance than Amazon's CDN currently. However, their service has a few drawbacks you won't find with Amazon. I won't get into all these but one of the bigger ones for me was the lack of custom CNAME support which is supposedly coming at some point in the future. With CNAME support you can setup a custom sub-domain to access your files such as "cdn.yourdomain.com".
Here's Amazon's S3 pricing for the US. For other areas, click the image to see full pricing.
Here's Amazon's CloudFront pricing for the US. For other areas, click the image to see full pricing.
Use Amazon's monthly calculator to get a better idea of your end bill. Last month, my total bill was less than $5, with the majority of that incurred from 20GB+ of data storage. As you can see, it's very, very cheap, especially when you take into consideration the performance and flexibility benefits.
Setup S3 & CloudFront
To get started, we need to sign up for Amazon's S3 and CloudFront services. If you already have an account with Amazon you'll just need to login and finish the signup. If not, you'll need to create an account then proceed to signup for S3 and CloudFront. The signup is simply adding the service to your account. There's nothing complicated involved.
Click each image to go to the service's information and signup page.
Once you've signed up, you'll get an Access Key ID and Secret Access Key which can be found under "Your Account" > "Security Credentials". This is basically your username and password for accessing S3.
To do this, we'll first log into our S3 account using the Access Key ID and Secret Access Key with an application like Transmit (OS X), which is what I'll be using. To see more apps or browser add-ons for accessing S3 see "Amazon S3 Simple Storage Service – Everything You Wanted to Know".
Once signed in, we'll create a bucket to put our files in. I've named mine "files.jremick.com". Buckets must have unique names, need to be between 3 and 63 characters and can contain letters, numbers and dashes (but can't end with a dash).
By unique, they mean unique on the AWS network. So it's a good idea to use something like a URL or something similar.
The files we put in this bucket can now be accessed at "files.jremick.com.s3.amazonaws.com". However, this URL is pretty long and we can quickly setup a shorter one. We'll setup a new CNAME entry at our web host to do this.
Setup Custom S3 Subdomain
To shorten the default URL we'll create a CNAME entry as I've done below (this is at your web host). I've chosen "files" as my subdomain but you could use whatever you like.
Now we can access these bucket files at "files.jremick.com". Much better! Then simply upload the files you want within the "files.jremick.com" bucket.
Once your files are uploaded, you'll want to set the ACL (Access Control List) to allow everyone to read the files (if you want them public). In Transmit you simply right click, select get info, under permissions set "Read" to "World" and click "Apply to enclosed items...". This will give all the files within this bucket read access to the world.
By default, files uploaded to your S3 account will only allow read and write access to the owner. So if you upload new files later on, you'll need to go through these steps again or apply different permissions for just those files.
Create CloudFiles Distribution
Now that we have setup S3, created a shorter URL and uploaded our files, we'll want to make those files accessable through CloudFront to get super low latency to reduce our load times. To do this we need to create a CloudFront distribution.
Log into your AWS account and navigate to your Amazon CloudFront management console (under "Your Account" drop down menu). Then click the "Create Distribution" button.
We'll select the origin bucket (the bucket we created earlier), turn on logging if you would like, specify a CNAME and comments and finally either enable or disable the distribution. You don't have to enter a CNAME or comments but we'll want to setup a shorter URL later like we did for S3. I would like to use "cdn.jremick.com" so that's what I'm setting here.
As you can see, the default URL is pretty ugly. That's not something you're going to want to try to remember. So now let's setup a CNAME for the pretty, short URL.
Setup Custom CloudFiles Subdomain
To setup the custom CloudFiles subdomain, we'll go through the same process as we did for S3.
Now we can access files through CloudFront using "cdn.jremick.com".
How It All Works
When someone accesses a file through your S3 bucket, it acts just like a regular file host. When someone accesses a file through CloudFiles though, it requests the file from your S3 bucket (the origin) and caches it at the CDN server closest to the orignial request for all subsequent requests. It's a little more complicated than that, but that's the general idea.
Think of a CDN as a smart network that is able to determine the fastest possible route for request delivery. Another example would be if the server closest is bogged down with traffic, it may be faster to get the file from a server a little farther away but with less traffic. So CloudFront will deliver the requested file from that location instead.
Caching Problems
Once a file is cached in the CloudFront network servers, it does not get replaced until it expires and is automatically removed (after 24 hours of inactivity by default). This can be a major pain if you're trying to push updates out immediately. To get around this you'll need to version your files. For example, "my-stylesheet.css" could be "my-stylesheet-v1.0.css". Then when you make an update that needs to go out immediately you would change the name to "my-stylesheet-v1.1.css" or something similar.
Performance Testing
Our content is uploaded to our S3 bucket, our CloudFront distribution is deployed and our custom subdomains are setup for easy access. It's time to put it to the test to see what kind of performance benefits we can expect.
I've setup 44 example images ranging in size from approximately 2KB up to 45KB. You might be thinking that this is more images than most websites will load on a single page. That may be true but there are many websites such as portfolios, ecommerce sites, blogs, etc. that load just as many and possibly more images.
Although I'm only using images for this example, what's important is the file size and the quantity for the comparison. Today's websites are loading several javascript, CSS, HTML and image files on every page. 44 file requests is probably fewer than most websites actually make so a CDN could have an even greater impact on your website than we'll see in this comparison.
I'm using Safari's Web Inspector to view performance results, I've disabled caches and shift + refresh 10-15 times (about every 2-3 seconds) for each test to get a decent average of total load time, latency and duration.
45 Total files (including HTML document)
561.13KB Total combined file size
Regular Web Host
Here are the performance results when hosted via my regular web host. Sorted by latency.
1.82-1.95 Seconds total load time
90ms Fastest latency (last test)
161ms Slowest latency (last test)
~65% of the images had a latency of less than 110ms
Sorted by duration.
92ms Fastest duration (last test)
396ms Slowest duration (last test)
Amazon S3
The exact same files were used for testing S3. Sorted by latency.
1.3-1.6 Seconds total load time
55ms Fastest latency (last test)
135ms Slowest latency (last test)
~90% of the images had a latency of less than 100ms
Sorted by duration.
56ms Fastest duration (last test)
279ms Slowest duration (last test)
S3 is faster than my regular web host but only marginally. If you didn't feel like messing around with a CDN, S3 is still a great option to give your website a decent speed boost. I still recommend using a CDN though and we'll see why in this next test.
Amazon CloudFiles
The exact same files were used for testing CloudFront.
750-850ms Total load time
25ms Fastest latency (last test)
112ms Slowest latency (last test)
~85% of the images had a latency of less than 55ms.
Only one file had a latency of more than 100ms.
Sorted by duration.
38ms Fastest duration (last test)
183ms Slowest duration (last test)
Comparison
Here's a quick breakdown of the performance comparison between my regular web host and the same files on Amazon's CloudFront service.
1.82-1.95 seconds vs 0.75-0.85 seconds total load time (~1.1 seconds faster)
90ms vs 25ms fastest latency (65ms faster)
161ms vs 112ms slowest latency (49ms faster)
CloudFront: Only one file with latency greater than 100ms and 85% of the files with latency less than 55ms
Regular Web Host: Only 65% of the files had a latency of less than 110ms
Duration comparison
92ms vs 38ms Fastest duration (54ms faster)
396ms vs 183ms Slowest duration (213ms faster)
50ms or even 100ms doesn't sound like a very long time to wait (0.1 seconds) but when you repeat that for 30, 40, 50 or more files then you can see how it quickly adds up to seconds.
Visual Comparison
Here's a quick video to show just how noticeable the increase in load time is. I've disabled caches and do a forced refresh (shift + refresh) to make sure images aren't being cached.
Other Ways To Increase Performance
There are several other ways to increase website performance when using a CDN.
Create different subdomains for different types of files to maximize parallel downloads. For example, load images from "images.jremick.com" and other files like scripts and CSS from "cdn.jremick.com". This will allow more files to load in parallel reducing the total load time.
One of the options above for increasing performance even more was providing gzipped files. Unfortunately CloudFront isn't able to automatically determine if a visitor can accept gzipped files or not and serve up the correct one. Fortunately, all modern browsers support gzipped files these days.
Create & Upload Your Gzipped Files
To serve gzipped files from CloudFront, we can give our website some logic to serve up the right files or we can set the Content-Encoding and Content-Type on a few specific files to keep things a little simpler. Gzip the files you want and rename them so it doesn't end it .gz. For example, "filename.css.gz" would become "filename.css" or to remind yourself that it is a gzipped file, name it "filename.gz.css". Now upload the gzipped file to the location in your S3 bucket you want (don't forget to set the ACL/Permissions).
We need to set the Content-Encoding and Content-Type (if it isn't already set) on our files so that when requested via browser it knows the content is gzipped and will be able to decompress it properly. Otherwise it will look like this.
We can do this easily with Bucket Explorer. Once you've downloaded it, enter your AWS Access Key and Secret key to log into your S3 account. Find the gzipped file you uploaded earlier, right click and select "Update MetaData".
As you can see, it already has the Content-Type set to text/css so we don't need to set that (javascript would be text/javascript). We just need to add the right Content-Encoding. Click "Add" and in the popup dialoge enter "Content-Encoding" in the Key field and "gzip" in the Value field. Click OK, then Save and you're done! Now the browser will view the file correctly.
Gzipping a file can greatly reduce the file size. For example, this test stylesheet was around 22KB and was reduced to approximately 5KB. For my blog I've combined all my jQuery plugins with jQuery UI Tabs. After minification it was reduced to 26.49KB, after being gzipped it was reduced to 8.17KB.
Conclusion
There are a lot of ways to increase the performance of your website and in my opinion they're worth trying. If visitors are only 0.5 seconds or even 1 second away from leaving your website, a CDN could keep that from happening. Plus, most of us are speed freaks anyway so why not crank up the performance of your website if you can? Especially if it could save you money in the process.
If you have any questions, please let me know in the comments and I'll try to respond to them. Thanks!
Follow us on Twitter, or subscribe to the Nettuts+ RSS Feed for more daily web development tuts and articles.
Related Posts
Check out some more great tutorials and articles that you might like
thanks for this but i don’t like this services i prefer Mediatemple i got a extreme dedicate server and i’m so happy… i was looking for like a 2 moths the service on amazon and i don’t they services.. by the way thank u =]
The Amazon services covered in this tutorial aren’t intended to replace your regular web hosting. They’re services intended to host static files. AWS S3 is the file storage service and is a great service for sharing large files as well. CloudFront is a service intended to help speed up the load time of static assets (like scripts, CSS, images, etc).
You would be quite surprised at how much faster you can get your website to load by using these services in combination with your regular web host (MT).
If you want to manage Amazon S3 and CloudFront on Windows with a freeware desktop tool check out CloudBerry Explorer. It will help you to set up http headers and Gzip. It is a freeware. http://cloudberrylab.com/ bloggers don’t like it but users do!
Thanks for this very intersting article !
If the price wasn’t a variable I would definitly use a CDN. But regular web hosting have fixed prices, meaning no surprises at the end of the month.
But anyway, I always wanted to know how this works and never got time to do it by myself, so thanks
I believe they have a max monthly charge amount that you can set but I can’t quite remember. Either way, if you estimate your bill then it’s not as much of a variable. It’s such a cheap service that it would take a lot of traffic to generate a substantial bill too.
Gonna take a look to see if I find anything about the max monthly charge or budget. Thanks
Ali Koubeissi October 13th
May I ask how are you going to control the traffic (cost)? I was going to use this service before but the price pulled me off. What if someone (a malicious user perhaps?) kept requesting your files hosted on your cloud? Isn’t this going to make the end month bill very high?
Thanks for the explanation – esp. the point in the comments about serving static files. Wonder what the bill would be if you got to the front page of Digg!
I’ve a website with round about 2500 Users every day with round about 4TB Traffic every moth. But the important fact: We’re (even) not commercial! So, our costs are about 200 Euros included a sponsored Server.
We have 3 Servers, one for web, one for gameservers, one for files. Our fileserver has 10TB of free traffic for 70 Euro. There’s no CDN, which can give you 10 (!) TB free Traffic für 70 Euro.
If we switch to a cdn, we have _no_ real benefits, but much more costs.
So, CDN is only for commercial websites useful, not for small ones.
That’s not true, a CDN is useful for anyone who finds its services useful, small or big. 2500 users per day may not be enterprise level but it’s not small either.
The advantage of a CDN is performance for visitors all over the world. If you aren’t concerned about performance and finding something as cheap as possible is your concern then that’s what fits for you, that doesn’t mean a CDN has no use for small websites.
Also, free isn’t free if it costs you 70 Euro. Your file server host may say they’re giving you 10TB of free traffic but if you have to pay in order to use that traffic then it’s not really free.
If you switched to a CDN you would get performance benefits.
yeah, we’re not really small, that’s true. But our main visitors are from germany, so for us its not really useful yet, because we didn’t have enough visitors from other countries. It’s too expensive, i’ve calculated this for us, so that the costs dont compensate performance benefits. For smaller sites, like 200 to 500 Users per day its also not really needed – that’s my point ov view^^
I know – a plenty of hosting offers are a little bit tricky and not even correct. But in this case i can choose between 10TB guaranteed traffic, 200mbit “fair use” trafficflat and a real 100mbit traffic flat, which is much more expensive than the 70 Euro for the 10TB guaranteed trafficlimit. Take a look at http://www.ovh.de (servers are hosted in france). It’s been tested by our users, i’ve never got an email with the request to limit our traffic^^
i really don’t say i’ll never use a cdn – but at this time, it’s too expensive. I’ll have to pay round about 400 up to 700 euro for a cdn based distribution. That’s actually impossible.
Great Article Jarel. Was thinking about using this for a long time now for some of my projects but wasnt really sure how exactly to use the amazon services. This article has helped a lot, thanks =)
The only downfall I see from this is if something like SmartFilter blocks content from the CDN. For example I can not read any tuts sites from work because all the images and demos are blocked.
Great. But are there any other ways around the caching issues in the CDN? I tend to update the CSS files on my sites all the time, and versioning seems like it could get out of hand fast.
This is a really, really good explanation. I think I’ll be using this more often for larger sites and If I forget how to do it, I’ll come back here, thanks!
Amazons Web Services does look very interesting, but I love my host. I don’t necessarily like the idea of paying for the storage, as apposed to playing $200 a year for unlimited space.
it seems to be go daddy. wow, only heard bad things about their hosting. i guess his requirements are not too high.
i’m happy w/ my current host.
Jarel October 14th
When a web host sells you a hosting plan with “unlimited” anything at a fixed price, it’s just a marketing ploy. However, if you’re happy with the price and the service, then that’s good for you. Keep in mind too that S3 and CloudFront aren’t intended to be replacements for your web host, but instead supplements for performance and flexibility.
I have googlede and tried to find the test that the article uses as on of arguments for improving the loaded time. Is there any one that have a link to the original tests ?
Caching is kind of a minor turn off for me…imagine an evolving/dynamic content website that might be updated frequently and we have to make sure we name the files correctly or end up with spending hours on debugging to find out that its the cache that’s the culprit. Why can’t they make it in a way so that when the file gets updated the cache gets refreshed?
Otherwise, it seems to be a great way to improve the sluggish performance threshold provided by the traditional web hosts.
I see what you are saying, but the point of caching is to reduce the load on a server regardless. If you refresh the cache every time you make an update, it partially negates the purpose. Unless its a high-priority fix or update, waiting a short amount of time for the cache to refresh should not be a problem.
There are several other ways around Amazon’s caching and there are lots of other CDN’s available that operate slightly different. You might check out Rackspace. I’ll be writing another tutorial on using their cloud storage and CDN here in a few days for blog.themeforest.net
Because that would beat the purpose of the latency win. In your scenario, where files get cached unless they are updated, the client browser still has to check each time if the timestamp was updated on each file. Thus, this means you’re still making the HTTP request, thus you are still waiting for the connection latency, thus the whole purpose of a CDN is lost.
The author explained the correct usage of caching: using versions. Huge sites like Flickr do it this way too.
“In 2006 Google’s tests showed that increasing load time by 0.5 seconds resulted in a 20% drop in traffic.
In 2007 Amazon’s tests showed that for every 100ms increase in load time, sales would decrease 1%.
This year (2009) Akamai (a CDN leader) revealed in a study that 2 seconds is the new threshold for eCommerce web page response times.”
I really wonder how you can measure this with 100% consistency. There are so many variables that sales depend on, and lot, if not most of them are much more related to the product, price and service than half a second slower download time. For some reason, this ‘milliseconds count’ argument seems more of a justification for services (CDN) that most people simply do not need.
The studies these companies made were to learn more about the way page load time (among other things) effects certain parts of their business. They aren’t trying to convince anyone to use a CDN or any other service.
These huge companies are also very aware of the variables that contribute to sales (and other things) and are able to compensate for those variables so they don’t affect the study on a particular aspect. Of course it won’t be 100% consistent or accurate, nothing can be, but it’s close enough to be considered reliable information.
You have to remember that these companies are the best in the world at what they do. They analyze every aspect of what they do in detail because one tiny change can make a difference of millions of dollars. They’re very good at optimizing their business down to the tiny details, even milliseconds if it will make a difference.
You see, I worked for one big corporation that was top of the line at what they did (non-IT related). And they too analyzed everything, everything had a flow chart, everything was documented etc. The problem with that was, that many times, it did not matter a bit. It was often done just for a job security of the supervisors. Kind of to show how and why they are irreplaceable
Now, I have no doubt that load speed is very important for overall user experience, and thus sales. But if 100ms longer load time means 1% decline in sales, does that work the other way around too – decrease of 100ms will make 1% jump in sales?
Anyway, despite of what you may think, I really liked this tutorial. Because unlike the millisecond parts that I questioned, your comparison showed much more dramatic improvement, definitely worth looking into.
Dennis October 15th
Yes, they are the best in the world at what the do, but are they the best in the world for what you do?
Having worked at Amazon I can tell you there where many times their own site and/or intranets where down or very slow. Now that may only be 1 or 2 percent of the time, but for every service you add to your site you are increasing the percentage of outages by adding a possible week link. I ‘m not saying its a bad idea, I just wonder how far I want to stretch the chain.
Jarel October 14th
Hey everyone, thanks for all the great comments!
I want to point out that a CDN is intended to be a supplement to your current web host, not a complete replacement. Its purpose is to provide better performance to your visitors from around the world, which of course will come at an additional cost. If you’re happy with your current web host performance and you don’t like the idea of paying a little more for increased performance, then this tutorial isn’t for you.
I’ve used the Amazon AWS for awhile as an offsite automated backup for my website…it’s an excellent and cheap solution. I’m roughly spending $1.59/month for nightly backups.
I’ve been contemplating setting up a CDN using the AWS and this is just what I was looking for. Excellent post…and very informational. Thanks a ton!
BTW…how do you get the ‘developer’ menu to show in Safari?
If you have a minute and are watching these comments, I’d love to hear how you automate your nightly site backups with AWS. Love to hear the author’s thoughts on this as well. Thanks!
Great article. I’m currently looking into expanding an European website to the US market and this could become handy.
A little downside for me would be that you have to get another tool to manage the files on S3. Is there a possibility to automate the process? Like an auto-sync between a folder on your web server’s harddisk and the S3 bucket?
At the moment I’m still running a single server setup with lots of optimizations (nginx ftw) which does his job fine, but serving the stuff to the US could become too much. So I’ll definetly take a look at the AWS.
Or maybe a site doesn’t get much repeat traffic because it loads too slow. It’s not much extra work once you know how to do it. As explained above its very simple to setup and not expensive.
I had problems with serving files ending in .css.gz to Safari PC and found that it always wants to download .gz files. The solution was simply to use cssgz as the extension.
Threre is one advantage you maybe forget to explain about using a cdn. You take the load from your server! If your webserver has to serve all your static files, this could highly increase the cpu load of your server – depends on requests and files. So using a cdn, enables your server to handle more requests.
Great article. My company has been serving media files for our digital asset manager using S3 and I’ll be using your guide to finally enable CloudFront.
I’d also like to verify everything Jarel has said about shared hosting. It’s fun while it lasts, but as soon as you start using too many of their resources (bandwidth, storage, cpu cycles), you will get the boot.
Regarding the pricing of AWS, I suggest everyone check out the AWS Calculator and do the math. I’m pretty sure we’re paying less than $20/month for resources we’d never get out of a shared hosting plan.
Great article, but didn’t you mean “compress” in this sentence: “# Gzip files like JavaScript and CSS” because in the end you can Gzip every content type… but only javascript and CSS van be compressed….
We currently use Velocix and it’s way cheaper than other competitors, especially compared to Akamai. Which has a very inefficient network in my opinion.
On the Mac, Win, or Linux, I’d recommend you to use CrossFTP Pro ( http://www.crossftp.com/ ) for the metadata handling, and automatic gzip compression for the S3 contents, which is a good S3 tool as well as FTP tool.
Hi, very nice information, however I have a small question about S3 and cloudfront, if I use both services, that mean I will be charge double??, or when my files get the first time from S3 then I will be charge for S3 and when the files comes from cloudfront then I will get charge just for cloudfront, thanks for the help!!!
Create Christmas Designs With These 50 Sets Of Pho Give your Christmas designs this holiday season a personal touch by creating your own designs. Here are 50 sets of quality Photoshop brushes for your projects.
Web Design Harmony: Concept, Conveyance And Theme Concept is what you want to communicate. Conveyance is how you will communicate it. Theme is how well you coordinate it. This post will help you with some higher level thoughts and ideas behind designing a website.
18 Impressive Web Form Validation Scripts To Enhan Forms are such a common element on the Internet we tend to blunder through them without too much thought. However, if the web site has added a few nice touches that make the process easier, it tends to speed up the process and reduce any frustration
200+ Javascript / JQuery Tutorials For Designing A In this list I have included some great tutorials on jQuery and Javascript. Jquery & Javascripts are used for making the websites for attractive and usable.
10 Javascript & CSS TreeView And Sitemap Plugins A Javascript Tree view and sitemap are good tools to display the structure of your website. I have collected 10 plugins and tutorials that will save your time hunting for one.
Site-Specific Browsers: Turn Sites To Desktop Apps As web designers/developers, desktop applications are harder to create as they have a totally different environment.
We can make use of site-specific browsers which enable us to lock a website inside an application-looking browser.
Over 38 Free Flash Photo Gallery Ready To Use For Flash galleries are getting more and more popular these days. If you’re going to give a professional look to your image gallery, this post is your chance to check out over 38 professional ones.
Free For All: Winter Holidays Edition Whatever winter event you're designing for, you'll want these free fonts, clip art, stock photos, and WordPress and Blogger templates.
User Comments
( ADD YOURS )Cris McLaughlin October 13th
Wow awesome article! Well explained!
( )Santana October 13th
thanks for this but i don’t like this services i prefer Mediatemple i got a extreme dedicate server and i’m so happy… i was looking for like a 2 moths the service on amazon and i don’t they services.. by the way thank u =]
( )Jarel October 13th
The Amazon services covered in this tutorial aren’t intended to replace your regular web hosting. They’re services intended to host static files. AWS S3 is the file storage service and is a great service for sharing large files as well. CloudFront is a service intended to help speed up the load time of static assets (like scripts, CSS, images, etc).
You would be quite surprised at how much faster you can get your website to load by using these services in combination with your regular web host (MT).
( )Crysfel October 13th
thanks you man, really interesting
( )Kevin Kirsche October 13th
very interesting read, thanks!
( )Vincent October 13th
If you want to manage Amazon S3 and CloudFront on Windows with a freeware desktop tool check out CloudBerry Explorer. It will help you to set up http headers and Gzip. It is a freeware. http://cloudberrylab.com/ bloggers don’t like it but users do!
( )Simon October 13th
Thanks for this very intersting article !
( )If the price wasn’t a variable I would definitly use a CDN. But regular web hosting have fixed prices, meaning no surprises at the end of the month.
But anyway, I always wanted to know how this works and never got time to do it by myself, so thanks
Jarel October 13th
I believe they have a max monthly charge amount that you can set but I can’t quite remember. Either way, if you estimate your bill then it’s not as much of a variable. It’s such a cheap service that it would take a lot of traffic to generate a substantial bill too.
( )Simon October 14th
Gonna take a look to see if I find anything about the max monthly charge or budget. Thanks
Ali Koubeissi October 13th
May I ask how are you going to control the traffic (cost)? I was going to use this service before but the price pulled me off. What if someone (a malicious user perhaps?) kept requesting your files hosted on your cloud? Isn’t this going to make the end month bill very high?
Thanks.
( )erkasoft web design October 13th
very interesting article.
( )Jesse October 13th
Thanks for the explanation – esp. the point in the comments about serving static files. Wonder what the bill would be if you got to the front page of Digg!
( )Patrick October 13th
Okay, its interesting – but nothing more.
I’ve a website with round about 2500 Users every day with round about 4TB Traffic every moth. But the important fact: We’re (even) not commercial! So, our costs are about 200 Euros included a sponsored Server.
We have 3 Servers, one for web, one for gameservers, one for files. Our fileserver has 10TB of free traffic for 70 Euro. There’s no CDN, which can give you 10 (!) TB free Traffic für 70 Euro.
If we switch to a cdn, we have _no_ real benefits, but much more costs.
So, CDN is only for commercial websites useful, not for small ones.
( )Jarel October 13th
That’s not true, a CDN is useful for anyone who finds its services useful, small or big. 2500 users per day may not be enterprise level but it’s not small either.
The advantage of a CDN is performance for visitors all over the world. If you aren’t concerned about performance and finding something as cheap as possible is your concern then that’s what fits for you, that doesn’t mean a CDN has no use for small websites.
Also, free isn’t free if it costs you 70 Euro. Your file server host may say they’re giving you 10TB of free traffic but if you have to pay in order to use that traffic then it’s not really free.
If you switched to a CDN you would get performance benefits.
( )Patrick October 15th
Hey
yeah, we’re not really small, that’s true. But our main visitors are from germany, so for us its not really useful yet, because we didn’t have enough visitors from other countries. It’s too expensive, i’ve calculated this for us, so that the costs dont compensate performance benefits. For smaller sites, like 200 to 500 Users per day its also not really needed – that’s my point ov view^^
I know – a plenty of hosting offers are a little bit tricky and not even correct. But in this case i can choose between 10TB guaranteed traffic, 200mbit “fair use” trafficflat and a real 100mbit traffic flat, which is much more expensive than the 70 Euro for the 10TB guaranteed trafficlimit. Take a look at http://www.ovh.de (servers are hosted in france). It’s been tested by our users, i’ve never got an email with the request to limit our traffic^^
i really don’t say i’ll never use a cdn – but at this time, it’s too expensive. I’ll have to pay round about 400 up to 700 euro for a cdn based distribution. That’s actually impossible.
Kriesi October 13th
Great Article Jarel. Was thinking about using this for a long time now for some of my projects but wasnt really sure how exactly to use the amazon services. This article has helped a lot, thanks =)
( )S3 Browser Team October 13th
If you are on Windows you can try S3 Browser – Windows Client for Amazon S3.
Enables you to upload/download files, share buckets and files, edit http headers and metadata, copy/move files and folders and many more!
S3 Browser is free for private/non profit use.
( )melsteve October 13th
very helpful, I was just looking into this. thanks.
( )Jef October 13th
A lot of works in this, thanks for the information.
( )MattD October 13th
The only downfall I see from this is if something like SmartFilter blocks content from the CDN. For example I can not read any tuts sites from work because all the images and demos are blocked.
( )Jarel October 13th
Either SmartFilter isn’t configured correctly or SmartFilter isn’t very smart if it’s blocking CDN’s.
( )Matt October 14th
It isn’t very smart. It doesn’t block Alkamai but does block amazonaws.com as Personal Storage.
Myfacefriends October 13th
wow only i can say thank you! this is a very clear explanation about CDN.
( )Montana Flynn October 13th
I really need to look into this for my eCommerce clients. Thanks!
( )Tylor October 13th
Really informative! Thanks man. I have been wanting to look into this for a long time!
( )John Sanders October 13th
Great article! Thanks.
( )Wayne October 13th
Great. But are there any other ways around the caching issues in the CDN? I tend to update the CSS files on my sites all the time, and versioning seems like it could get out of hand fast.
( )Gavin October 13th
Great article, I am going to give this a go.
( )I was wondering if you can use the same bucket? to serve different sites(URLS) or if that matters?
Jarel October 14th
Yup, you can certainly do that.
( )Mathieu RENE October 13th
Really interresting…
( )Peter October 14th
Great article!
Jarel, you said your last bill was less than 5$. Can you tell us how many visitors and how much traffic your site has monthly?
( )Jarel October 14th
Only a few thousand visitors per month. The bulk of my AWS bill is from storage costs because I use it to back up certain important data.
( )Benjamin Reid October 14th
This is a really, really good explanation. I think I’ll be using this more often for larger sites and If I forget how to do it, I’ll come back here, thanks!
( )Steven October 14th
Thanks alot. Very interesting article. I’d love to see more articles and more in depth articles about high scalability etc.
( )susann October 14th
Amazon cloudfront is cool but only for public files. You can not use it for private files and an authenticated_s3_url for a period of time.
( )David Moreen October 14th
Amazons Web Services does look very interesting, but I love my host. I don’t necessarily like the idea of paying for the storage, as apposed to playing $200 a year for unlimited space.
( )Brian Temecula October 14th
To love a host is rare. I have used many, and all seem to have problems. Do you mind sharing who this host is?
( )jasmu November 19th
it seems to be go daddy. wow, only heard bad things about their hosting. i guess his requirements are not too high.
i’m happy w/ my current host.
Jarel October 14th
When a web host sells you a hosting plan with “unlimited” anything at a fixed price, it’s just a marketing ploy. However, if you’re happy with the price and the service, then that’s good for you. Keep in mind too that S3 and CloudFront aren’t intended to be replacements for your web host, but instead supplements for performance and flexibility.
( )Owain Llewellyn October 14th
Great Article, very informative. I’ll deffinatley be considering this for future sites.
thanks for the insight..
( )Martin Leblanc October 14th
Great article – exactly what I was looking for.
( )Ulrik Hvide October 14th
Was just thinking about using AWS. Thanks a lot.
( )rglarsen October 14th
Great Article, very informative
I have googlede and tried to find the test that the article uses as on of arguments for improving the loaded time. Is there any one that have a link to the original tests ?
Regards
( )Rene
DemoGeek October 14th
Caching is kind of a minor turn off for me…imagine an evolving/dynamic content website that might be updated frequently and we have to make sure we name the files correctly or end up with spending hours on debugging to find out that its the cache that’s the culprit. Why can’t they make it in a way so that when the file gets updated the cache gets refreshed?
Otherwise, it seems to be a great way to improve the sluggish performance threshold provided by the traditional web hosts.
( )Shaun October 14th
I see what you are saying, but the point of caching is to reduce the load on a server regardless. If you refresh the cache every time you make an update, it partially negates the purpose. Unless its a high-priority fix or update, waiting a short amount of time for the cache to refresh should not be a problem.
( )Jarel October 14th
There are several other ways around Amazon’s caching and there are lots of other CDN’s available that operate slightly different. You might check out Rackspace. I’ll be writing another tutorial on using their cloud storage and CDN here in a few days for blog.themeforest.net
( )Ferdy October 15th
Because that would beat the purpose of the latency win. In your scenario, where files get cached unless they are updated, the client browser still has to check each time if the timestamp was updated on each file. Thus, this means you’re still making the HTTP request, thus you are still waiting for the connection latency, thus the whole purpose of a CDN is lost.
The author explained the correct usage of caching: using versions. Huge sites like Flickr do it this way too.
( )Americo Savinon October 14th
Great Post, I really like the way you structured the tutorial.
( )Lamin Barrow October 14th
I considering checking out EC2 and S3 services. I have read this article and i find it to be quite enlightening so thanks for posting.
( )Rick October 14th
Excellent post, I have been playing around with these and you have helped answer a few nagging questions
( )Palusko October 14th
“In 2006 Google’s tests showed that increasing load time by 0.5 seconds resulted in a 20% drop in traffic.
In 2007 Amazon’s tests showed that for every 100ms increase in load time, sales would decrease 1%.
This year (2009) Akamai (a CDN leader) revealed in a study that 2 seconds is the new threshold for eCommerce web page response times.”
I really wonder how you can measure this with 100% consistency. There are so many variables that sales depend on, and lot, if not most of them are much more related to the product, price and service than half a second slower download time. For some reason, this ‘milliseconds count’ argument seems more of a justification for services (CDN) that most people simply do not need.
( )Jarel October 14th
The studies these companies made were to learn more about the way page load time (among other things) effects certain parts of their business. They aren’t trying to convince anyone to use a CDN or any other service.
These huge companies are also very aware of the variables that contribute to sales (and other things) and are able to compensate for those variables so they don’t affect the study on a particular aspect. Of course it won’t be 100% consistent or accurate, nothing can be, but it’s close enough to be considered reliable information.
You have to remember that these companies are the best in the world at what they do. They analyze every aspect of what they do in detail because one tiny change can make a difference of millions of dollars. They’re very good at optimizing their business down to the tiny details, even milliseconds if it will make a difference.
( )Palusko October 14th
You see, I worked for one big corporation that was top of the line at what they did (non-IT related). And they too analyzed everything, everything had a flow chart, everything was documented etc. The problem with that was, that many times, it did not matter a bit. It was often done just for a job security of the supervisors. Kind of to show how and why they are irreplaceable
Now, I have no doubt that load speed is very important for overall user experience, and thus sales. But if 100ms longer load time means 1% decline in sales, does that work the other way around too – decrease of 100ms will make 1% jump in sales?
Anyway, despite of what you may think, I really liked this tutorial. Because unlike the millisecond parts that I questioned, your comparison showed much more dramatic improvement, definitely worth looking into.
Dennis October 15th
Yes, they are the best in the world at what the do, but are they the best in the world for what you do?
Having worked at Amazon I can tell you there where many times their own site and/or intranets where down or very slow. Now that may only be 1 or 2 percent of the time, but for every service you add to your site you are increasing the percentage of outages by adding a possible week link. I ‘m not saying its a bad idea, I just wonder how far I want to stretch the chain.
Jarel October 14th
Hey everyone, thanks for all the great comments!
I want to point out that a CDN is intended to be a supplement to your current web host, not a complete replacement. Its purpose is to provide better performance to your visitors from around the world, which of course will come at an additional cost. If you’re happy with your current web host performance and you don’t like the idea of paying a little more for increased performance, then this tutorial isn’t for you.
Thanks for reading the tutorial everyone!
( )GrandmasterB October 14th
I’ve used the Amazon AWS for awhile as an offsite automated backup for my website…it’s an excellent and cheap solution. I’m roughly spending $1.59/month for nightly backups.
I’ve been contemplating setting up a CDN using the AWS and this is just what I was looking for. Excellent post…and very informational. Thanks a ton!
BTW…how do you get the ‘developer’ menu to show in Safari?
( )Jarel October 14th
In Safari’s preferences under the Advanced tab, checkmark “Show Develop menu in menu bar”.
I’m glad you liked the post, thanks for reading!
( )Bill October 14th
If you have a minute and are watching these comments, I’d love to hear how you automate your nightly site backups with AWS. Love to hear the author’s thoughts on this as well. Thanks!
( )Chris October 15th
very nice! i actually read through it, unlike most of the articles i skim. keep it up jarel. thanks for taking the time.
( )Mark Dijkstra October 16th
Great Article Jarel, keep up the good work
( )Matthias October 16th
Great article. I’m currently looking into expanding an European website to the US market and this could become handy.
A little downside for me would be that you have to get another tool to manage the files on S3. Is there a possibility to automate the process? Like an auto-sync between a folder on your web server’s harddisk and the S3 bucket?
At the moment I’m still running a single server setup with lots of optimizations (nginx ftw) which does his job fine, but serving the stuff to the US could become too much. So I’ll definetly take a look at the AWS.
Cheers
( )Matthias
awake October 16th
sorry…
this is just extra work and money if a site does not get that much traffic
( )Dan October 16th
Or maybe a site doesn’t get much repeat traffic because it loads too slow. It’s not much extra work once you know how to do it. As explained above its very simple to setup and not expensive.
( )Dominic Morin October 17th
Great great article, Bravo!
Dominic
( )David Caunt October 18th
Great article.
I had problems with serving files ending in .css.gz to Safari PC and found that it always wants to download .gz files. The solution was simply to use cssgz as the extension.
( )dolito October 18th
Great article, tnx.
Threre is one advantage you maybe forget to explain about using a cdn. You take the load from your server! If your webserver has to serve all your static files, this could highly increase the cpu load of your server – depends on requests and files. So using a cdn, enables your server to handle more requests.
Regards
( )dolito
TC October 19th
Great article. My company has been serving media files for our digital asset manager using S3 and I’ll be using your guide to finally enable CloudFront.
I’d also like to verify everything Jarel has said about shared hosting. It’s fun while it lasts, but as soon as you start using too many of their resources (bandwidth, storage, cpu cycles), you will get the boot.
Regarding the pricing of AWS, I suggest everyone check out the AWS Calculator and do the math. I’m pretty sure we’re paying less than $20/month for resources we’d never get out of a shared hosting plan.
( )Jasper October 20th
Great article, but didn’t you mean “compress” in this sentence: “# Gzip files like JavaScript and CSS” because in the end you can Gzip every content type… but only javascript and CSS van be compressed….
We currently use Velocix and it’s way cheaper than other competitors, especially compared to Akamai. Which has a very inefficient network in my opinion.
Gr,
( )Jasper
Alex October 26th
On the Mac, Win, or Linux, I’d recommend you to use CrossFTP Pro ( http://www.crossftp.com/ ) for the metadata handling, and automatic gzip compression for the S3 contents, which is a good S3 tool as well as FTP tool.
( )Dan November 6th
Hmmm so is there going to be a rackspace follow up or not?
( )Juan November 13th
Hi, very nice information, however I have a small question about S3 and cloudfront, if I use both services, that mean I will be charge double??, or when my files get the first time from S3 then I will be charge for S3 and when the files comes from cloudfront then I will get charge just for cloudfront, thanks for the help!!!
( )