Quick Tip: Cross Domain AJAX Request with YQL and jQuery
videos

Quick Tip: Cross Domain AJAX Request with YQL and jQuery

Tutorial Details
  • Technologies: YQL, jQuery, Regular Expressions
  • Difficulty: Moderate
  • Length: 8 Minute Video

For security reasons, we cannot make cross-domain AJAX requests with jQuery. For example, I can’t call the load() method, and pass in ‘cnn.com’. As we’d be loading in scripts and such, as well as our desired content, this would present a significant security risk. Nonetheless, there may be times when this is specifically what you require. Thanks to YQL, we can allow for this functionality rather easily!

The Script

// Accepts a url and a callback function to run.
function requestCrossDomain( site, callback ) {
	
	// If no url was passed, exit.
	if ( !site ) {
		alert('No site was passed.');
		return false;
	}
	
	// Take the provided url, and add it to a YQL query. Make sure you encode it!
	var yql = 'http://query.yahooapis.com/v1/public/yql?q=' + encodeURIComponent('select * from html where url="' + site + '"') + '&format=xml&callback=cbFunc';
	
	// Request that YSQL string, and run a callback function.
	// Pass a defined function to prevent cache-busting.
	$.getJSON( yql, cbFunc );
	
	function cbFunc(data) {
	// If we have something to work with...
	if ( data.results[0] ) {
		// Strip out all script tags, for security reasons.
		// BE VERY CAREFUL. This helps, but we should do more. 
		data = data.results[0].replace(/<script[^>]*>[\s\S]*?<\/script>/gi, '');
		
		// If the user passed a callback, and it
		// is a function, call it, and send through the data var.
		if ( typeof callback === 'function') {
			callback(data);
		}
	}
	// Else, Maybe we requested a site that doesn't exist, and nothing returned.
	else throw new Error('Nothing returned from getJSON.');
	}
}

Call the Function

requestCrossDomain('http://www.cnn.com', function(results) {
   $('#container').html(results);
});

Stripping Out the Script Tags

I had to progress rather quickly in the video, so perhaps the regular expression that strips out the <script> tags require further detail.

.replace(/<script[^>]*>[\s\S]*?<\/script>/gi, ''); 

When we load our desired page, it’s also going to load scripts! You must be very careful when making cross domain request. It definitely helps to strip out the <script> tags, but you should do more in an actual project.

Let’s take the regular expression step by step.

<script[^>]*>

Find all open script tags; however, they could come in many forms: <script type=”text/javascript” src=”bla.js”></script> , or <script type=”text/javascript”>lots of code here…</script> . For this reason, we add a character class ( [^>]* ), which mean, “Find zero or more of anything that IS NOT a closing bracket. This will take care of the attributes and values.

[\s\S]*?

Next, we want to strip out all code, as well as any spacing. \s refers to a space. \S refers to anything that IS NOT a space. Once again, we add a * after the character class to designate that we want zero or more occurrences.

<\/script>

Finally, find the closing script tags.


Further Reading

This is only meant to provide a glimpse of how we can achieve this functionality. Only so much can be covered in a five minute video. Feel free to discuss in the comments, and you’re always encouraged to fork the source code to improve upon it!

Note: Want to add some source code? Type <pre><code> before it and </code></pre> after it. Find out more
  • john

    very nice, I was always wondering how to do something like this, thanks.

    • http://priostudio.co.cc Sisir

      Hi, i was wondering if anyone can guide me how to do a api request by yql? Suppose the api returns JSON data so what command should i send.

      You used here : select * from html…. but if the api returns JSON data then what will be the command select * from JSON?? or something like this? where can i find the properties i can select by yql? i am searching for it about two hours now in yahoo…

      please help. Thanks!

  • http://newarts.at Drazen Mokic

    Man i hate youtube when i`m watching tutorials becouse of the poor quality the videos have -.-

    I subsrcibed know on youtube but it keeps telling me “This video is private”. Someone help? :)

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      It’s a bug with YouTube right. Bunches of people are complaining about it.

      The Nettuts+ videos on YouTube should still display pretty well. Just make sure that you select the 720 or 1080 option to view in HD.

      The embeds on Nettuts+ should automatically begin playing in HD.

      • Anto

        Should just stick with screenr. No fuss, no mess.

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        The bug has been fixed. The video will play now. :)

      • http://newarts.at Drazen Mokic

        If i switch to 720 it is even worse. Its really horrible, letters are quite impossible to read. I tested in Firefox and Google Chrome. Is someone else having the same issues or is this a bad configuration (whatever it could be) on my side?

        ps: The video is still not working :(

      • http://newarts.at Drazen Mokic

        Error Message (Translated from German):

        This video is private. If this video was sent to you, you have to accept the “senders” request.

        Bug 2 xD

        The comments form does not remember my information (name, email, url) but cookies are turned on.

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        @Drazen – Sounds like you’re viewing at the lowest resolution. When you watch the video above, does it say 1080 in the video (lower right)? It should. For me, it’s displaying really well.

      • Jeremy

        Same here, quality is absolutely terrible, and also worse here at 720, I really hope you can go back to screenr, or offer extra ways, because even full screen on my macbook in safari it’s pretty difficult to read the code

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        @Drazen – I just checked, and am pretty sure that YouTube has fixed the problem. You shouldn’t need to sign-in/sign-up to view it.

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        This is really strange. Are you guys viewing full screen? I’m not experiencing this blurriness at all.

    • http://newarts.at Drazen Mokic

      Ok now its works! On 1080 the quality is really good, nice. Maybe i am starting to like YT. (On 720 it`s still ugly).

      • Jeremy

        Agreed, same here, looks pretty good now that I checked. I was going by the “How to Change your Website’s Background with Each WordPress Post” post because the new one wasn’t showing up for me and that looked bad even at 720, but the new one looks good. Thanks

      • http://newarts.at Drazen Mokic

        Yes, all the older videos look really horrible on 720 for me.

        Example: http://tinyurl.com/yafcecb

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        Ok – just saw what you’re referring to. Must be another YouTube issue. So you load that page, enter full screen, and then change the resolution to 720. Then, it displays really badly right? Trying switching out of full screen, and right back into it. The display should fix itself.

        That’s really weird! Just watch the auto-HD videos on Nettuts+. ;) I’ll also add Screenr/Blip versions when applicable.

  • http://twitter.com/DanielMorosan Daniel

    Huh, YouTube is too buggy. Please try Vimeo or Blip.

  • http://www.geetpurwar.com Geet

    Cool Jeffrey :)

  • http://Yqlblog.net Jonathan

    Using jquery to do a callback this way busts YQLs caches and hurts your performance. Please take a look at the caching blog entry we posted recently on yqlblog.net

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      @Jonathan – That’s really good to know. So if we replace the anonymous callback with a define function, that’ll solve the problem?

      • http://www.jeffrey-way.com Jeffrey Way
        Author

        Okay – I’ve updated the article and the source code accordingly:

        http://github.com/JeffreyWay/jQuery-Cross-Domain-Requests

        Thanks Jonathan! Great tip.

      • http://johndturner.com John David Turner

        I’m still seeing cache busting code being sent, something like callback=jsonp1269612290623 being appended to the call. I tried

        $.ajax({
        url: query,
        dataType: ‘jsonp’,
        jsonp: ‘callback’,
        jsonpCallback: ‘cbfunc’
        });

        as described in the yql blog article and it seemed to fix it.

  • CK

    I am getting an “An error has occurred. Try again later.” in the embedded clip above.

  • http://www.quizzpot.com Crysfel

    Awesome! I didn’t know about this, thanks you! :)

  • http://www.matthoneywill.net Matt

    Is there a way to change the client headers? I’m wanting to make a forum reader using this but they’ve disallowed all bots via robots.txt and YQL doesn’t like this…

  • http://www.jordanwalker.net Jordan Walker

    Learning something new every day, thanks for the tutorial.

  • http://nooshu.com Matt

    Thanks for the tutorial, i’ve been hearing lots about YQL as Christian Heilmann has mentioned it quite a lot in his tweets. This is a perfect introduction.

  • Lou

    now if only there were a way to post to a different domain….sigh

  • Joy

    Thanks, that was a good intro to YQL. I’d previous solved the cross-domain loading by using PHP to download the content I need, and reading the results using jQuery.

    I read the blog entry Jonathan pointed out, and realized that some applications really can’t cache, especially if they’re gathering real-time data that really can change every ten seconds. Maybe those apps shouldn’t use this particular solution (because it whales on Yahoo servers)?

    Really useful tutorial!

  • Richard

    YouTube Eh? If that’s the future of Nettuts+ then that is unfortunate. I watch these at work and will no longer be able to view them if they are all YouTube.

  • http://www.greenanysite.com/ Tal Ater

    I created something very similar to this a few weeks ago, and after running for a few days, YQL started blocking our query…

    It didn’t matter that we never passed YQL’s query limits, and cached everything that could be cached…We still got blocked.

    We had to rewrite the whole thing to be done on our server with PHP.

  • http://www.yycevents.com Chad

    Wow, only if i had found this a six months ago for a project i was working on.

  • Khaled

    well youtube doesn’t work here in my country

  • http://adrusi.com adrian

    There’s a cross domain ajax library called cows ajax http://cows-ajax.sourceforge.net/tech.php

  • alex

    like to make an ajax/jquery tutorial request: how to make the back browser button to work correctly in a filter search ->result page->detail page->result page

    kind regards

  • http://andrewtrivette.com Andrew

    Actually it is possible to make a cross-domain AJAX request with Jquery alone for $.getJSON requests. I do it all the time. The Jquery site even has an example of a remote call in the documentation: http://api.jquery.com/jQuery.getJSON/

  • http://www.warpdesign.fr leo

    Nice article, as usual !

    I always thought cross-domain ajax wasn’t allowed: I’m wondering how does using getJSON changes that…

    Any idea ? If I wanted to create some API that could be called by cross-domain Ajax, what should I do ? Returing JSON is enough ?

  • http://benjaminroesner.de benroe

    how can i make 3 or 4 request in a loop? one after another

  • George

    Hello there,
    That’s a really goog article.

    I’m trying to use your script to retrieve an Oauthenticated query using as passing “yql” value a signed one returned by the function of the followinbg script -> http://paul.donnelly.org/2008/10/31/2-legged-oauth-javascript-function-for-yql/ script, but unseccesfull.

    I wonder if you or somebody else know the reason of not working or any alternative way to use Cross Domain AJAX Request with YQL and jQuery using an authenticated query.

    Thank you very much.

    Regards,
    George K.

  • http://faressoft.org faressoft

    This is a great tutorial, Thank you very much.

    but how can i get all content not only

  • Matt

    Just wanted to know how the jQuery would look if I were trying to use css selectors.

    So what you have here on the tut:
    var yql = ‘http://query.yahooapis.com/v1/public/yql?q=’ + encodeURIComponent(‘select * from html where url=”www.holylandmoments.org/devotionals/the-sabbath-experience”‘) + ‘&format=xml&callback=?’;

    And the YQL syntax looks like such:
    use ‘http://yqlblog.net/samples/data.html.cssselect.xml’ as data.html.cssselect; select * from data.html.cssselect where url=”www.holylandmoments.org/devotionals/the-sabbath-experience” and css=”#main-content”

    How can I incorporate the USE attribute into the jquery yql var.

    Thanks!

  • 8w_gremlin

    excellent post – but how can you access a page that requires basic loggin in with an email and password?

  • michael

    nice article !
    i found a similar article on YQL, the author has given sample code also !

    http://www.tutkiun.com/2010/11/yahoo-query-language.html

  • http://priostudio.co.cc Sisir

    Hi, i am requesting the json data from a api via YQL. I am receiving the result ok in YQL console but when i am calling from my local computer i am getting null ! Can any one tell what i did wrong?

    here is the yql i am using:

    select * from json where url=”http://hyperlocal-api.outside.in/v1.1/states/Illinois/cities/chicago/stories?dev_key=MY_KEY&sig=MY_SIGNATURE&limit=3″

    please advice..

  • http://www.twitter.com/harkinlarjar Darmie

    can this be implemented in your recent article –> http://net.tutsplus.com/tutorials/javascript-ajax/how-to-build-an-rss-reader-with-jquery-mobile-2/ ?

    please if it can be implemented, explain how i could go about it.

    Thanks in advance
    Darmie

  • http://www.engiguide ajay

    i want to pass direct url and want result… how can i ?

    (‘select * from html where url=”‘ + site + ‘”‘) instade of like this (‘select * from html where url=”www.google.com”‘)

    and i want parsed json output on pageload….

  • http://abdullah.webdesignerart.com Abdullah

    very nice affective its solve my problem :) thnkx professionals!

  • joe

    I changed the format from XML to JSON then what do I need to make change in here?
    Please help!

    *******************************************************

    var yql = ‘http://query.yahooapis.com/v1/public/yql?q=’ + encodeURIComponent(‘select * from html where url=”‘ + site + ‘”‘) + ‘&format=jason&callback=?’;

    alert(yql);
    // Request that YSQL string, and run a callback function.
    // Pass a defined function to prevent cache-busting

    $.getJSON(yql, cbFunc);

    function cbFunc(data) {
    // If we have something to work with…
    if (data.results[0]) {

    //data = data.results[0].replace(/]*>[\s\S]*?/gi, ”);
    data = data.results[0].item;

    // If the user passed a callback, and it
    // is a function, call it, and send through the data var.
    if (typeof callback === ‘function’) {
    callback(data);
    }
    }
    // Else, Maybe we requested a site that doesn’t exist, and nothing returned.
    else throw new Error(‘Nothing returned from getJSON.’);
    }

  • Amit
  • http://www.coderrific.com Mike

    Thanks! Very helpful. I gotta love YQL API, I just used it and for further reading:
    http://coderrific.com/forum/detail/how_to_use_external_rss_feeds_for_your_site_in_json_with_jquery

  • Bahadir

    That’s a really good article conguratulations…
    I am trying to use this technique to try if i can run it but unfortunately i failed. here is my code :

    Cross-Domain Ajax Request

    function cbfunc(data){
    if( data.results[0] ){
    data = data.results[0].replace(/]*>[\s\S]*?/, ”);
    $(‘#container’).html(data);
    }
    else{
    throw new Error(‘Nothing returned from getJSON.’);
    }
    }

    function sfunc(){
    //”http://www.google.com.tr/search?hl=tr&q=”
    var site = $(‘#sitename’).val();
    if ( !site ) {
    alert(‘No site was passed.’);
    return false;
    }
    var yql = ‘http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%20%3D%20%22′ + site + ‘%22&format=json&diagnostics=true&callback=cbfunc’;
    $.getJSON(yql, cbfunc);
    return false;
    /*
    requestCrossDomain(path, function(results){
    $(‘#container’).html(results);
    });
    event.preventDefault();
    return false;*/
    }

    Type a URL :

    Can anyone show my mistake on this code? Because when i debug it with firebug on firefox, it’s seemed that it can’t enter into the function “cbfunc”. Thanks fellas.

    • Bahadır

      Ok fellas i fixed it, the problem was yql. The expression “callback=cbfunc” at the end waas wrong, it was searching to use a function called “cbfunk” but i wanted to use my callback function so i replaced it with “callback=?” and it worked, thanks any way keep coding.

  • Luke

    I can’t seem to get it to work :(
    I’ve copied the code directly and nothing gets returned. Does it require json to be installed on the server, or is it built into jquery?

  • Yogish

    Hi Its well clear that the get process on cross domain is running perfect and even i have implemented in the code. Really wonderful !!!

    But what is the solution to a $.get where the web service method requires more than one paramater?
    &
    How can we implement the post process using YQL?

  • Gurmeet Singh

    Too good explanation. Thanks a ton

  • http://www.facebook.com/chris.h.cheng Chris Cheng

    do you know an easy way for me to modify or add a request header that yahoo would then to the destination server?

  • http://www.facebook.com/avitalbe Avital Ben Lulu

    I can’t download the files. Can someone send it to me ?