Update: When I pasted this script I had some problems with the slashes that got lost on the way. I did not notice it and most who use it fixed them but I changed the script now to a functional version.

This has to be regarded as an educational script and should not be used outside Google.com guidelines! Check here for The MSN Version and this is the link to a Google results counter script.

Warning : This article uses and relies on the PHP Script cUrl Class which is mandatory for the following script to work!

Why would one need to parse the Google Results?

You should not need to do this. This script is educational and not for unethical use!

But … hypothetically speaking … this could be used for several reasons. To find your ranking in the SERPs, to see how many pages you have indexed, to see competition and watch certain keyphrases for new sites or sites that drop and so on …

If you can't help it and you will use this use common sense and do not hammer Google. You will get a temporary ban and it's not so great. Keep at least 30-60 seconds between queries and do not do more then 25 in a row. Simulate human behaviour or be kicked. Even if you help Google save bandwidth by not viewing the search page dependencies you loose him money for not clicking the ads!

Detect supplemental pages with this script.

There is a trick, don't know how long it will last that allows you to use a * at the end of the search to find non-supplemental pages.

So site:yoursite.com/ will show all pages indexed, and site:yoursite.com/* will list those in main index (non-supplemental). Do the math and get the list of supplemental pages by differencing the two arrays. PHP coders are always lucky on this site:

<? // Easy way to spot your supplemental pages $AllLinks = googleLinks("site:yoursite.com/",1,100); $MainIndex = googleLinks("site:yoursite.com/*",1,100); $Supplementals = array_diff($AllLinks,$MainIndex); ?>

Virtual legends say that site:yoursite.com/& brings supplementals directly but the numbers do not add up so I do not consider this query functional.

The PHP Script To Parse Google Results

Enough with the chit-chat! Let's get on to the code …

$page parameter is the page number and starts from 1. Never use 0-9 pages but 1-10!

<?
function googleResults(
    $query,$page=1,$perpage=10,
    $dc="www.google.com",$filter=true
){
    if($page) $page--;
    $url=sprintf("http://%s/ie?q=%s&num=%d&start=%d&hl=en&ie=UTF-8&filter=%d&c2coff=1&safe=off",
        $dc,urlencode($query),$perpage,$page*$perpage,$filter);
    $hc=new eHttpClient();
    $hc->setReferer("http://".$dc."/");
    $html=$hc->get($url);
    if(!preg_match_all( "/<nobr>(.+?)<\/nobr>/is", $html, $matches))
        return false;
    $matches=$matches[1];
    $results=array();
    for($i=0;$i<count($matches);$i++){
        $match=trim($matches[$i]);
        if(!preg_match_all( "/(.+?)\.\s<a title=[\"](.+?)[\"] href=(.+?)>(.+?)<\/a>/i",
            $match, $parts)) continue;
        $parts[4][0]=strip_tags($parts[4][0]);
        array_splice($parts,0,1);
        $LinkTitle    =trim($parts[3][0],"\r\n\t \"");
        $LinkDesc    =trim($parts[1][0],"\r\n\t \"");
        $Rank        =trim($parts[0][0]);
        $LinkUrl    =trim($parts[2][0],"\r\n\t \"");
        if(!strstr($LinkUrl,"://"))
            continue;
        if(!preg_match("/^([^:]+):\/\/([^\/]+)[\/]?(.*)$/",$LinkUrl,$Dom)){
            continue;
        }
        $Http=$Dom[1];
        $Rel="/".$Dom[3];
        $Dom=$Dom[2];
        $serp=array(
            "Rank"            => $Rank,
            "Url"            => $LinkUrl,
            "Title"            => trim(html_entity_decode(strip_tags($LinkTitle))),
            "Host"            => $Dom,
            "Protocol"        => $Http,
            "Path"            => $Rel,
            "Summary"        => trim(html_entity_decode(strip_tags($LinkDesc))),
        );
        array_push($results,$serp);
    }
    return $results;
}
// --
function googleLinks(
    $query,$page=1,$perpage=10,
    $dc="www.google.com",$filter=true
){
    $res=googleResults($query,$page,$perpage,$dc,$filter);
    $links=array();
    for($i=0;$i<count($res);$i++){
        $link=$res[$i]['Url'];
        array_push($links,$link);
    }
    return $links;
}
?>

The googleLinks is a helper function and will only list the URLs of the results.

And finally a sample output …

It is the first element of an array of 10 output by:

<? print_r(googleResults("site:tellinya.com/",1,10)); ?>

Array ( [0] => Array ( [Rank] => 1 [Url] => http://www.tellinya.com/ [Title] => (I'm)TellinYa - Bits of wisdom published by regular people [Host] => www.tellinya.com [Protocol] => http [Path] => / [Summary] => (I'm)TellinYa - Bits of wisdom published by regular people. ) )

n-Joy and stay safe! I take no responsabilities, ... bla bla bla.

28 Comments Posted By Readers :

Add your comment

#1 Ceros from Philippines

Posted on Wednesday, 19 September, 2007

cud you kindly tell me how do i scrap the top 5 url links in a google search???using curl?

#2 5ubliminal web

Posted on Wednesday, 19 September, 2007

Copy the code from here and the one I refer on top of the page and then use:

$links=googleLinks("your search goes here",1,10);

$links=array_slice($links,0,5);

And you got the 5 links.

#3 Référencement Google from Switzerland web

Posted on Thursday, 06 March, 2008

This script is nice but it doesn't always work on shared hosting, you need to set a proxy. Adding proxies will also let you run this function in a loop to retrieve ranks for multiple requests. Thanks for the script :-)

#4 Small Business SEO from United States web

Posted on Friday, 04 April, 2008

Great script thanks.

#5 Sharry from Great Britain web