Basic usage of cUrl with PHP to GET content of pages and POST data : 5ubliminal's TellinYa

<a href="http://www.tellinya.com/art2/12/">Basic usage of cUrl with PHP to GET content of pages and POST data : 5ubliminal's TellinYa</a>
Must Reads: Web Scraping | Link Farming | Code Snippets | SEO Freeware
Reveal More!

This article has been superseeded by a more complex implementation of cUrl and PHP. The new implementation is a PHP class as easy to use as code on this page but with some extra features.

Basic understanding of cUrl and PHP

Basic understanding of cUrl and PHP is easy from the documentation available on PHP. But having functions prewritten that can serve most porpouses is always even better!

These functions have served all my cUrl usage needs including getting contents of webpages, downloading files or posting data. One more thing you need to use is handle COOKIES which is also simple and you have cUrl understood well enough to server most fo your cUrl PHP needs.

I added here 3 functions of which one is just a helper function:

  1. getPage($url, $referer, $timeout, $header)
    1. $url - the url you are about to POST to
    2. $referer - the url referer you are about to fake
    3. $timeout - timeout in seconds
    4. $header - extra headers to post as multi-line string
    Function downloads the html contents of a web page using basic cUrl.
  2. postPage($url,$pvars,$referer,$timeout)
    1. $url - the url you are about to POST to
    2. $pvars - an array containing the posted vars
    3. $referer - the url referer you are about to fake
    Function POSTs variables to a dynamic page using basic cUrl.

Retrieving pages with cUrl and PHP

<?
//-----------------------------------------------------------------------------------
function getPage($url$referer$timeout$header){
    if(!isset(
$timeout))
        
$timeout=30;
    
$curl curl_init();
    if(
strstr($referer,"://")){
        
curl_setopt ($curlCURLOPT_REFERER$referer);
    }
    
curl_setopt ($curlCURLOPT_URL$url);
    
curl_setopt ($curlCURLOPT_TIMEOUT$timeout);
    
curl_setopt ($curlCURLOPT_USERAGENTsprintf("Mozilla/%d.0",rand(4,5)));
    
curl_setopt ($curlCURLOPT_HEADER, (int)$header);
    
curl_setopt ($curlCURLOPT_RETURNTRANSFER1);
    
curl_setopt ($curlCURLOPT_SSL_VERIFYPEER0);
    
$html curl_exec ($curl);
    
curl_close ($curl);
    return 
$html;
}
//-----------------------------------------------------------------------------------
?>

Posting data to pages with cUrl and PHP

<?
//-----------------------------------------------------------------------------------
function postPage($url,$pvars,$referer,$timeout){
    if(!isset(
$timeout))
        
$timeout=30;
    
$curl curl_init();
    
$post http_build_query($pvars);
    if(isset(
$referer)){
        
curl_setopt ($curlCURLOPT_REFERER$referer);
    }
    
curl_setopt ($curlCURLOPT_URL$url);
    
curl_setopt ($curlCURLOPT_TIMEOUT$timeout);
    
curl_setopt ($curlCURLOPT_USERAGENTsprintf("Mozilla/%d.0",rand(4,5)));
    
curl_setopt ($curlCURLOPT_HEADER0);
    
curl_setopt ($curlCURLOPT_RETURNTRANSFER1);
    
curl_setopt ($curlCURLOPT_SSL_VERIFYPEER0);
    
curl_setopt ($curlCURLOPT_POST1);
    
curl_setopt ($curlCURLOPT_POSTFIELDS$post);
    
curl_setopt ($curlCURLOPT_HTTPHEADER,
        array(
"Content-type: application/x-www-form-urlencoded"));
    
$html curl_exec ($curl);
    
curl_close ($curl);
    return 
$html;
}
//-----------------------------------------------------------------------------------
?>

Last but not least

If you find it useful, and you certainly will ;), link to it and help spread the love!

6 Comments Posted By Readers :

Add your comment
#1 Renee from Singapore
Posted on Wednesday, 23 January, 2008
Hi,
May I know how to redirect to another page after posting a form to a page without using the FOLLOWLOCATION?

Thanks.
#2 5ubliminal web
Posted on Wednesday, 23 January, 2008
You just get the headers of the reply. And then find the Location: … value.
And then go to that location. Even if automatic follow is easiest.

PS: If you need more details lemme know.
#3 Lee Gordon from Great Britain
Posted on Monday, 18 February, 2008
Hi how do you enter the url you want to get the contents of.
#4 5ubliminal web
Posted on Tuesday, 19 February, 2008
The first parameter in the getPage function is the url but I strongly encourage you to check out the new version mentioned on top of the post.
#5 luis from Peru web
Posted on Thursday, 21 February, 2008
hi, Could you put some example how to use that functions?
#6 5ubliminal web
Posted on Thursday, 21 February, 2008
As I say on top of the POST I no longer support these scripts on this page. They're really easy to use but …
I encourage you to use the newer version of cURL and PHP I wrote which is an easy to use class.
Post Feedback 
Name *
Mail *
URL
« Anti-Spam
» URL will only go live after a review. Comments are moderated. «
5ubliminal's TellinYa.com SEM & SEO Blog © 2007 - All rights reserved unless mentioned otherwise .
Rendered On : [Thursday, 07 August, 2008 - 23:33:51 GMT]   No Ajax / Flash Used Here
" Basic usage of cUrl with PHP to GET content of pages and POST data : 5ubliminal's TellinYa "