PHP : Web Automation - [11.3] Fetching a URL with the POST Method

11.3.1 Problem

You want to retrieve a URL with the POST method, not the default GET method. For example, you want to submit an HTML form.

11.3.2 Solution

Use the cURL extension with the CURLOPT_POST option set:

$c = curl_init('http://www.example.com/submit.php');
curl_setopt($c, CURLOPT_POST, 1);
curl_setopt($c, CURLOPT_POSTFIELDS, 'monkey=uncle&rhino=aunt');
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec($c);
curl_close($c);

If the cURL extension isn't available, use the PEAR HTTP_Request class:

require 'HTTP/Request.php';

$r = new HTTP_Request('http://www.example.com/submit.php');
$r->setMethod(HTTP_REQUEST_METHOD_POST);
$r->addPostData('monkey','uncle');
$r->addPostData('rhino','aunt');
$r->sendRequest();
$page = $r->getResponseBody();

11.3.3 Discussion

Sending a POST method request requires special handling of any arguments. In a GET request, these arguments are in the query string, but in a POST request, they go in the request body. Additionally, the request needs a Content-Length header that tells the server the size of the content to expect in the request body.

Because of the argument handling and additional headers, you can't use fopen( ) to make a POST request. If neither cURL nor HTTP_Request are available, use the pc_post_request( ) function, shown in Example 11-1, which makes the connection to the remote web server with fsockopen( ).

Example 11-1. pc_post_request( )

function pc_post_request($host,$url,$content='') {
    $timeout = 2;
    $a = array();
    if (is_array($content)) {
        foreach ($content as $k => $v) {
            array_push($a,urlencode($k).'='.urlencode($v));
        }
    }
    $content_string = join('&',$a);
    $content_length = strlen($content_string);
    $request_body = "POST $url HTTP/1.0
Host: $host
Content-type: application/x-www-form-urlencoded
Content-length: $content_length

$content_string";

    $sh = fsockopen($host,80,&$errno,&$errstr,$timeout)
        or die("can't open socket to $host: $errno $errstr");

    fputs($sh,$request_body);
    $response = '';
    while (! feof($sh)) {
        $response .= fread($sh,16384);
    }
    fclose($sh) or die("Can't close socket handle: $php_errormsg");

    list($response_headers,$response_body) = explode("\r\n\r\n",$response,2);
    $response_header_lines = explode("\r\n",$response_headers);
        
    // first line of headers is the HTTP response code
    $http_response_line = array_shift($response_header_lines);
    if (preg_match('@^HTTP/[0-9]\.[0-9] ([0-9]{3})@',$http_response_line,
                   $matches)) {
        $response_code = $matches[1];
    }

    // put the rest of the headers in an array 
    $response_header_array = array();
    foreach ($response_header_lines as $header_line) {
        list($header,$value) = explode(': ',$header_line,2);
        $response_header_array[$header] = $value;
    }
    
    return array($response_code,$response_header_array,$response_body);
}

Call pc_post_request( ) like this:

list($code,$headers,$body) = pc_post_request('www.example.com','/submit.php',
                                             array('monkey' => 'uncle',
                                                   'rhino' => 'aunt'));

Retrieving a URL with POST instead of GET is especially useful if the URL is very long, more than 200 characters or so. The HTTP 1.1 specification in RFC 2616 doesn't place a maximum length on URLs, so behavior varies among different web and proxy servers. If you retrieve URLs with GET and receive unexpected results or results with status code 414 ("Request-URI Too Long"), convert the request to a POST request.