PHP : Web Automation - [11.5] Fetching a URL with Headers

11.5.1 Problem

You want to retrieve a URL that requires specific headers to be sent with the request for the page.

11.5.2 Solution

Use the cURL extension and the CURLOPT_HTTPHEADER option:

$c = curl_init('http://www.example.com/special-header.php');
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_HTTPHEADER, array('X-Factor: 12', 'My-Header: Bob'));
$page = curl_exec($c);
curl_close($c);

If cURL isn't available, use the addHeader( ) method in HTTP_Request:

require 'HTTP/Request.php';

$r = new HTTP_Request('http://www.example.com/special-header.php');
$r->addHeader('X-Factor',12);
$r->addHeader('My-Header','Bob');
$r->sendRequest();
$page = $r->getResponseBody();

11.5.3 Discussion

cURL has special options for setting the Referer and User-Agent request headers — CURLOPT_REFERER and CURLOPT_USERAGENT:

$c = curl_init('http://www.example.com/submit.php');
curl_setopt($c, CURLOPT_VERBOSE, 1);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_REFERER, 'http://www.example.com/form.php');
curl_setopt($c, CURLOPT_USERAGENT, 'CURL via PHP');
$page = curl_exec($c);
curl_close($c);