PHP - Remote Login Using Curl

My real job is as a radio announcer.  We are required to play advertising commercials for various programs that we broadcast.  I wrote a script a few years ago to automatically login to the providers website (PHP, Curl) and download the mp3's that we are supposed to play each day.

Now, the provider has updated their website and changed the login process.

It used to be (in a browser) that when you logged in, you were taken to a landing page with a unique "rowid" in the in URL.  Once you knew that rowid, you could simply go to that page, and bypass the login process.  Not very secure...but easy to scrape!

Now, when you login, you are assigned a random rowid.  Using the Firefox Deveoloper Tools, I see that there is NO cookie associated with the login.  The rowid is now set in the form processing page.  In the Developer Tools, this is seen in the Params...Query String section.

How can I extract these params (rowid) from the form processing script, and proceed to the landing page...with the rowid in the Curl command?

This just returns me to the login page..since the rowid isn't set.

Thanks!

 

$url = 'http://domain.com/formprocessing.html';
$fields = array(
	'username' => urlencode('xxx'),
	'password' => urlencode('xxx')
);

$fields_string = '';

foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string, '&');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_POST, count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);

//execute post
$result = curl_exec($ch);

//close connection
curl_close($ch);

I'm trying to login to a site, navigate a couple of pages deep, and then scrape a table.  I had all this working, but the site changed their login workflow...and I can't figure out what I need to do to make this work.

The form used in a browser to login has itself as the action page.  In a browser, you are automatically redirected to another page, that puts a random get variable at the end of the URL.

Here's the main login page:  https://www.premiereaffidavits.com/scripts/cgiip.exe/WService=webprime/login.html

When I run a standard curl login code...I only get the login page returned to me.  How can I get the redirected page (including the random GET variable at the end of the URL) that you get using a browser?  I see no javascript, or other ways for the action page to redirect to the second page you get in a browser.  What am I missing here?

I can generate login info if you need it.

Thanks.

I had a script to automatically login to a site, and download data from it.  The provider just updated their pages.  They now appear to be JSON.  The previous code was straight php and curl.  When trying to login now, I get this error page echoed:

405 - HTTP verb used to access this page is not allowed.

Any pointers as to where I can look to re-write the code based on this error?  I'm not looking for someone to write the code for me...just a pointer to where to look for how to login to this site.

FYI, the site is:  https://www.premiereaffidavits.com/login

Thanks!

username=username&password=password&Submit=Conectare&option=com_users&task=user.login&return=aW5kZXgucGhwP0l0ZW1pZD0yMTY%3D&0dbf64fe20e2395a7d72ed5b64b3cf7c=1

<fieldset class="userdata">
    
<p id="form-login-username">
        <label for="modlgn-username">Nume Utilizator</label>
        <input id="modlgn-username" type="text" name="username" class="inputbox" size="18">
    </p>
    <p id="form-login-password">
        <label for="modlgn-passwd">Parola</label>
        <input id="modlgn-passwd" type="password" name="password" class="inputbox" size="18">
    </p>
        <p id="form-login-remember">
        <label for="modlgn-remember">Retine utilizator</label>
        <input id="modlgn-remember" type="checkbox" name="remember" class="inputbox" value="yes">
    </p>
    
    <input type="submit" name="Submit" class="button" value="Conectare">
    <input type="hidden" name="option" value="com_users">
    <input type="hidden" name="task" value="user.login">
    <input type="hidden" name="return" value="aW5kZXgucGhwP0l0ZW1pZD0yMTY=">
    <input type="hidden" name="11b09608b3184e6258012d44846c81ed" value="1">    

</fieldset>

function login_to_website($targetURL){ 

    global $browser_user_agent; 
    if(empty($targetURL)) { return; } 
    if(empty($login_url)) { $login_url = $targetURL; } 
    $url = $login_url; 

    $login_user     = "loginusername"; 
    $login_password = "loginpassword"; 
    $thetoken       = "this-is-my-problem-the-token-from-the-hidden-input";         

    $post_data = array();    
    $post_data['username']  = "$login_user";  
    $post_data['password']  = "$login_password";  
    $post_data['Submit']    = "Conectare"; 
    $post_data['option']    = "com_users"; 
    $post_data['task']      = "user.login"; 
    $post_data['return']    = "aW5kZXgucGhwP0l0ZW1pZD0yMTY%3D"; 
    $post_data[$thetoken]   = "1";           

    $postthis = http_build_query($post_data); 

    $login = curl_init(); 

    curl_setopt($login, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookie.tmpz"); 
    curl_setopt($login, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookie.tmpz"); 
    curl_setopt($login, CURLOPT_VERBOSE, true); 
    curl_setopt($login, CURLOPT_URL, $url); 
    curl_setopt($login, CURLOPT_USERAGENT, random_user_agent()); 
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE); 
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);   
    curl_setopt($login, CURLOPT_POST, TRUE); 
    $timeout = 5; 
    curl_setopt( $login, CURLOPT_CONNECTTIMEOUT, $timeout ); 
    curl_setopt( $login, CURLOPT_TIMEOUT, $timeout ); 
    curl_setopt( $login, CURLOPT_MAXREDIRS, 10 );    

    curl_setopt($login, CURLOPT_POSTFIELDS, $postthis); // POST vars 

    curl_setopt($login, CURLOPT_HEADER, 0); // debug headers sent - 1 

      $data = curl_exec ($login); 

      curl_setopt($login, CURLOPT_URL, $targetURL); 

      $datax = curl_exec ($login); 
      return $datax; 

      // close cURL resource, and free up system resources 
      curl_close($login); 
}  

    $htmlx = file_get_contents('http://www.v-tac.ro'); 
    $htmlx = mb_convert_encoding($htmlx, 'UTF-8', mb_detect_encoding($htmlx)); //make sure this is utf8 
    if(!strlen($htmlx)) {echo "No HTML here . stoping execution ."; return;} 
    $doc = new DomDocument; 
    @$doc->loadHTML($htmlx); 
    $xpath = new DOMXPath($doc); 

    echo $xpath->query('//fieldset[@class="userdata"]/input[5]')->item(0)->getAttribute("name"); 
    $thetoken = $xpath->query('//fieldset[@class="userdata"]/input[5]')->item(0)->getAttribute("name");  

<form method=post>
<input type="hidden" name="action" value="logon">
<table border=0>
<tr>
<td>Username:</td>
<td><input name="username" type="text" size=30></td>
</tr>
<tr>
<td>Password:</td>
<td><input name="password" type="password" size=30></td>
</tr>
<td></td>
<td align="left"><input type=submit value="Sign In"></td>
</tr>
<tr>
<td align="center" colspan=2><font size=-1>Don't have an Account ?</font> <a href="?action=newuser"><font size=-1 color="#0000EE">Sign UP Now !</font></a></td>
</tr>
</table>

<?php
$username="username"; 
$password="password"; 
$url="http://www.example.com/index.php"; 

$postdata = "username=".$username."&password=".$password;

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_REFERER, $url); 
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch); 
header('Location: track.html'); 
//echo $result;  
curl_close($ch);
?>