PHP - Trying To Get $postdata And Curl To Get Members.txt Data

I am trying to go to http://lirr42.mta.info/
and get the data from the submitted form. I have rebuilt the form exactly as needed, and done a few adjustments.  The posts that I am sending from MY form, are the same as the one on the site.  This site allows public data mining by the way.  So does someone seem something wrong with my code?

I know the basics are working. I have another page that is working fine. It's just this one ends up returning something..it returns part of the data but it's something wrong with it.

Any advice is appreciated.
Code: [Select]
<?php
define( 'DEBUG', true );

define( 'DEFAULT_STOP', 'Broadway' );

// report all errors during development/debug mode
if ( DEBUG ) 
error_reporting( E_ALL );
else
error_reporting( 0 );

include( 'simplehtmldom/simple_html_dom.php' );

$mta_post_url = 'http://lirr42.mta.info/schedules.php';

// GTFS Specification File Definitions
$gtfs_files = array
(
'agency'          =>  'agency.txt',                  // required
'stops'           =>  'stops.txt',                   // required
'routes'          =>  'routes.txt',                  // required
'trips'           =>  'trips.txt',                   // required
'stop_times'      =>  'stop_times.txt',              // required
'calendar'        =>  'calendar.txt',                // required
'calendar_dates'  =>  'calendar_dates.txt',
'fare_rules'      =>  'fare_rules.txt',
'fare_attributes' =>  'fare_attributes.txt',
'shapes'          =>  'shapes.txt',
'frequencies'     =>  'frequencies.txt',
'transfers'       =>  'transfers.txt'
);

$gtfs_pickup_codes = array
(
    0 => 'Regularly scheduled pickup',
    1 => 'No pickup available',
    2 => 'Must phone agency to arrange pickup',
    3 => 'Must coordinate with driver to arrange pickup'
);

$gtfs_dropoff_codes = array
(
0 => 'Regularly scheduled drop off', 
    1 => 'No drop off available',
    2 => 'Must phone agency to arrange drop off',
    3 => 'Must coordinate with driver to arrange drop off'
);

// load stops file 
$stopsFile = fopen( 'data/lir/' . $gtfs_files[ 'stops' ], "r" );

if ( $stopsFile )
{
// get (and toss) header row
$stopsHeader = fgetcsv( $stopsFile, 1000 );

// print_r( $stopsHeader );

// will hold HTML output for Select dropdown for stops
$stopSelectOptions = '';

// build array from file data
while ( $data = fgetcsv( $stopsFile, 1000 ) )
{
$stopsData[ $data[1] ] = $data[0]; 
}

// get a sorted array of the stop names ( yes this could be done with array_multisort or usort but I didnj't feel like it right now )
$stopNames = array_keys( $stopsData );
sort( $stopNames );

// loop through sorted array and create SELECT options with default SELECTED at Grand Central Terminal
foreach( $stopNames as $stop )
$stopSelectOptions .= sprintf( '<option %s value="%d">%s</option>', ( DEFAULT_STOP == $stop ) ?  'selected="selected"' : '' , $stopsData[ $stop ], $stop ) . "\n\r";
}
fclose( $stopsFile );


// load ads file 
$adsFile = fopen( 'ads.txt', "r" );

if ( $adsFile )
{
// get (and toss) header row
$adsHeader = fgetcsv( $adsFile, 1000 );

// will hold ads available for stops
$ads = array();

// build array from file data
while ( $data = fgetcsv( $adsFile, 1000 ) )
{
$ads[ $data[0] ] = array
(
'filename' => $data[1],
'url'      => $data[2],
'text'     => $data[3]
);
}
fclose( $adsFile );
}


// printout the html header
require_once( 'header.php' );


if ( 'POST' == $_SERVER[ 'REQUEST_METHOD' ] ) 
{
$from_stop = $_POST[ 'FromStation' ];  // @todo Filter! http://www.php.net/manual/en/filter.filters.validate.php
$orig_station = $from_stop;

$to_stop = $_POST[ 'ToStation' ];  // @todo Filter! http://www.php.net/manual/en/filter.filters.validate.php
$dest_station = $to_stop;

if (  $to_stop == $from_stop ) 
{
$error = 'Originating and Destination stops are the same.';
}
else
{

print_ad( $orig_station, $dest_station, $location='top' );

$travel_date = $_POST[ 'RequestDate' ];
$requestTime = $_POST[ 'RequestTime' ];
$am_pm       = $_POST[ 'RequestAMPM' ];
$filter      = $_POST[ 'sortBy' ];

/*
* Lets post this to MTA.info
*/

$mta_curl = curl_init();

$mta_curl_options = array
(
CURLOPT_FAILONERROR    => true,
CURLOPT_FOLLOWLOCATION => false,
// CURLOPT_MUTE           => true,
CURLOPT_POST           => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_CONNECTTIMEOUT => 30,
CURLOPT_TIMEOUT        => 30,
CURLOPT_URL            => $mta_post_url,
CURLOPT_REFERER        => 'http://lirr42.mta.info/',
CURLOPT_USERAGENT      => 'MTA Info Scrape/1.0',
CURLOPT_POSTFIELDS     => http_build_query( $_POST )
);
curl_setopt_array( $mta_curl, $mta_curl_options );
$mta_response = curl_exec( $mta_curl );

if ( false === $mta_response )
{
curl_close( $mta_curl );
die( sprintf( 'Response Code:%s, Curl Error No:%s, Curl Error Message:%s', curl_getinfo( $mta_curl, CURLINFO_HTTP_CODE ),curl_errno( $mta_curl ), curl_error( $mta_curl ) ) );
}
else
{
curl_close( $mta_curl );
echo '<pre>' . htmlentities( $mta_response ) . '</pre>';

$html = new simple_html_dom();
$html->load( $mta_response );
$schedule_table = $html->find( 'table', 0 );  // find second table in the response
$row_count = 0;

// will hold HTML output for table
$stopSchedule = '<table><th>Departs</th><th>Arrives</th><th>Minutes</th><th>Transfer</th><th>Fare</th></tr>';

foreach ( $schedule_table->find('tr') as $schedule_row ) 
{
$row_count++;
// check for and skip header row
if ( 1 == $row_count )
continue;

// check for and skip last row which first TD has rowspan attribute
if ( $schedule_row->children(0)->colspan )
{
continue;
}
// extract table data for this row
$depart_time      = $schedule_row->children( 0 )->innertext;
$arrive_time      = $schedule_row->children( 2 )->innertext;
$minutes_traveled = $schedule_row->children( 4 )->innertext;
$transfer         = $schedule_row->children( 5 )->innertext;
$fare             = $schedule_row->children( 6 )->plaintext; 
// add table row to html output
$stopSchedule .= sprintf( '<tr><td>%s</td><td>%s</td><td>%s</td><td>%s</td><td>%s</td></tr>', $depart_time, $arrive_time, $minutes_traveled, $transfer, $fare ) . "\n\r";
}  // end foreach schedule_row

// add table end tag to html output
$stopSchedule .= '</table>';

// output table to browser
echo $stopSchedule;

print_ad( $orig_station, $dest_station, $location='bottom' );

} // end if mta response

} // end if not same orig and dest 

} // end if POST


if ( 'GET' == $_SERVER[ 'REQUEST_METHOD' ] || ! empty( $error ) ) 
{

print_ad( null, null, $location = 'default' );

?>

<form method="post">

<?php

if ( ! empty ( $error ) )
{
echo sprintf( '<div class="error">%s</div>', $error );
}

?>

From:<br />
<select id="orig_station" name="FromStation" tabindex="2">
<?php echo $stopSelectOptions; ?>
</select>

<br />
<br />

To:<br />
<select id="dest_station" name="ToStation" tabindex="3">
<?php echo $stopSelectOptions; ?>
</select>

<br />
<br />

<input id="date1" name="RequestDate" size="12" maxlength="10" tabindex="4" value="<?php echo date( 'm/d/Y' ); ?>" />

<?php 
$currentHour = trim( date( 'h' ) );
$currentMinutes = trim( date( 'i' ) );
$currentMinutesFloor =  sprintf( '%02d' , $currentMinutes - ( $currentMinutes % 30 ) );
$currentAMPM = trim ( date( 'A' ) );

$start_time = strtotime( '1:00 AM' );
$end_time = strtotime( '1:00 PM' );
?>

<select name="RequestTime" tabindex="5">
<?php
while( $start_time < $end_time )
{
$option_time = date( 'h:i', $start_time );
$option_hour = trim( date( 'h', $start_time ) );

$option_minutes = trim( date( 'i', $start_time ) );
$option_selected = (  $option_hour == $currentHour && $option_minutes  == $currentMinutesFloor );

echo sprintf( '<option %s>%s</option>', ( $option_selected ) ? 'selected="selected"' : '',  $option_time ) . "\r\n";

$start_time = $start_time + 30*60;
}
?>
</select>

<select name="RequestAMPM" tabindex="6">

<option value="AM" <?php echo ( 'AM' == $currentAMPM ) ? 'selected="SELECTED"' : ''; ?>>AM</option>
<option value="PM" <?php echo ( 'PM' == $currentAMPM ) ? 'selected="SELECTED"' : ''; ?>>PM</option>
</select>

<input name="sortBy" type="hidden" value="1" />
<input type="submit" name="submit" value="Check Schedule" />
</form>


<!--  images/base/
ic_menu_back.png
ic_menu_home.png

-->
<?php 

}

require_once( 'footer.php' );

?>

I have json code  and i want to prine only {"currency":"DOGE","available":"0","reserved":"420.000000000"} data.

my json data

[{"currency":"1ST","available":"0","reserved":"0"},{"currency":"DOGE","available":"0","reserved":"420.000000000"},{"currency":"ZSC","available":"0","reserved":"0"}]    

I use this code

 <?php
  $ch = curl_init('http://arbitrage.adquash.com/balance.json'); 
 curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('accept: application/json'));
 $return = curl_exec($ch); 
  curl_close($ch); 
$result=json_decode($return);
echo "<pre>";
foreach($result as $row){
  echo $row->$a['currency']['available']['reserved'];

?>

First of all... this is a GREAT forum. Been lurking around here for a long time and searched all over but still can't find a solution to a problem I'm facing.

Here's my cURL code - I am basically retrieving data from a web site:
<?php

$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_URL,'http://myprimemobile.com/update.html');
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);

if (empty($buffer))
{
    print "Connection Timeout<p>";
}
else
{
    print $buffer;
}
?>

If you take a look at the site http://myprimemobile.com/update.html -- it will begin to update new information. I want to basically retrieve the data from that page but in my own way.

I want to loop through the page and search for all the names, addresses, and phone numbers only (not store hours or anything else), and want to put it in a variable so I can create a table with all that information on my page.

Based on the code, all the information is stored in the $buffer variable. I want to break this variable down to extract certain data in a 'div' or a 'table'. Please help me out. Thanks!

I'm using cURL to crawl and scrape data from a website. This website contains tables with rows of data. When I send a cURL POST for the underlying data at a specific row(A), it will return the expected data. But when I move to the second row(B), the data returns blank or specifically, a tons of spaces (or &#nbsp's.) When I access the cURL's POST location by browser, I can see (B)'s data. The only difference in the 2 POST's are location ID's for the data. I don't think it's a problem with JavaScript as I can successfully return data from row (A) as I mentioned.

OK, I have the initial cURL working but need to figure out how to extract data I want off that webpage to display or store in a database, I tried using dom and xpath, but because of the way the page displays using css, i think its not picking it up. Here is my cURL script:
<?php
$userAgent = 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';

$target_url = "www.test.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$html = curl_exec($ch);
if (!$html) {
echo "<br />cURL error number:" .curl_errno($ch);
echo "<br />cURL error:" . curl_error($ch);
exit;
}
// parse the html into a DOMDocument
$dom = new DOMDocument();
$dom->loadHTML($html);

// grab all the on the page
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//td");

for ($i = 0; $i < $hrefs->length; $i++) {
$href = $hrefs->item($i);
$url = $href->getAttribute('href');
storeLink($url,$target_url);
echo "<br />Link stored: $url";
}

?>
and here is a snippet of the source of the page I am getting:

<span id="lblTest"><h1 id='surrZipTitle'>Agents in Surrounding Zip Codes</h1><table cellpadding='0' cellspacing='0' border='0' class='tblDent'><tr><td class='tdEliteTitle'><span class='caaSubHead3 addwidth'>H.K. Dent Elite</span></td></tr><tr><td class='tdEliteContent'><table cellpadding='0' cellspacing='0' border='0'><tr><td valign='top'><span class='caaAgencyName2 addwidth'>PROFESSIONAL INS ASSOC, INC.</span></td><td valign='top'>&nbsp;</td></tr></table><table cellpadding='0' cellspacing='0' border='0'><tr><td width='360px' valign='top'><div class='addressBlock'><span>4444 MANZANITA AVE STE 6</span><br /><span>CARMICHAEL                    , CA 95608-1488</span><br /><a class='faaBlueLink' id='lnkContact' href='http://www.safeco.com/portal/server.pt/gateway/PTARGS_0_20656_395_362_0_43/http%3B/por-portlets-prd.int.apps.safeco.com%3B13425/dotcom/FindAnAgent/find-an-agent/contactanagent.aspx?RequestType=agency&level=elite&Id=0415199904150295&lat=38.646142&lng=-121.327623' onclick='oOobj4.Preferences.Plugins.Events.poX=0;'>Contact &amp; Directions</a> &nbsp;&nbsp; <a class='faaBlueLink' id='lnkWebSite' style='display: none;' href='http://' target='_blank' onclick="return trackEvent('/External-Link/AgentWebsite/                                                                      ','PROFESSIONAL INS ASSOC, INC.                                ');">Website</a></div></td><td valign='top'>&nbsp;</td></tr></table></td></tr></table><table cellpadding='0' cellspacing='0' border='0' class='tblDent'><tr><td class='tdEliteTitle'><span class='caaSubHead3 addwidth'>H.K. Dent Elite</span></td></tr><tr><td class='tdEliteContent'><table cellpadding='0' cellspacing='0' border='0'><tr><td valign='top'><span class='caaAgencyName2 addwidth'>AMERICAN AIM AUTO INS AGY, INC</span></td><td valign='top'>&nbsp;</td></tr></table><table cellpadding='0' cellspacing='0' border='0'><tr><td width='360px' valign='top'><div class='addressBlock'><span>5339 SAN JUAN AVE</span><br /><span>FAIR OAKS                     , CA 95628-3318</span><br /><a class='faaBlueLink' id='lnkContact' href='http://www.safeco.com/portal/server.pt/gateway/PTARGS_0_20656_395_362_0_43/http%3B/por-portlets-prd.int.apps.safeco.com%3B13425/dotcom/FindAnAgent/find-an-agent/contactanagent.aspx?RequestType=agency&level=elite&Id=0415911704151222&lat=38.66237&lng=-121.292429' onclick='oOobj4.Preferences.Plugins.Events.poX=0;'>Contact &amp; Directions</a>

So basically I want to extract the agency name like "<span class='caaAgencyName2 addwidth'>PROFESSIONAL INS ASSOC, INC.</span>" and the address which always use the same div class like "caaAgencyName2" and "addressBlock".  How can this be accomplished?