|
PHP - Best Way To Parse Webpage Using Curl
Hi Guys,
I was wondering what the best way to parse the html from a webpage recovered by curl? $go = curl_exec($c); kinda thing, any advice would be great cheers Graham Similar TutorialsHey guys I have the following code and am trying to get the body of the webpage, however it is not currently working and the array at the end is empty. Any help appreciated!!!!! Code: <?php $word = $_GET['word']; function get_web_page( $url,$curl_data ) { $options = array( CURLOPT_RETURNTRANSFER => true, // return web page CURLOPT_HEADER => false, // don't return headers CURLOPT_ENCODING => "", // handle all encodings CURLOPT_USERAGENT => "spider", // who am i CURLOPT_AUTOREFERER => true, // set referer on redirect CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect CURLOPT_TIMEOUT => 120, // timeout on response CURLOPT_MAXREDIRS => 10, // stop after 10 redirects CURLOPT_POST => 1, // i am sending post data CURLOPT_POSTFIELDS => $curl_data, // this are my post vars CURLOPT_SSL_VERIFYHOST => 0, // don't verify ssl CURLOPT_SSL_VERIFYPEER => false, // CURLOPT_VERBOSE => 1 // ); $ch = curl_init($url); curl_setopt_array($ch,$options); $content = curl_exec($ch); $err = curl_errno($ch); $errmsg = curl_error($ch) ; $header = curl_getinfo($ch); curl_close($ch); // $header['errno'] = $err; // $header['errmsg'] = $errmsg; //$header['content'] = $content; return $content; } $curl_data = "?tranword=".$word; $url = "http://www.wordreference.com/es/translation.asp?tranword=".$word; $response = get_web_page($url,$curl_data); preg_match('~<body>(.*)</body>~', $response, $output); print_r($output); ?> Thanks lots, Jake good day dear community, i am workin on a Curl loop to fetch multiple pages: i have some examples - and a question: Example: If we want to get information from 3 sites with CURL we can do it like so: $list[1] = "http://www.example1.com"; $list[2] = "ftp://example.com"; $list[3] = "http://www.example2.com"; After creating the list of links we should initialize the cURL multi handle and adding the cURL handles. $curlHandle = curl_multi_init(); for ($i = 1;$i <= 3; $i++) $curl[$i] = addHandle($curlHandle,$list[$i]); Now we should execute the cURL multi handle retrive the content from the sub handles that we added to the cURL multi handle. ExecHandle($curlHandle); for ($i = 1;$i <= 3; $i++) { $text[$i] = curl_multi_getcontent ($curl[$i]); echo $text[$i]; } In the end we should release the handles from the cURL multi handle by calling curl_multi_remove_handle and close the cURL multi handle! If we want to another Fetch of sites with cURL-Multi - since this is the most pretty way to do it! Well I am not sure bout the string concatenation. How to do it - Note I want to fetch several hundred pages: see the some details for this target-server sites - /(I have to create a loop over several hundred sites). * siteone.example/?show_subsite=9009 * siteone.example/?show_subsite=9742 * siteone.example/?show_subsite=9871 .... and so on and so forth Question: How to appy this loop into the array of the curl-multi? <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "siteone", "sitetwo", "sitethree" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> I look forward to your ideas.
try {
echo "<br>";
foreach($dbh->query("SELECT * FROM test_shot WHERE sold=1 ORDER BY year ASC") as $row) {
if($row['picture'] != "" && $row['picture'] != null)
{
echo "<div class='image-holder'><img src ='".$row['picture']."' width=300px /><br>";
}
if($row['year'] != "" && $row['year'] != null)
{
echo $row['year'];
}
if($row['description'] != "" && $row['description'] != null)
{
echo $row['description'];
}
if($row['sold'] == 1) {
echo "<img src='images/sold1.png'><br>";//Add your image code here
}
elseif ($row['sold'] == 0) {
echo "</div><br>";
}
}
}
catch (PDOException $e) {
print $e->getMessage();
}
?>
hi all am new to this forum help me to overcome from this error Parse error: parse error, expecting `T_STRING' or `T_VARIABLE' or `T_NUM_STRING' in D:\wamp\www\quiz1\quiz1.php on line 12 Code: [Select] <?php include("contentdb.php"); $display = mysql_query("SELECT * FROM $table ORDER BY id",$db); if (!$submit) { echo "<form method=post action=$_SERVER['PHP_SELF']>"; echo "<table border=0>"; while ($row = mysql_fetch_array($display)) { $id = $row["id"]; $question = $row["question"]; $opt1 = $row["opt1"]; $opt2 = $row["opt2"]; $opt3 = $row["opt3"]; $answer = $row["answer"]; echo "<tr><td colspan=3><br><b>$question</b></td></tr>"; echo "<tr><td>$opt1 <input type=radio name=q$id value=\"$opt1\"></td><td>$opt2 <input type=radio name=q$id value=\"$opt2\"></td><td>$opt3 <input type=radio name=q$id value=\"$opt3\"></td></tr>"; } echo "</table>"; echo "<input type='submit' value='See how you did' name='submit'>"; echo "</form>"; } elseif ($submit) { $score = 0; $total = mysql_num_rows($display); while ($result = mysql_fetch_array($display)) { $answer = $result["answer"]; $q = $result["q"]; if ($$q == $answer) { $score++; } } echo "<p align=center><b>You scored $score out of $total</b></p>"; echo "<p>"; if ($score == $total) { echo "Congratulations! You got every question right!"; } elseif ($score/$total < 0.34) { echo "Oh dear. Not the best score, but don't worry, it's only a quiz."; } elseif ($score/$total > 0.67) { echo "Well done! You certainly know your stuff."; } else { echo "Not bad - but there were a few that caught you out!"; } echo "</p>"; echo "<p>Here are the answers:"; echo "<table border=0>"; $display = mysql_query("SELECT * FROM $table ORDER BY id",$db); while ($row = mysql_fetch_array($display)) { $question = $row["question"]; $answer = $row["answer"]; $q = $row["q"]; echo "<tr><td><br>$question</td></tr>"; if ($$q == $answer) { echo "<tr><td>»you answered ${$q}, which is correct</td></tr>"; } elseif ($$q == "") { echo "<tr><td>»you didn't select an answer. The answer is $answer</td></tr>"; } else { echo "<tr><td>»you answered ${$q}. The answer is $answer</td></tr>"; } } echo "</table></p>"; } ?> thanks in adavance Hello I have one problem with fwrite() I have one script to get width and height from javascript and echo it with PHP. echo "Screen width is: ". $_GET['width'] ."<br />\n"; echo "Screen height is: ". $_GET['height'] ."<br />\n"; It works but i want to store the result in a file fwrite($info,"Height: $_GET['height'] <br />"); But then I get error Parse error: parse error, expecting `T_STRING' or `T_VARIABLE' or `T_NUM_STRING //////////////////////// it would be even better if I don't have to print the values but directly store them in $iinfo sorry if something similar was soved but those examples were different or i was unable to transform it to solve my problem I am trying to put a href on a line to link me to another php module, but get an error: Parse error: parse error in C:\wamp\www\editpolicy.php on line 47 My code: while ($row2 = mysql_fetch_array($result2) ) { print "<tr><td>"; <a href='editpayment.php?payid=$row2[PaymentID]'>$row2[PaymentID]</a> .........THIS IS LINE 47 ??? print "</td><td>"; print $row2['Actioned']; print "</td><td align='right'>"; print $row2['PremiumPaidAmount']; print "</td><td align='right'>"; print $row2['Risk']; print "</td></tr>"; } echo "</select></td>\r\n </tr>\r\n \r\n <tr>\r\n <td align=\"right\">Total Views Allowed:</td>\r\n <td><input type=\"text\" id=\"ad_total_views\" style=\"width:50px;\" value=\"\" disabled=\"disabled\" /> <label><input type=\"checkbox\" checked=\"checked\" onclick=\"if(this.checked==true){$('#ad_total_views').val('');$('#ad_total_views').attr('disabled','disabled');}else{$('#ad_total_views').val('');$('#ad_total_views').removeAttr('disabled');$('#ad_total_views').focus()}\" />Unlimited</label></td>\r\n </tr>\r\n \r\n <tr>\r\n <td align=\"right\">Total Clicks Allowed:</td>\r\n <td><input type=\"text\" id=\"ad_total_clicks\" style=\"width:50px;\" value=\"\" disabled=\"disabled\" /> <label><input type=\"checkbox\" checked=\"checked\" onclick=\"if(this.checked==true){$('#ad_total_clicks').val('');$('#ad_total_clicks').attr('disabled','disabled');}else{$('#ad_total_clicks').val('');$('#ad_total_clicks').removeAttr('disabled');$('#ad_total_clicks').focus()}\" />Unlimited</label></td>\r\n </tr>\r\n </table></td>\r\n </tr>\r\n <tr bgcolor=\"#FFFFFF\">\r\n <td class=\"td_th\" align=\"center\">Smarty Code (Developer)</td>\r\n <td>{\$ads->getAdCode(<span id=\"smartycode\">1</span>)}</td>\r\n </tr>\r\n <tr bgcolor=\"#FFFFFF\">\r\n <td class=\"td_th\" align=\"center\"> </td>\r\n <td><input type=\"button\" value=\"Create New Campaign\" onclick=\"new_ad()\" /></td>\r\n </tr>\r\n </TBODY></TABLE>\r\n<br />\r\n\r\n<TABLE cellSpacing=1 cellPadding=4 width=\"100%\" border=0>\r\n <TBODY>\r\n <TR class=\"td_title\">\r\n <TD colSpan=7>Ad Campaigns</TD></TR>\r\n <TR bgColor=#ffffff>\r\n \r\n <TD width=\"10%\" align=\"center\" class=\"td_th\"> </TD>\r\n <TD width=\"4%\" align=\"center\" class=\"td_th\">ID</TD>\r\n <TD width=\"29%\" align=\"center\" class=\"td_th\">Campaign Name</TD>\r\n <TD width=\"12%\" align=\"center\" class=\"td_th\">Start Date</TD>\r\n <TD width=\"11%\" align=\"center\" class=\"td_th\">End Date</TD>\r\n <TD width=\"17%\" align=\"center\" class=\"td_th\">Viewed / Views Allowed</TD>\r\n <TD width=\"17%\" align=\"center\" class=\"td_th\">Clicked / Clicks Allowed</TD>\r\n </TR>\r\n "; $BoxSize = array("smallbox" = array("length" => 12, "width" => 10, "depth" => 2.5), "mediumbox" = array("length" => 30, "width" => 20, "depth" => 4), "largebox" = array("length" => 60, "width" => 40, "depth" => 11.5)); Code: [Select] This is a clip of my PHP file: <?php $con = mysql_connect($host,$username,$password); if (!$con) { die('Could not connect: ' . mysql_error()); } mysql_select_db($database, $con); $result = mysql_query("SELECT * FROM purchase_order WHERE purchase_order_number=='$purchase_order_number'"); while($row = mysql_fetch_array($result)) { $vendor=$row['vendor']; $purchase_order_date=$row['purchase_order_date']; $ship=$row['ship']; $state=$row['state']; $fob=$row['fob']; $terms=$row['terms']; $buyer=$row['buyer']; $freight=$row['freight']; $req_date=$row['req_date']; $confirming_to=$row['confirming_to']; $remarks=$row['remarks']; $tax=$row['tax'] } // [i][u][b]<-- This is line 88 (The one with the error)[/b][/u][/i] ?>Thanks in advance! Hi. I found this code for my website, and it work well. Its just that there comes some text I wan't to delete. Code: Code: [Select] <?php /* index.php */ Session_start(); if($_SESSION['login'] == false) { header("Location:login.php"); } $a = $_GET['a']; $source='http://****.elementfx.com/test.php'; //$source='sample.txt'; $page_all = file_get_contents($source); $div_array=array(); preg_match_all('#<div id="intro">(.*?)</div>#sim', $page_all, $div_array); //print_r($div_array); ?> <html> <head> <title>Home</title> </head> <body> <center> <p><b><font color="blue" size="20">*****</font></b> <font color="blue" size="2">version 0.9_01</font></p> <br/> <br/> <br/> <textarea cols="50" rows="10"><?php print_r($div_array[1]);?></textarea> </center> </body> </html> The text it should get is: Quote Hello I'm Something! <p>asdoasduiasdasnda</p> asdasdaksdjas<br/> sdffdsg But the output is: Quote Array ( => Hello I'm Something! <p>asdoasduiasdasnda</p> asdasdaksdjas<br/> sdffdsg ) I need to get rid of the Array( ... thing.. Regards Worqy I have a large list of websites that have allowed me to submit my website and description every 24 hours to there sites. I'm really getting tired of sitting by my computer and submitting the information to every site manually. I never understood curl and php dom stuff. does anyone have a easy code where I can post a pre scripted text of my information into three boxes and it auto submits every 24 hours to the websites? The boxes look like this Code: [Select] <input maxlength="255" value="http://" name="sPlugurl" size="20" style="font-family: Verdana; color: FE9999; font-size: 10pt; background-color: 656565; font-weight: none; border: 1px solid #0C0C0C" type="text"> <input maxlength="30" value="Sitename" name="sPlugname" size="20" style="font-family: Verdana; color: FE9999; font-size: 10pt; background-color: 656565; font-weight: none; border: 1px solid #0C0C0C" type="text"> <input maxlength="255" value="Description" name="sPlugdescription" size="20" style="font-family: Verdana; color: FE9999; font-size: 10pt; background-color: 656565; font-weight: none; border: 1px solid #0C0C0C" type="text"> <input type="submit" value="Dump" style="background-color: #656565; font-family: Verdana; color: <? echo ;?>; font-size: 10pt; border: 1px solid #0C0C0C"> the php script would auto fill out the value of the first 3 boxes with my info and then the program would submit it to there forum. I looked on google but nothing I found I can figure out? I want to grab the title of a cl page using curl, can anyone help me with this?
I have the urls stored in a table I just need can't get it to read the title of the post in CL, let me know if you can help.
how to pass data on other domain in PHp or curl ?? and also how to get data on other domain ?? if anyone knows how to do it please attach code hy all long time no see recently i did find this problem i'm trying to login to a webpage with curl ,but without success ,ofcourse the basic posting is not the problem xD but the page have a hidden field (for security) so everytime you load the page it changes so post wont work if you first use get to take the page and extract the hash from it. the page contains something like input type= hidden value=A RANDOM VALUE THAT CHANGES EVERYTIME without that value passed to post it wont login. anyone knows how can i pass it so the post will be valid? thank you I have just installed cURL and am learn to use it but I'm not sure exactly what I am doing. For example I have a page with this code on it. if($_GET['number'] == "6") { echo("<p>the number is six</p>"); } Then I am trying to use cURL to send the number 6 to this page so that it returns "the number is 6". However I am just getting an output of "1". Below is the code I am using to try to get the output of "the number is 6". $ch2 = curl_init(); curl_setopt($ch2,CURLOPT_URL,'http://localhost/crawler/crawlertest.php'); curl_setopt($ch2, CURLOPT_POST, 1); curl_setopt($ch2,CURLOPT_POSTFIELDS,'number=6'); $return = curl_exec($ch2); echo ($return); curl_close($ch2); Thanks for any help I am using the following code. Code: [Select] $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); $response = curl_exec($ch); curl_close($ch); I was given a .cer file should I set CURLOPT_SSL_VERIFYPEER to TRUE and set the path to the .cer file? Any help would be awesome. Thanks Hey guys I'm trying to log into this website using curl. I did some reading and think I'm pretty close but I cant get the login working It just returns nothing. there is a commented out part which is the page i'm trying to load after I successfully login.
<?
$username = 'xxxxxxx';
$password = 'xxxxxxx';
$postinfo = "username=".$username."&password=".$password;
$cookie="cookie.txt";
$ch = curl_init();
// extra headers
$headers[] = "Accept: */*";
$headers[] = "Connection: Keep-Alive";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_URL, 'https://weblogin.asu.edu/cas/login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
$data = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
echo $data;
curl_close($ch);
// //page with the content I want to grab after logging in (test after login works)
// curl_setopt($ch, CURLOPT_URL, "https://webapp4.asu.edu/myasu/");
// curl_setopt($ch, CURLOPT_POST, false);
// $data = curl_exec($ch);
// if (curl_error($ch)) {
// echo curl_error($ch);
// }
?>
I'm not sure but I think im maybe missing some other parts the form needs? How do I see exactly what I need to send?
Hello, I'm new to lot of this and especially php. I'm having hard time with one of my app that I created. If you go to ticketlawyerorlando.com/sms5.html and you try to send text, (which is set to go to only one number (mine)) than I get error code from php that is supposed to curl it. Error: Warning: curl_setopt() [function.curl-setopt]: Invalid curl configuration option in /home/content/s/j/1/sj10mil/html/ticketlawyerorlando/sms5.php on line 10 SUCCESS My Html code is : Code: [Select] <html> <body> <br/> <form action="sms5.php" method="post"> <b>Your Phone Number:</b> <input type="text" name="to" /> <br/> <b>Message:</b> <input type="text" name="msg" /> <br/> <input type="submit" /> </form> </body> </html> My sms5.php code is : Code: [Select] <html> <head> <title>SMS STATUS</title> </head> <body> <?php //print $to; //print $msg; $curl_handle=curl_init(); curl_setopt($curl_handle, CURLOPT_GET,1); curl_setopt($curl_handle, CURLOPT_URL,'https://api.tropo.com/1.0/sessions?to=3213314633&msg=test4&token=010fc32265db5444bbf84e92bd28f1887395ce3ade6cf69e1424d586add66b17bdc7c1d8061803f1c2319d66'); curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,5); curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,2); $response = curl_exec($curl_handle); $code = curl_getinfo($curl_handle,CURLINFO_HTTP_CODE); //print $code; //print $response; print "SUCCESS"; curl_close($curl_handle); ?> </body> </html> In my php, the URL that's being curled, in the part where it says "test4" that should be actual message that is typed in the form but I don't know what kind of parameter to insert there in order for it to send message that's written, it always sends message "test4". Please, if you could help me I would greatly appreciate it. Thank You so much again. MOD EDIT: code tags added Hi, I have server which is posting me these data through curl. $strPost = "<?xml version=\"1.0\" encoding=\"utf-8\"?>"; $strPost .= "<request>"; $strPost .= "<type>login</type>"; $strPost .= "<session>21</session>"; $strPost .= "</request>"; $url = "http://www.abc.com/xyz.php"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_HTTPGET,1); curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Content-Type: text/xml")); curl_setopt($ch, CURLOPT_HEADER, FALSE); curl_setopt($ch, CURLOPT_POSTFIELDS, $strPost); // add POST fields curl_setopt($ch, CURLOPT_POST, 1); $result = curl_exec($ch); if($result == true) { echo $result; } curl_close($ch); I wanna read the above xml send i.e $strPost at http://www.abc.com/xyz.php i m trying to read it with $_POST['strPost'] but shows empty. Can anyone help me. |
|||||||