PHP - Curl Loop To Fetch Multiple Pages - Running Curl-multi
good day dear community,
i am workin on a Curl loop to fetch multiple pages: i have some examples - and a question: Example: If we want to get information from 3 sites with CURL we can do it like so: $list[1] = "http://www.example1.com"; $list[2] = "ftp://example.com"; $list[3] = "http://www.example2.com"; After creating the list of links we should initialize the cURL multi handle and adding the cURL handles. $curlHandle = curl_multi_init(); for ($i = 1;$i <= 3; $i++) $curl[$i] = addHandle($curlHandle,$list[$i]); Now we should execute the cURL multi handle retrive the content from the sub handles that we added to the cURL multi handle. ExecHandle($curlHandle); for ($i = 1;$i <= 3; $i++) { $text[$i] = curl_multi_getcontent ($curl[$i]); echo $text[$i]; } In the end we should release the handles from the cURL multi handle by calling curl_multi_remove_handle and close the cURL multi handle! If we want to another Fetch of sites with cURL-Multi - since this is the most pretty way to do it! Well I am not sure bout the string concatenation. How to do it - Note I want to fetch several hundred pages: see the some details for this target-server sites - /(I have to create a loop over several hundred sites). * siteone.example/?show_subsite=9009 * siteone.example/?show_subsite=9742 * siteone.example/?show_subsite=9871 .... and so on and so forth Question: How to appy this loop into the array of the curl-multi? <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "siteone", "sitetwo", "sitethree" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> I look forward to your ideas. Similar TutorialsHi - good evening! thanks for the answer and all the help so far Oncemore the "Fetching-part" for a little parser-script: Here we go - new target urls: see the overview: http://dms-schule.bildung.hessen.de/index.html http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html Search by pressing the button "type" and then choose all schools with the mouse! Results 2400 schools: Here i can provide some "more help for getting the target!" - btw: see some details for this target-server: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9009 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9742 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9871 well - you see i have to iterate over the sites - with a function /(a loop) http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=1000 to 10000 BTW - after fetching the page i have to see which one are empty - those ones do not need to be parsed! Well - i want to do this with curl-multi since this is the most advanced way to do this: I see i have an array that can be filled I have to try it out...!! Well, I'm not pretty sure - but judging by the low level of PHP i have: I think that i may need a double array to define the URLs like: Can i do like so?: $urls = array( "siteone" => "suche_schul_db.html?show_school=%i", "sitetwo" => "suche_schul_db.html?show_school=%i", "sitethree" =>"suche_schul_db.html?show_school=%i" ); $params = array ( for ($i = 1; $i <= 10000; $i++) { // body of loop } // well i have to define the variables in a open form like abvove nand not below: // better than doing it like so: "siteone" => array(9009, 9742, 9871), ); then pass them to curl-multi as: foreach ($urls as $id => $url) { foreach ($params[$id] as => $param) { $i = $id . $param; $finalurl = sprintf($url, $param); $conn[$i] = curl_init($finalurl); I don't know if that's fits the needs tip that i am looking for but I hope so ;-) Can i do like so [see above]? What do yu think. i send you many greetings martin Hello PHP Freaks! I have the following PHP Curl code which is supposed to fetch the twitter timeline feed for a specific username and parse and display it on a page, but for some reason Curl is unable to fetch data from twitter: Code: [Select] <?php function get_data($url) { $ch = curl_init(); $timeout = 5; curl_setopt($ch,CURLOPT_URL,$url); curl_setopt($ch,CURLOPT_RETURNTRANSFER,1); curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout); $data = curl_exec($ch); curl_close($ch); return $data; } $json = get_data("https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=digioz&count=2"); if ($json != false) { $obj = json_decode($json); foreach($obj as $var => $value) { echo "Message number: $var <br/>"; echo "Name: " . $obj[$var]->user->name; echo "Handle: " . $obj[$var]->user->screen_name . "<br/>"; echo "Message: " . $obj[$var]->text; echo "Created" . $obj[$var]->created_at . "<br/>"; echo "URL" . $obj[$var]->user->url . "<br/>"; echo "Location" . $obj[$var]->user->location . "<br/>"; echo "<br/>"; } } else { echo "Could not fetch Twitter Data"; } ?> Anyone have any idea why the data is not being fetched? If I copy and paste the URL into my browser window it returns results just fine, so I know the problem is not with the URL. Thanks, Pete Hi, I have this curl string required by an email verification vendor I use. Inside my php file I wish to run it. Apparently there is a problem with the syntax as I’m getting the following error in the browser: ”Parse error: syntax error, unexpected '--' (T_DEC) in /var/www/html/wp-content/plugins/dw_functionality_plugin/dw_functionality_plugin.phpon line 398 Here is the code string:
< > any help much appreciated! Edited March 23 by dwest100i want to do it with cURL-Multi since this is the most pretty way to do it! Well i am not sure bout the string concatenation. How to do it - Note i want to fetch several hundred pages: see the some details for this target-server sites - /(i have to create a loop over several hundred sites) siteone_dot_com?show_subsite=9009 siteone_dot_com?show_subsite=9742 siteone_dot_com?show_subsite=9871 and so on and so fort how to appy this loop into the array of the curl-multi <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "siteone", "sitetwo ", "sitethree" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> I have an array of proxies and I need one to be chosen randomly and applied to each curl handle. This script runs successfully when proxy support is removed, but as soon as I try to add a proxy it refuses to connect. My proxies are good. I have private proxies that validate by confirming that the IP address of the requester is on a whitelist of IPs I configured. I've also tried working public proxies with no validation, with the same result. The error it returns is that it "cannot connect to google.com:3238". Why it's trying to connect to that port is beyond me, the proxies I use, their port is 3238, but it should be connecting to IP:PORT then Google.com:80. Not sure why it's mixing them up. Here's my code: Code: [Select] function multi($urls,$proxies,$agents,$referer) { // create the multi curl handle $mh = curl_multi_init(); $handles = array(); $i = 0; foreach ($urls as $url) { // create a new single curl handle $ch = curl_init(); // setting several options like url, timeout, returntransfer // simulate multithreading by calling the wait.php script and sleeping for $rand seconds curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_PROXY, $proxies[rand(0,count($proxies))]); curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1); curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt'); curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt'); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30); curl_setopt($ch, CURLOPT_REFERER, $referer); curl_setopt($ch, CURLOPT_USERAGENT, $agent); // add this handle to the multi handle curl_multi_add_handle($mh,$ch); // put the handles in an array to loop this later on $handles[] = $ch; $i++; } // execute the multi handle $running=null; do { curl_multi_exec($mh,$running); // added a usleep for 0.25 seconds to reduce load usleep (250000); } while ($running > 0); echo 'Curl error: ' . curl_error($ch)."<br />".print_r(curl_getinfo($ch)); // get the content of the urls (if there is any) for($i=0;$i<count($handles);$i++) { // get the content of the handle $output[] = curl_multi_getcontent($handles[$i]); // remove the handle from the multi handle curl_multi_remove_handle($mh,$handles[$i]); } // close the multi curl handle to free system resources curl_multi_close($mh); // echo the output to the screen return $output; } I only post on forums as a last resort. Typically a few Google searches solves the problem, but I've spent too much time researching this one with no progress. I'm assuming it's simply a parameter that needs to be added to the CURL handles, but I've tried every proxy related config option out there and still no dice. Iv'e been looking in to methods of scraping data from pages and has found several examples of using multi-curl to achieve this. But i am not used to curl and is not completely sure how it works and i need to find the fastest reliable (i do need all, or close to all, pages every run) method of getting the content of a number of pages (about 160). Here is an example i got from searching the web which i managed to implement: <?php /** * *@param $picsArr Array [0]=> [url], *@$picsArr Array will filled with the image data , you can use the data as you want or just save it in the next step. **/ function getAllPics(&$picsArr){ if(count($picsArr)<=0) return false; $hArr = array();//handle array foreach($picsArr as $k=>$pic){ $h = curl_init(); curl_setopt($h,CURLOPT_URL,$pic['url']); curl_setopt($h,CURLOPT_HEADER,0); curl_setopt($h,CURLOPT_RETURNTRANSFER,1);//return the image value array_push($hArr,$h); } $mh = curl_multi_init(); foreach($hArr as $k => $h) curl_multi_add_handle($mh,$h); $running = null; do{ curl_multi_exec($mh,$running); }while($running > 0); // get the result and save it in the result ARRAY foreach($hArr as $k => $h){ $picsArr[$k]['data'] = curl_multi_getcontent($h); } //close all the connections foreach($hArr as $k => $h){ $info = curl_getinfo($h); preg_match("/^image\/(.*)$/",$info['content_type'],$matches); echo $tail = $matches[1]; curl_multi_remove_handle($mh,$h); } curl_multi_close($mh); return true; } ?> Since time is critical in my script i would ask if you think this is a good implementation or if you can point me in the direction of one that will save me noticeable run-time. Hey guys, Recently I bumped into a little coding problem. I'm trying to submit information to a multi-part form (including an image upload). The receiving server couldn't read my input, figured I had a nill object somewhere (RoR). First I thought the problem was in my curl setopts, but I think they are fine. It looks like the problem is with the data I'm submitting. I read a bit about multi part posting, and every article I read advised to make a postfield array, which curl would interpret and send out correctly. I did and this was my resulting array: $postfield = array ("social[comments]" => "Test", "avatar_file" => "", "friend_avatar_file" => "", "forum_avatar_file" => "", "social[signature]" => "", "forum_signature_file" => "", "save_profile.x" => "25", "save_profile.y" => "9"); That failed. So I tried making it into a URLENCODED string: $postfield = "social%5Bcomments%5D=".stripslashes($_POST['textarea'])."&avatar_file=&friend_avatar_file=&forum_avatar_file=".$file."&social%5Bsignature%5D=&forum_signature_file=&save_profile.x=25&save_profile.y=9"; This finally worked. I was able to submit the info I wanted. I'd like to send a file with it this time though. I'd need to put this into a postfield array to work though. if(!$file = upload( )) { $file = ''; } else { $file = '@'.$file; } function upload( ) { if(isset($_FILES['file']) && !empty($_FILES['file']['name'])) { $uploaddir = "temp"; if(is_uploaded_file($_FILES['file']['tmp_name'])) { move_uploaded_file($_FILES['file']['tmp_name'],$uploaddir.'/'.$_FILES['file']['name']); $searchfile = $uploaddir.'/'.$_FILES['file']['name']; return $searchfile; } } return false; } So, I got the uploading covered. I need to put generate the postfield array though. I have the feeling the problem is with the brackets [ ] I'm using. I tried urlencoding the keys in the array, but that didn't work either. Does anybody have an idea on how to handle brackets in post (multi-part) data? Thanks in advance! hello dear php-friends i currently work on a little parser project i have to find solutions for the a. fetching part b. parser part here we go - the target urls: see the overview: http://dms-schule.bildung.hessen.de/index.html http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html Search by pressing the button "type" and then choose all schools with the mouse! Results 2400 schools Here i can provide some "more help for getting the target!" - btw: see some details for this target-server: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9009 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9742 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9871 well - you see i have to itterate over the sites - with a function /(a loop) http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=1000 to 10000 BTW - after fetching the page i have to see which one are empty - those ones do not need to be parsed! Well - i want to do this with curl-multi since this is the most advanced way to do this: I see i have an array that can be filled -... but i have to think about the string-concatenation - i guess that i have make some sophisticated string concatenation. this one does not fit - for($i=1;$i<=$match[1];$i++) { $url = "http://www.example.com/page?page={$i}"; and besides this i have an array - i c an fill the array. can you help me how to run in a loop with <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "http://www.google.com/", "http://www.altavista.com/", "http://www.yahoo.com/" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> i am new to PHP - and i want to learn some thing bout PHP - currently i have a little project - in order to get the links visited that this site presents http://www.educa.ch/dyn/79363.asp?action=search [search with wildcard % ] i parse with a loop. <?php $data = file_get_contents('http://www.educa.ch/dyn/79363.asp?action=search'); $regex = '/Page 1 of (.+?) results/'; preg_match($regex,$data,$match); var_dump($match); echo $match[1]; ?> in order to get the following pages http://www.educa.ch/dyn/79376.asp?id=4438 http://www.educa.ch/dyn/79376.asp?id=2939 If we are looping over a set of values, then we need to supply it as an array. I would guess something like this. As i am not sure which numbers which are filled with content - i therefore have to loop from 1 to 10000. So i make sure that i get all data. What do you think!? for ($i = 1; $i <= 10000; $i++) { // body of loop } according the following description: http://www.php.net/manual/en/control-structures.for.php I have never used cURL before but I need to use it for developing a facebook application. What I am trying to do is create a status update scheduler. There are two tables in my database. One table has generic updates for each day of the week an the other had special events. My script checks the day, checks to see if there is a special event in the future on that day, and if there is reminds guests of the upcoming event. If there is no event it posts the generic status. Code: [Select] <?php include('config.php'); $result2 = mysql_query("select facebook_id,facebook_access_token from admin where username='users'"); $row = mysql_fetch_array($result2); $facebook_id = $row['facebook_id']; $facebook_access_token = $row['facebook_access_token']; $day = date('N'); $result2 = mysql_query("select * from events where date > CURDATE() order by date ASC") or die(mysql_error()); $num_rows = mysql_num_rows($result2); if($num_rows == 0) { $result2 = mysql_query("select * FROM alerts WHERE id='$day'") or die(mysql_error()); $row1 = mysql_fetch_array($result2); echo 'There are no events just alerts'; } else { while($row = mysql_fetch_array($result2)){ if(date('N',strtotime($row[date])) == $day) { $id = $row['id']; $result1 = mysql_query("SELECT * FROM events WHERE id = $id") or die(mysql_error()); $row1 = mysql_fetch_array($result1); } else { $result1 = mysql_query("SELECT * FROM alerts WHERE id = $day") or die(mysql_error()); $row1 = mysql_fetch_array($result1); } } } $params = array('access_token' => $facebook_access_token, 'message' => $row1[alert_message], 'name'=> $row1[alert_name], 'caption' => $row1[alert_caption], 'link' => $row1[alert_link], 'description' => $row1[alert_description], 'picture' => $row1[alert_picture] ); $url = "https://graph.facebook.com/$facebook_id/feed"; $ch = curl_init(); curl_setopt_array($ch, array( CURLOPT_URL => $url, CURLOPT_POSTFIELDS => $params, CURLOPT_RETURNTRANSFER => true, CURLOPT_SSL_VERIFYPEER => false, CURLOPT_VERBOSE => true )); $result = curl_exec($ch); ?> The problem seems to be occurring when the While loop is accessed. I have used 'echo' to follow the variables through the script and it all seems to be working. However if I try and execute the cURL and the While Loop has run then it does not seem to work. I know the cURL part works becuase if the variables cause the first IF statement default to ELSE then it runs fine. Any help would be GREATLY appreciated. When using cURL, how would I be able to include a call inside my get_all that basically will loop through all the next_page, get and store the data and then output it to $response->data when the "next_page" parameter becomes null? **Here is the method**: public function get_all() { return $response->data; } **This is what $response->data is returning as of now** (The cURL code wasn't included here): "paginator": { "total": 3092, "per_page": 500, "current_page": 2, "last_page": 7, "prev_page": "https://oc.com/api/v1/employees/?page=1", "next_page": "https://oc.com/api/v1/employees/?page=3" }, "data": [ { "id": 1592, etc.... Here are all of my unsuccessful attempts: public function get_all() { // $next_url = $response->paginator->next_page; // // foreach ($response as $next => $next_page) { // print_r2($next); // // if ($next_url !== null) { // $next_page = $response->data; // } // } // foreach ($response as $paginator => $next_page) { // if ($next_url !== null) { // $return[] = $response->data; // } // } // var_dump($response->paginator); // if ($next_url !== null) { // $this->get_all($path, $args, $next_url); // } return $response->data; } Edited October 30, 2019 by Sema314 I have this code and its downloading file nicely. But I want to show a progress bar while downloading file from external server. My existing code shows progress bar but its not effective (ex. if i try to download two media file, one video and another audio, and video size is larger than audio size the audio finishes first and the progress bar shows 100% also suddenly it drops to 50% as the video is still downloading ). I mean it actually shows two progress and I need one. If I could get average percentage value of two progress that would be better. Any suggestion regarding this will be greatly appreciated. <?php set_time_limit ( 0 ); function define_progress_callback($i) { global $conn; curl_setopt($conn[$i], CURLOPT_PROGRESSFUNCTION, function ($resource,$download_size, $downloaded, $upload_size, $uploaded) { if($download_size > 0) $progress = round($downloaded / $download_size * 100); echo '<script language="javascript">$(".loader").loader("setProgress", '.$progress.');</script>'; echo str_pad("",1024," "); flush(); usleep(20000); }); } $urls = array("http://example.com/file.mp3", "http:/example.com/file.mp4"); $save_to='./tmp/'; $conn = array(); $fp = array(); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $g = $save_to . basename($url); $conn[$i]=curl_init($url); $fp[$i]=fopen ($g, "wb"); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); curl_setopt ($conn[$i], CURLOPT_FILE, $fp[$i]); curl_setopt ($conn[$i], CURLOPT_HEADER ,0); curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,60); curl_setopt ($conn[$i], CURLOPT_MAXCONNECTS, 10); curl_setopt($conn[$i], CURLOPT_NOPROGRESS, false); define_progress_callback($i); curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); fclose($fp[$i]); } curl_multi_close($mh); ?> Edited by requinix, 18 January 2015 - 08:16 AM. no code tags meant auto-embedding of mp3 Here is my dilemma... I basically have a script which by means of CURL posts to a 3rd party website to perform a login and then makes another post to update a users details based on that login session. Now as my site is getting busy I have multiple users doing the same thing and it seems that on occasion curl is getting confused and updating one users details with a different users information. This is causing real problems. It seems to be that the cookie which is being used by a user after one login is being shared by other users and they end up logging in with the same cookie - confusing the 3rd party system. My code is posted below and I need to use the cookiefile and cookiejar to maintain the php session to allow me to do what I need to do. But it seems like the same cookie is being reused by all users.... Does that make sense? Is there anything I can do to change this? Please advise.... Thanks so much! Below is the code i use to both login and post the user update function hitForm($postURL, $postFields, $referer="", $showerr = FALSE, $ispost = TRUE) { global $islocal, $path_escape; $ch = curl_init(); curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt"); curl_setopt($ch, CURLOPT_URL, $postURL); if ($ispost) curl_setopt($ch, CURLOPT_POST, 1); else curl_setopt($ch, CURLOPT_HTTPGET, 1); curl_setopt($ch, CURLOPT_REFERER, $referer); curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE); $ret = curl_exec($ch); if ($error = curl_error($ch)) { if ($showerr) echo 'ERROR: ' . $error; return -1; exit; } $CU_header = curl_getinfo($ch); $CU_header["err"] = curl_errno($ch); $CU_header["errmsg"] = curl_error($ch); curl_close($ch); $returnout = $ret; //for debugging purposes for now we are logging all form posts SaveLog("hitform", "F[".$this->curruserid." - ".$this->currfunc." - ".date("d-m-y h:i:s")."]".$postFields); return $ret; } I am using the following code. Code: [Select] $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); $response = curl_exec($ch); curl_close($ch); I was given a .cer file should I set CURLOPT_SSL_VERIFYPEER to TRUE and set the path to the .cer file? Any help would be awesome. Thanks hello i`m new here . i need someone to build a php script with CURL and <iframe> i can pay for the script PM back or email me here icemerc@yahoo.com I want to grab the title of a cl page using curl, can anyone help me with this?
I have the urls stored in a table I just need can't get it to read the title of the post in CL, let me know if you can help.
I have a few questions that deals with cURL. Question #1 Lets say I'm using cURL to fill out a form with multiple pages. Is cURL able to navigate through each page of the form retaining all the previous pages information until I reach the last page, then submit it all without losing any data? I'm pretty sure this is possible, I just can't seem to find any tutorials or help with it. Question #2 Lets just say I was trying to fill out a form with a CAPTCHA (which obviously I can't get past). Is there a way to display the web page with all the other information filled out, and just leave the CAPTCHA blank to be manually inputted? It would be greatly appreciated if you are able to point me in the right direction! Thanks in advance! I have just installed cURL and am learn to use it but I'm not sure exactly what I am doing. For example I have a page with this code on it. if($_GET['number'] == "6") { echo("<p>the number is six</p>"); } Then I am trying to use cURL to send the number 6 to this page so that it returns "the number is 6". However I am just getting an output of "1". Below is the code I am using to try to get the output of "the number is 6". $ch2 = curl_init(); curl_setopt($ch2,CURLOPT_URL,'http://localhost/crawler/crawlertest.php'); curl_setopt($ch2, CURLOPT_POST, 1); curl_setopt($ch2,CURLOPT_POSTFIELDS,'number=6'); $return = curl_exec($ch2); echo ($return); curl_close($ch2); Thanks for any help Hi, I have server which is posting me these data through curl. $strPost = "<?xml version=\"1.0\" encoding=\"utf-8\"?>"; $strPost .= "<request>"; $strPost .= "<type>login</type>"; $strPost .= "<session>21</session>"; $strPost .= "</request>"; $url = "http://www.abc.com/xyz.php"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_HTTPGET,1); curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Content-Type: text/xml")); curl_setopt($ch, CURLOPT_HEADER, FALSE); curl_setopt($ch, CURLOPT_POSTFIELDS, $strPost); // add POST fields curl_setopt($ch, CURLOPT_POST, 1); $result = curl_exec($ch); if($result == true) { echo $result; } curl_close($ch); I wanna read the above xml send i.e $strPost at http://www.abc.com/xyz.php i m trying to read it with $_POST['strPost'] but shows empty. Can anyone help me. |