PHP - Opening An Array Of Htmls Using Curl Init
Code: [Select]
<html> <head> <title>Testing</title> </head> <body> <?php $config = parse_ini_file("config.ini",1); $email = $config['Email']; $host = $config['Host']; print_r ($host); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $host); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $contents = curl_exec ($ch); $fp = fopen($host.'.txt','w+'); $fw = fwrite($fp, $contents); curl_close ($ch); fclose($fp); ?> </body> </html> Ok, I am having a problem here. I have an automation script I am trying to make, but I hit a road block. I need to open up multiple URL's from using an array of URL's stored in my config.ini file. It looks like this: Code: [Select] Config.ini Host[] = www.websitea.com Host[] = www.websiteb.com Host[] = www.websitec.com Host[] = www.websited.com I want to use curl_setopt to open up each one of the urls stored in $host. If I use... Code: [Select] print_r ($host); ...I get this output: Code: [Select] Array ( [0] => www.websitea.com [1] => www.websiteb.com [2] => www.websitec.com [3] => www.websited.com ) Now, I need to take those websites in the array and open each one of them using curl_unit. I tried doing this... Code: [Select] curl_setopt($ch, CURLOPT_URL, $host); Then Code: [Select] $fp = fopen($host.'.txt','w+'); But I knew that wouldnt work. Basically, I need to open up all of the pages listed in the .ini file and save the HTML Content of those pages to their own text file, that's named after the URL. I figured I could open an array of URLS from my INI and write each one to their own text file. Example: websitea.com.txt (HTML Content from www.websitea.com) websiteb.com.txt (HTML Content from www.websiteb.com) websitec.com.txt (HTML Content from www.websitec.com) websited.com.txt (HTML Content from www.websited.com) Similar TutorialsI have this line of code in a function but I can't figure out what its doing. $this->Component->init($this); when I print_r($this->Component); i get Component Object ( [__controllerVars] => Array ( [plugin] => [name] => [base] => ) [_loaded] => Array ( ) [_primary] => Array ( ) [__settings] => Array ( ) [_log] => ) There is no init method or object in the Component Object as you can see, so I am lost. Any ideas? Thanks go on with the title, i mean, when i click the download button and it starts downloading, i can't refresh the page or go to other adress of my site, it will really take a long time. the header information is something like this (except some judgement) header("Content-Disposition: attachment; filename=" . $name); header("Content-Type: application/force-download"); header("Content-Type: application/octet-stream"); header("Content-Type: application/download"); header("Content-Description: File Transfer"); header("Content-Length: " . filesize($adress)); flush(); // this doesn't really matter. $fp = fopen($adress, "r"); while (!feof($fp)) { echo fread($fp, 65536); flush(); // this is essential for large downloads } fclose($fp); it takes me days but i cant find any way...thanks a lot!!! thanks a lot!!! thanks a lot!!! Hi - good evening! thanks for the answer and all the help so far Oncemore the "Fetching-part" for a little parser-script: Here we go - new target urls: see the overview: http://dms-schule.bildung.hessen.de/index.html http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html Search by pressing the button "type" and then choose all schools with the mouse! Results 2400 schools: Here i can provide some "more help for getting the target!" - btw: see some details for this target-server: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9009 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9742 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9871 well - you see i have to iterate over the sites - with a function /(a loop) http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=1000 to 10000 BTW - after fetching the page i have to see which one are empty - those ones do not need to be parsed! Well - i want to do this with curl-multi since this is the most advanced way to do this: I see i have an array that can be filled I have to try it out...!! Well, I'm not pretty sure - but judging by the low level of PHP i have: I think that i may need a double array to define the URLs like: Can i do like so?: $urls = array( "siteone" => "suche_schul_db.html?show_school=%i", "sitetwo" => "suche_schul_db.html?show_school=%i", "sitethree" =>"suche_schul_db.html?show_school=%i" ); $params = array ( for ($i = 1; $i <= 10000; $i++) { // body of loop } // well i have to define the variables in a open form like abvove nand not below: // better than doing it like so: "siteone" => array(9009, 9742, 9871), ); then pass them to curl-multi as: foreach ($urls as $id => $url) { foreach ($params[$id] as => $param) { $i = $id . $param; $finalurl = sprintf($url, $param); $conn[$i] = curl_init($finalurl); I don't know if that's fits the needs tip that i am looking for but I hope so ;-) Can i do like so [see above]? What do yu think. i send you many greetings martin Hi, This is the code I made to show the problem: $useragent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)"; $timeout = 10 ; $cookie = tempnam ("/tmp", "CURLCOOKIE"); $post = array('_method'=>"put", 'authenticity_token'=>' zcvcxfsdfvxcv', 'profile_image[a]'=>"@Girl-Next-Door-movie-f01.jpg" ); $ch = curl_init(); curl_setopt( $ch, CURLOPT_USERAGENT, $useragent); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:')); curl_setopt($ch, CURLOPT_URL, "http://localhost/test.php"); curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie); curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie ); curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true ); curl_setopt( $ch, CURLOPT_ENCODING, "" ); curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true ); curl_setopt( $ch, CURLOPT_AUTOREFERER, true ); curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false ); # required for https urls curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, $timeout ); curl_setopt( $ch, CURLOPT_TIMEOUT, $timeout ); curl_setopt($ch, CURLOPT_POSTFIELDS, $post); $html = curl_exec($ch); curl_close($ch); Now this link used above: http://localhost/test.php has this code: print_r($_POST); print_r($_FILES); It simply prints whats in post and files. So the above code displays this on screen: Code: [Select] Array ( [_method] => put [authenticity_token] => zcvcxfsdfvxcv ) Array ( [profile_image] => Array ( [name] => Array ( [a] => Girl-Next-Door-movie-f01.jpg ) [type] => Array ( [a] => image/jpeg ) [tmp_name] => Array ( [a] => /tmp/phppLJPQV ) [error] => Array ( [a] => 0 ) [size] => Array ( [a] => 55377 ) ) ) but we need to modify the code so that it should display this: Code: [Select] Array ( [_method] => put [authenticity_token] => zcvcxfsdfvxcv ) Array ( [profile_image[a]] => Array ( [name] => Girl-Next-Door-movie-f01.jpg [type] => image/jpeg [tmp_name] => /tmp/phppLJPQV [error] => 0 [size] => 55377 ) ) Meaning, it is taking this(profile_image[a]) as an array when we are defining $post because that's how curl recognizes that we want to upload only a file or an array of files on server by http post. So, basically the problem is the name of the input field that is defined as an array (profile_image[a]) on the web page that we are trying to mock. If this(profile_image[a]) was this (profile_image_a)(without []) on that webpage, then it would not have been a problem. So, if you understand, this is basically a syntax problem. I do not know how stop curl from reading 'profile_image[a]' as an array here 'profile_image[a]'=>"@Girl-Next-Door-movie-f01.jpg. I need curl to read 'profile_image[a]' as an string and not array. I have to use [], otherwise I will not be able to mock the webpage as the name will change. It will give error. I hope I explained the problem and also gave you a way to test if you have a solution. Again, if your code starts displaying this: Code: [Select] Array ( [_method] => put [authenticity_token] => zcvcxfsdfvxcv ) Array ( [profile_image[a]] => Array ( [name] => Girl-Next-Door-movie-f01.jpg [type] => image/jpeg [tmp_name] => /tmp/phppLJPQV [error] => 0 [size] => 55377 ) ) ,then we have a solution. Thanks for helping in advance. Regards, Manoj hello dear php-friends i currently work on a little parser project i have to find solutions for the a. fetching part b. parser part here we go - the target urls: see the overview: http://dms-schule.bildung.hessen.de/index.html http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html Search by pressing the button "type" and then choose all schools with the mouse! Results 2400 schools Here i can provide some "more help for getting the target!" - btw: see some details for this target-server: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9009 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9742 http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=9871 well - you see i have to itterate over the sites - with a function /(a loop) http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=1000 to 10000 BTW - after fetching the page i have to see which one are empty - those ones do not need to be parsed! Well - i want to do this with curl-multi since this is the most advanced way to do this: I see i have an array that can be filled -... but i have to think about the string-concatenation - i guess that i have make some sophisticated string concatenation. this one does not fit - for($i=1;$i<=$match[1];$i++) { $url = "http://www.example.com/page?page={$i}"; and besides this i have an array - i c an fill the array. can you help me how to run in a loop with <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "http://www.google.com/", "http://www.altavista.com/", "http://www.yahoo.com/" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> good day dear community, i am workin on a Curl loop to fetch multiple pages: i have some examples - and a question: Example: If we want to get information from 3 sites with CURL we can do it like so: $list[1] = "http://www.example1.com"; $list[2] = "ftp://example.com"; $list[3] = "http://www.example2.com"; After creating the list of links we should initialize the cURL multi handle and adding the cURL handles. $curlHandle = curl_multi_init(); for ($i = 1;$i <= 3; $i++) $curl[$i] = addHandle($curlHandle,$list[$i]); Now we should execute the cURL multi handle retrive the content from the sub handles that we added to the cURL multi handle. ExecHandle($curlHandle); for ($i = 1;$i <= 3; $i++) { $text[$i] = curl_multi_getcontent ($curl[$i]); echo $text[$i]; } In the end we should release the handles from the cURL multi handle by calling curl_multi_remove_handle and close the cURL multi handle! If we want to another Fetch of sites with cURL-Multi - since this is the most pretty way to do it! Well I am not sure bout the string concatenation. How to do it - Note I want to fetch several hundred pages: see the some details for this target-server sites - /(I have to create a loop over several hundred sites). * siteone.example/?show_subsite=9009 * siteone.example/?show_subsite=9742 * siteone.example/?show_subsite=9871 .... and so on and so forth Question: How to appy this loop into the array of the curl-multi? <?php /************************************\ * Multi interface in PHP with curl * * Requires PHP 5.0, Apache 2.0 and * * Curl * ************************************* * Writen By Cyborg 19671897 * * Bugfixed by Jeremy Ellman * \***********************************/ $urls = array( "siteone", "sitetwo", "sitethree" ); $mh = curl_multi_init(); foreach ($urls as $i => $url) { $conn[$i]=curl_init($url); curl_setopt($conn[$i],CURLOPT_RETURNTRANSFER,1);//return data as string curl_setopt($conn[$i],CURLOPT_FOLLOWLOCATION,1);//follow redirects curl_setopt($conn[$i],CURLOPT_MAXREDIRS,2);//maximum redirects curl_setopt($conn[$i],CURLOPT_CONNECTTIMEOUT,10);//timeout curl_multi_add_handle ($mh,$conn[$i]); } do { $n=curl_multi_exec($mh,$active); } while ($active); foreach ($urls as $i => $url) { $res[$i]=curl_multi_getcontent($conn[$i]); curl_multi_remove_handle($mh,$conn[$i]); curl_close($conn[$i]); } curl_multi_close($mh); print_r($res); ?> I look forward to your ideas. $post='{"cart_items":[{"configuration":{"price":100,"recharge_number":"9999999999"},"product_id":"999","qty":1}]}';i try this n reslut was :There are no valid items in cart: help me plz Edited by ShivaGupta, 30 November 2014 - 01:11 AM. Hi there, I am new to PHP and trying to put my learning together as I progress. Here's the problem: I would like to open HTML file after a certain condition is met. I tried to fopen() but it gave an empty page. Here's the code: Code: [Select] <?php $shape = $_POST['shape']; if($shape=='Triangle') { fopen('ogc_show_triangle.php', 'r'); } else { echo "Select a shape."; } ?> Thanks in advance! ^_^ Hi, I'm migrating a site from PHP4 (4.4.6) to PHP5 (5.3.5), am replacing all short open tags, and have many '<?=' opening tags to migrate as well. I saw a reference in the online docs about an equivalent '<?php=' tag, but it doesn't appear to work. What is the best way to 'find and replace' a PHP4 snippet like '<?= $var ?>' into a PHP5, non-short-open-tag equivalent? thanks, Barry I'm planning on writing a library script holding pdf files which will serve the purpose as ebooks in which people could search and open. I don't know for sure, but doubt that you can open a pdf file and make it readable in php. I've noticed adobe reader has managed to do it, and what method do they use to do so? This topic has been moved to Other. http://www.phpfreaks.com/forums/index.php?topic=349781.0 Hey all, I'm just having a problem with this code.. the local server says it has an error on a particular line that will be pointed out in the coding section. Code: [Select] <?php $hd = fopen('content.txt', 'r'); $line = fgets($hd); $content = array(); while($line) { $word = explode(',', $line); $content [$word[0]] => array('img_path' => $word[1], 'description' => $word[2], 'price' => $word[3]); // the error message is indicating this line is broken!! Is it because of an equal sign missing? I tried putting an equal sign, but still it doesn't run. $line = fgets($hd); } fclose($hd); ?> Your help is greatly needed.. I have a page that will sort my DB table by name and telephone number. And it has a button. I want visitors to be able to input a last name, and have the telephone number provided in a SEPERATE and NEW window after they click the button. Any suggestions for best procedure? My two initial concepts a 1- Make this a form, with the "action" leading to a file like SELECT *FROM myTable WHERE last name="$inputname" and have then use Javascript ONSUBMIT to open a NEW window. (not sure it will succeed as I desire) 2- Make this a simple BUTTON with Javascript pointing everything to the file to process the quiery. PS: I've heard IE doesn't always handle "_BLANK" properly, and I'm unaware of a PHP method. Haven't thought it through entirely, so I'm looking for pointers and advise. Hey guys, I've been searching the forum, but haven't found a post describing the following problem: I want to Open and Manipulate a word document using the COM class. This is my code so far: $Wrd = new COM("Word.Application"); $DocName = TEMPLATE_DIR.'template.doc'; $Wrd->Application->Visible = True; $WrdDoc = $Wrd->Documents->Open(realpath($DocName)); # do something here $WrdDoc->SaveAs(realpath($DocName)); $Wrd->ActiveDocument->Close(false); $Wrd->Quit(); $Wrd = NULL; There's no problem creating the COM-object, but when i try $WrdDoc = $Wrd->Documents->Open(realpath($DocName)); the browser is either complaining about insufficient memory or will chew on the code for ages... I can see that a temporary word file is created, so it seems that the document has been opened... still, the browser is working and working.... can anyone tell me what's going on? Many Thanks! Hi. I am working with some rather large files. Sometimes I am just opening a file and trying to find an entry on a single line. Sometimes I am opening a file and working on each line.
Whats the most efficient way to work with a large file?
If I open a file, and place all lines into an array, I will use a lot of memory.
If I open a file, and work on it line by line, I should use much less memory, but will the process take longer?
Whats most efficient?
I have underlined my main question above.
Thanks :-)
Warning: include(): Failed opening 'lang/en.php' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/name/public_html/db/config.php on line 13What does the for inclusion mean. Line 13: include 'lang/'.$_SESSION['default_lang'].'.php';Thank you. There are no opening and closing tags in the code example of the file PostsController.php in this tutorial:
http://book.cakephp....g/part-two.html
Should I always use opening and closing PHP tags throughout with CakePHP? |