|
PHP - Help With Function To Crawl For Links In All Website.
I'm trying to crawl for links in a specific website and show them at the end. The problem i'm facing is that it only show the links from the specific page not the whole pages in the website. I tried several loops with no success please give some advise.
Here is the code: <?php if (isset($_POST['Submit'])) { function getLinks($link) { /*** return array ***/ $ret = array(); /*** a new dom object ***/ $dom = new domDocument; /*** get the HTML (suppress errors) ***/ @$dom->loadHTML(file_get_contents($link)); /*** remove silly white space ***/ $dom->preserveWhiteSpace = false; /*** get the links from the HTML ***/ $links = $dom->getElementsByTagName('a'); /*** loop over the links ***/ foreach ($links as $tag) { $ret[$tag->getAttribute('href')] = $tag->childNodes->item(0)->nodeValue; } return $ret; } /*** a link to search ***/ $link = $_POST['address']; /*** get the links ***/ $urls = getLinks($link); /*** check for results ***/ if(sizeof($urls) > 0) { foreach($urls as $key=>$value) { if (preg_match('/^(http|https):\/\/([a-z0-9-]\.+)*/i',$key)) { echo '<span style="color:RED;">' . $key .' - external</span><br >'; } else { echo '<span style="color:BLUE;">' . $link . $key . ' - internal</span><br >'; } } } else { echo "No links found at $link"; } } ?> <br /><br /> <form action="" method="post" enctype="multipart/form-data" name="link"> <input name="address" type="text" value="" /> <input name="Submit" type="Submit" /> </form> Similar TutorialsI tried searching on google but couldn't find any relevant information, please redirect me to relevant source or help me with the code. I want to pass a domain name in text field which will be scanned and then the script will display entire site map. Not external links or links on a page. Sorry it is not easy for me to explain. Eg. if i pass abc.com Script will display abc.com/12/adn.php abc.com/asd/asd/ etc Whatever their url format is. All the links on that domain. HI, Does anybody knows a script that extract all of a website links.I mean I enter a website url and It begins to extract all of the links that exist in that website. Thanks Is it possible to extract all links from a website (not a single webpage) by php? I am asking about the general idea, as I wish to customize: e.g. from a specified directory and certain domains only. Thanks So, I have this web crawler that I want to use to index .onion websites through the TOR network. I amOk, here's what I need. I have a PHP based web crawler. It is accessible he http://rz7ocnxxu7ka6ncv.onion/ Now, my problem is that my spider that actually crawls pages needs to do so on a SOCKS port 9050. The thing is, I have to tunnel its connection through TOR so that It can resolve .onion domains, which is what I'm indexing. (Only ending in .onion.) I call this script from the command line using php crawl.php, and I add the appropriate parameters to crawl the page. Here is what I think: Is there any way to force it to use TOR? OR can i force my ENTIRE MACHINE to tunnel things through tor, and how? (Like forcing all traffic through 127.0.0.1:9050) perhaps if i set up global proxy settings, php would respect them? If any of my solutions work, how would I do it? (Step by step instructions please, I am a noob.) I just want to crate my own TOR search engine. (Don't recommend my p2p search engines- it's not what I want for this- I know they exist, I did my homework.) Here is the crawler source if you are interested to take a look at: Perhaps someone with a kind heart can modify it to use 127.0.0.1:9050 for all crawling requests? spider.php: http://pastebin.com/kscGJCc5 spiderfuncs.php: http://pastebin.com/m5y54RUh PLEASE someone help me! I am desperate. Is it possible to log into a secure page to perform a web crawl? I have the code to crawl, it's just that it's on a password protected page. Also, is this secure doing this? Or can someone potentially hack into the secure page? This topic has been moved to Miscellaneous. http://www.phpfreaks.com/forums/index.php?topic=317436.0 Hello,
I am hoping to get some help with this. I want to apologize in advance as I am not a developer more of a systems admin guy. Okay guys, here is the deal. My boss has a website, which uses from my understanding Wordpress for design, but also uses PHP. Since, I am not too familiar with the general uses of PHP, I cannot explain why they are doing it that way. The website also uses MYSQL, which I would imagine works with PHP to gather data from some back end server.
The website has five tabs on the top right corner
about services resources contact us login
When you hover over these five tabs they expand and show more menus. The tab with the issue is the services tab. When you hover over it, it works as anticipated; it expands and shows our services. Upon getting on there, there are a bunch of services, which you can click on. When you first click, it works as anticipated. However, if you try to click on another service within that category, it simply does not load the page. Now if you decide to go into a different service it works, but once you try to click on another services within x category it just does not work.
I would really appreciate everyone's help on this. It would be nice to get this resolve.
I can provide you guys with the website and php scripts and codes if need be.
This is the website
http://beta.morrowco.com/
Thanks,
Jeff M
need help i startet to regenerate an old website but the one or other thing dont works so like that: Code: [Select] <? require("droplist.inc.php"); if (!isset($region)) $region = 0; $region_names = array("Laglamia", "Dekardi", "Dekadun", "Dekaran", "Shilon", "Searoost", "Paros", "GWH"); $region_monster_count = array(14, 12, 18, 19, 16, 26, 19, 8); ?> <head> <title>Dropliste</title> <meta http-equiv=content-type content="text/html; charset=windows-1252"> <meta http-equiv=imagetoolbar content=no> <link href="image/style.css" rel=stylesheet type="text/css"> </head> <body bgColor="#2d2d2d" link="#FFFFFF" vlink="#FFFFFF" alink="#FF9900"> <hr> <div align="center"><font size="2" face="Verdana, Arial, Helvetica, sans-serif"> <? for ($idx = 0; $idx < count($region_names); $idx++) { print("<a href=\"droplist.php?region=".$idx."\">"); if ($idx == $region) printf("<font color=\"#FF9900\">"); print($region_names[$idx]); if ($idx == $region) printf("</font>"); print("</a>"); if ($idx < count($region_names) - 1) print(" | "); print("\n"); } ?> </font></div> <hr> <table cellSpacing=0 cellPadding=1 width="95%" align=center border=0> <tr><td colSpan=3 height=1><br></td></tr> <tr><td bgColor=black colSpan=3 height=1></td></tr> <tr><td bgColor=silver colSpan=3 height=1></td></tr> <tr><td colSpan=3 height=1><br></td></tr> <? for ($idx = 0; $idx < $region_monster_count[$region]; $idx++) { printf(" <tr> <td width=135 valign=top align=center> <font face=\"Arial, Helvetica, sans-serif\" size=2> <img width=130 height=140 src=\"image/droplist/".$monsters[$region][$idx][0].".gif\" border=0> </font> </td> <td width=* valign=top> <font face=\"Arial, Helvetica, sans-serif\" size=2> <b>".$monsters[$region][$idx][1]."</b> <strong>[ Level ".$monsters[$region][$idx][2]." ]</strong> <br> <font color=#ffff00>Drops:</font><br><font face=\"Arial, Helvetica, sans-serif\" size=1>".$monsters[$region][$idx][3]."</font> <br> <font color=#ffff00>Random-Drops:</font><br><font face=\"Arial, Helvetica, sans-serif\" size=1>".$monsters[$region][$idx][4]."</font> </font> </td> </tr> <tr><td colSpan=3 height=1><br></td></tr> <tr><td bgColor=black colSpan=3 height=1></td></tr> <tr><td bgColor=silver colSpan=3 height=1></td></tr> <tr><td colSpan=3 height=1><br></td></tr> "); } ?> </table> <hr> <div align="center"><font size="2" face="Verdana, Arial, Helvetica, sans-serif"> <? for ($idx = 0; $idx < count($region_names); $idx++) { print("<a href=\"droplist.php?region=".$idx."\">"); if ($idx == $region) printf("<font color=\"#FF9900\">"); print($region_names[$idx]); if ($idx == $region) printf("</font>"); print("</a>"); if ($idx < count($region_names) - 1) print(" | "); print("\n"); } ?> </font></div> <hr> </body> </html> The content iss in another file the problem iss the links (you can see the page the http://psychadelics.co.de/droplist.php ) dont work if i click on it so the funktions must be have errors can annyone help me that if i click for example Dekardi that the list shows up for it? i hope annyone can help^^ I am trying to use the php mail function and want to include links to a dynamic page, so I need to know how to do this... Something like: Code: [Select] $id=$_POST['id']; $mailto="email@address.com"; $mailsub="Subject"; $mailhead="From: other@address.com"; $mailbody="Your comment has been posted. You may view it by <a href='domain.com/page.php?id=".$id."'>clicking here</a>."; mail($mailto, $mailsub, $mailbody, $mailhead); I know this is not correct but I'm trying to figure out the actual way to do this... Say I have a chunk of code that looks like this (contains imagesx): Code: [Select] <span style="color: #0000BB">$width </span><span style="color: #007700">= </span><span style="color: #0000BB">imagesx</span><span style="color: #007700">(</span><span style="color: #0000BB">$img</span><span style="color: #007700">); <br /> within the code I would like to make all the php functions link to php.net I thought that the following code would work, but it does not, well because there is no spaces around the above word public function phpLinks($string){ $words = explode(" ", $string); $str = array(); foreach($words as $word){ if(function_exists($word)){ $str[] = preg_replace("/$word/i", '<a href="//php.net/'.strtolower($word).'">'.$word.'</a>', $word); }else{ $str[] = $word; } } return implode(" ", $str); } I now can not think of a good way to do this, so I am asking does anyone have any suggestions? Looking for help writing a function that does the following: Look for http:// or https:// in a string and replaces it with an <a href> tag. For example: Here is my link http://www.mydomain.com some further text here. Becomes: Here is my link <a href="http://www.mydomain.com">http://www.mydomain.com</a> some further text here. Hi everyone, I'm building a website for a music band and want to automate the adding of new album/disc so they wont have to update a php file to print out . I mean, a new album is release , then dump the mp3's, lyrics, cover pics in their respective directory using naming convention so my php read the directory structure and so one... but my problem is the php function opendir read the filesystem... is there a more secure way to do this ? and I presume the web host wont let me do this... thanks for any tips and tricks steph Anybody know a good mobile website redirect function? Cant find a good one. Thanks in advance! Jason I'm writing a PHP script to pull information from another website which I don't own. That website uses javascript for pagination, splitting data across 3 pages. They also have a function to list all their data on one page but it requires clicking the link on the page which runs a javascript AJAX call. I want to pull all that data with PHP but how do I emulate the Javascript AJAX call in my code so I can get all that data instead of just the first page? Even if I could just call that function and then parse the information it returns to me. Thanks.
I get this error on my homepage : Warning: implode() [function.implode]: Invalid arguments passed in /home/shqip1/peqini.com/wp-content/themes/sportpress/functions/wpzoom-widgets.php on line 1469 This is the code to modify http://pastebin.com/RiBsyG3c Thanks <html> <?php $id = $_GET['id']; $dbusername="web148-matt"; $dbpassword="matt"; $dbdatabase="web148-matt"; mysql_connect(localhost,$dbusername,$dbpassword); @mysql_select_db($dbdatabase) or die( "Unable to select database"); mysql_query("UPDATE count SET clicks=clicks+1 WHERE id='$id'"); $sql = mysql_query("SELECT link FROM count WHERE id='$id'"); $fetch = mysql_fetch_row($sql); $result = mysql_query("SELECT * FROM count"); while($row = mysql_fetch_array($result)) { echo "<a href=" .$row['link']. ">Link</a>"; } ?> <a href='http://www.google.com'>Google</a> <a href='/index.php?id=2'>link2</a> </html> So I have been working on my website for a while which all is php&mysql based, now working on the social networking part building in similar functions like Facebook has. I encountered a difficulty with getting information back from a link. I've checked several sources how it is possible, with title 'Facebook Like URL data Extract Using jQuery PHP and Ajax' was the most popular answer, I get the scripts but all of these scripts work with html links only. My site all with php extensions and copy&paste my site links into these demos do not return anything . I checked the code and all of them using file_get_contents(), parsing through the html file so if i pass 'filename.php' it returns nothing supposing that php has not processed yet and the function gets the content of the php script with no data of course. So my question is that how it is possible to extract data from a link with php extension (on Facebook it works) or how to get php file executed for file_get_contents() to get back the html?
here is the link with code&demo iamusing: http://www.sanwebe.c...-php-and-jquery
thanks in advance.
I'm trying to make a League of Legends (a video game) community website, both as a personal project and for practice. Now the game has a lot of champions, each of whom have 5 unique abilities. Now, I thought about manually inputting all the details about each champion into a MySQL database, but that would long and tedious, and I don't really have the time for it now. Also, the game patches very oftern (like, once every 2 weeks) which changes many of the stats, etc. of the champion, and it is not possible for me to keep manually updating these every time there is a patch. Fortunately, there is a League of Legends Wiki which has all the data I need in their specific champion pages, which they keep updated per patch. So I was wondering if there was any way to get the data from the divs in the wiki, and have it display on my site. What I want to do in my website is that whenever someone types a champion's name (in a post or whatever), I want it to display a hover-over dialog with some of the champions details. And a lot of other features such as that. In plain English I need a way to : > Tell PHP to go to the wiki's source code on a specific page > Find a specific div container > Get X data from there > Pass X data into a function to display the hover-over I think this way, I would not have to maintain a database as I can leech off the wiki's data. I have not coded anything like this before, so I would like a few pointers as to how to achieve this. Any help will be appreciated! hello My database is in a same server with seperate domain name , then I want to insert from website1 mysql data on website2 mysql data. can anyone help me? Is it possible to download files from a website to my online websites root directory? reason I'm asking is because I have been downloading large video files to my computer which take about 20 minutes!! Then I use FTP to upload them to my site but it takes about 2-3 hours per video!! I was looking for a faster way! All help would be great! |
|||||||