|
PHP - Php Web Spider/bot
i am making a search engine and i need help because i need it so you submit your url then it automatically scrolls it for the description and keywords then puts it in the database. my html code:
Code: [Select] <form action="submit_url.php" method="get"> <input type="text" name="url" value="url" /> <input type="submit" name="submit" value="submit" /> </form> php code so far: <html> <head> <title>submitting url: <?php $url = $_GET['url']; echo $url; ?></title> <link href="style.css" rel="stylesheet" type="text/css" /> </head> <body> <?php echo "Submitting <b>$url</b>"; $url = fopen("http://knexideas.co.cc", "r") or exit("Unable to open"); while (!feof($url)) { fgetc($url); } fclose($url); ?> </body> </html> all it does is reads the website if you put 'echo $url' at the bottom it just reads and prints the web page. Similar TutorialsHi , i've spend some time looking how its possible to spider a phpbb forum with a php script. I'd like to -for example- do a search with the CURL functions , and read out some of the links in the searchresults(topics..). Finally save the links that i want into a mysql database. Somebody got an idea? Hi all ,Its my first post here and I'm still very new to PHP . Im trying to wright a Web crawler script except i want this script to just crawl the 1 target website I enter. Basically i want my script to go to ultimateguitar.com or 911tabs.com or any other guitar tabs website and crawl the site and index any guitar tabs they have in there database. This will provid my website with a "phonebook" of guitar tabs. Its not illeagle or in breach of any copyrights im only making a database of links. Any help would be greatly appreciated! Hey guys, I am making a site where certain content will be limited to "members only", where membership is free. Now I want the google bot or whatever bot to be able to see and index this content, but when a user visits it, I want to hide it from them unless they are a member (I already do that). So basically I want to have a function that I can call that will return true or false if the current page is being requested by a search engine spider. I know it's possible because I regularly see forums doing that; posts are hidden unless you register but if you look through the google cached version, the posts are visible. How can I do that? so far all I have is the following, so that the rest of my code works. function is_spider(){ return true; } I read a http://iarematt.com/how-to-detect-a-search-engine-spidercrawler-with-php/ which talks about this, but I don't really trust it... What do you guys think, How can this be done? Hi all, I need to get all businesses including details from http://www.nswbusinesschamber.com.au/Business-directory.aspx?name=&location= . What is the best way to approach this? Are there any scripts out there? Thanks... |
|||||||