|
PHP - Getting Data From External Website (a Wiki) Into My Website
I'm trying to make a League of Legends (a video game) community website, both as a personal project and for practice.
Now the game has a lot of champions, each of whom have 5 unique abilities. Now, I thought about manually inputting all the details about each champion into a MySQL database, but that would long and tedious, and I don't really have the time for it now. Also, the game patches very oftern (like, once every 2 weeks) which changes many of the stats, etc. of the champion, and it is not possible for me to keep manually updating these every time there is a patch. Fortunately, there is a League of Legends Wiki which has all the data I need in their specific champion pages, which they keep updated per patch. So I was wondering if there was any way to get the data from the divs in the wiki, and have it display on my site. What I want to do in my website is that whenever someone types a champion's name (in a post or whatever), I want it to display a hover-over dialog with some of the champions details. And a lot of other features such as that. In plain English I need a way to : > Tell PHP to go to the wiki's source code on a specific page > Find a specific div container > Get X data from there > Pass X data into a function to display the hover-over I think this way, I would not have to maintain a database as I can leech off the wiki's data. I have not coded anything like this before, so I would like a few pointers as to how to achieve this. Any help will be appreciated! Similar TutorialsWhen I use PHP to gain data from another webpage I usually use file_get_contents Is there a faster way to retrieve data from webpages faster since it only seems to do around 200 website line retrievals per minute? Is cURL faster? Thank you hello My database is in a same server with seperate domain name , then I want to insert from website1 mysql data on website2 mysql data. can anyone help me? Hi, I would like to know how to get The Value Of An Hidden TextBox From an External Website than Mine. I'm Coding an Addons for CCBChurch.com (mychurch.ccbchurch.com/app/trx_submit_bp.php), This is a donation website, They want me to code an addons so people can place it on their website and people can donate. I made it in cURL with POST but they generate a TimeStamp and a TimeStamp encoded, The 2 are on mychurch.ccbchurch.com/app/trx_submit_bp.php on hidden textbox. I would like to get the TS and TSH from the hidden textbox on the website, Is that possible ? I would too like to put it in Variable $ts and $tsh... Thanks ! Hi all, Im just wondering how you go about building a search function to another website, So on my webpage i could have a search that searchs the itunes store or something? Has anybody previous experience with this or can point me in the right direction, I have googled this but all im being shown is an overwhelming amount of results for google instant! thanks in advance I have been trying to figure this out for a friend of mine that wants this done, but I have begun to think that it just isn't possible - for security reasons... I need to run a HTML form on an external website for each url selected - so run through a loop processing the form over and over... Something like: for($i=0;$i<=count($urls);$i++){ // run form on http://website.com using $url[$i] } Can this be done? I am thinking that without recreating the PHP script the site uses in the form, it can't. Part of my page is taken from external webpages by preg_match. Since It is a partial section, it contains incomplete tags such as <div>, <span>, etc. Thus, they mix up with my entire page and my </div>, </span> are used for the incomplete tags in the borrowed section. How I can end such tags? or separate this section from the main codes (I mean having an isolated section)? I would like to display weather conditions on my website and store them in mysql database. I wonder if it is possible to load web page into php for parsing so the required info could be found and used? Is it doable with php and what functions could you recommend for this functionality? How can I have program running in the background on the server which would be triggered every so often to perform this task? Another question is if I want to trigger some action once a week like Tuesday at noon, is there function in php which could be used to check what day of the week and time it is ? I am new to website design so any help will be greatly appreciated. OK, I have the initial cURL working but need to figure out how to extract data I want off that webpage to display or store in a database, I tried using dom and xpath, but because of the way the page displays using css, i think its not picking it up. Here is my cURL script: <?php $userAgent = 'Googlebot/2.1 (http://www.googlebot.com/bot.html)'; $target_url = "www.test.com"; $ch = curl_init(); curl_setopt($ch, CURLOPT_USERAGENT, $userAgent); curl_setopt($ch, CURLOPT_URL,$target_url); curl_setopt($ch, CURLOPT_FAILONERROR, true); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_setopt($ch, CURLOPT_AUTOREFERER, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER,true); curl_setopt($ch, CURLOPT_TIMEOUT, 10); $html = curl_exec($ch); if (!$html) { echo "<br />cURL error number:" .curl_errno($ch); echo "<br />cURL error:" . curl_error($ch); exit; } // parse the html into a DOMDocument $dom = new DOMDocument(); $dom->loadHTML($html); // grab all the on the page $xpath = new DOMXPath($dom); $hrefs = $xpath->evaluate("/html/body//td"); for ($i = 0; $i < $hrefs->length; $i++) { $href = $hrefs->item($i); $url = $href->getAttribute('href'); storeLink($url,$target_url); echo "<br />Link stored: $url"; } ?> and here is a snippet of the source of the page I am getting: <span id="lblTest"><h1 id='surrZipTitle'>Agents in Surrounding Zip Codes</h1><table cellpadding='0' cellspacing='0' border='0' class='tblDent'><tr><td class='tdEliteTitle'><span class='caaSubHead3 addwidth'>H.K. Dent Elite</span></td></tr><tr><td class='tdEliteContent'><table cellpadding='0' cellspacing='0' border='0'><tr><td valign='top'><span class='caaAgencyName2 addwidth'>PROFESSIONAL INS ASSOC, INC.</span></td><td valign='top'> </td></tr></table><table cellpadding='0' cellspacing='0' border='0'><tr><td width='360px' valign='top'><div class='addressBlock'><span>4444 MANZANITA AVE STE 6</span><br /><span>CARMICHAEL , CA 95608-1488</span><br /><a class='faaBlueLink' id='lnkContact' href='http://www.safeco.com/portal/server.pt/gateway/PTARGS_0_20656_395_362_0_43/http%3B/por-portlets-prd.int.apps.safeco.com%3B13425/dotcom/FindAnAgent/find-an-agent/contactanagent.aspx?RequestType=agency&level=elite&Id=0415199904150295&lat=38.646142&lng=-121.327623' onclick='oOobj4.Preferences.Plugins.Events.poX=0;'>Contact & Directions</a> <a class='faaBlueLink' id='lnkWebSite' style='display: none;' href='http://' target='_blank' onclick="return trackEvent('/External-Link/AgentWebsite/ ','PROFESSIONAL INS ASSOC, INC. ');">Website</a></div></td><td valign='top'> </td></tr></table></td></tr></table><table cellpadding='0' cellspacing='0' border='0' class='tblDent'><tr><td class='tdEliteTitle'><span class='caaSubHead3 addwidth'>H.K. Dent Elite</span></td></tr><tr><td class='tdEliteContent'><table cellpadding='0' cellspacing='0' border='0'><tr><td valign='top'><span class='caaAgencyName2 addwidth'>AMERICAN AIM AUTO INS AGY, INC</span></td><td valign='top'> </td></tr></table><table cellpadding='0' cellspacing='0' border='0'><tr><td width='360px' valign='top'><div class='addressBlock'><span>5339 SAN JUAN AVE</span><br /><span>FAIR OAKS , CA 95628-3318</span><br /><a class='faaBlueLink' id='lnkContact' href='http://www.safeco.com/portal/server.pt/gateway/PTARGS_0_20656_395_362_0_43/http%3B/por-portlets-prd.int.apps.safeco.com%3B13425/dotcom/FindAnAgent/find-an-agent/contactanagent.aspx?RequestType=agency&level=elite&Id=0415911704151222&lat=38.66237&lng=-121.292429' onclick='oOobj4.Preferences.Plugins.Events.poX=0;'>Contact & Directions</a> So basically I want to extract the agency name like "<span class='caaAgencyName2 addwidth'>PROFESSIONAL INS ASSOC, INC.</span>" and the address which always use the same div class like "caaAgencyName2" and "addressBlock". How can this be accomplished? Hello! I would like to grab data from a website. What I want is to take the name and time and output it like following:
Ayarith 59 seconds Indirarc 54 seconds
The data is taken from an online game with the following link: https://medivia.online/community/online/legacy
Exemple data from this link: <li><div class="med-width-25">59 seconds ago</div><div class="med-width-35"><a href="/community/character/Ayarith">Ayarith</a></div><div class="med-width-15">Druid</div><div class="med-width-25 med-text-right med-pr-40">32</div></li> <li><div class="med-width-25">54 seconds ago</div><div class="med-width-35"><a href="/community/character/Indirarc">Indirarc</a></div><div class="med-width-15">Druid</div><div class="med-width-25 med-text-right med-pr-40">20</div></li>
I hope you can help me out! I am wondering if it's possible to get some data from another website via PHP? I would like to get data from website http://www.gamersfirst.com/warrock/?q=Player&nickname=soldier, and the exact data which I need is "Level" which is in this case 2. Can this value be grabbed and if it can, where can I get some info about how to do it? Hi, I am quite new to PHP and to learn more about it I decided I would make a little project to easy my life a little bit I want to make a site that fetches lunch menus from the restaurants nearby my work. I would just like some help getting pointed in the right direction on how to make this happen Thanks! Br, Niklas I have a page (input.php) that will allow a user to upload a CSV file. This file has 5 columns (SKU, Product, Quantity, Retail Price, and Total Retail Price). The CSV upload will only have the SKU number and Quantity filled in. When the user hit upload the (import.php) page is suppose to go to the site and pull the product up by searching the SKU number and pulling the price and product (brand and title). I paid a freelancer to create this code. I watched it work on his machine. I cant seem to get it to work on mine (wont pull price or product) and he is non-responsive now. Any help would be greatly appreciated!! I added some note in the code as I was troubleshooting.
<?php
ini_set('max_execution_time', 0);
error_reporting(0);
move_uploaded_file($_FILES["file"]["tmp_name"], "upload/". $_FILES["file"]["name"]);
$handle = fopen("upload/". $_FILES["file"]["name"], "r");
$file = '';
$line .= "SKU,Product,Quantity,Retail Price,Total Retail Price";
$file .= $line . PHP_EOL;
for ($i = 0; $row = fgetcsv($handle ); ++$i) {
// Do something will $row array
if($row[0]!="" AND $i>0)
{
$line="";
#echo "<pre>";
#print_r($row);
$SKU=$row[0];
$quantity=$row[2];
$loop=1;
do{
$url = "https://www.homedepot.com/s/".$SKU;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, 1);
$response = curl_exec($ch);
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headers = substr($response, 0, $header_size);
$body = substr($response, $header_size);
curl_close($ch);
header("Content-Type:text/plain; charset=UTF-8");
$headers_arr = explode("\r\n", $headers);
$str=$headers_arr[5];
$arr=explode(":",$str);
$check=trim($arr[0]);
#echo $check; ### remove troubleshooting
if($check=="location") # made lowercase so it would get inside the If statement
{
#echo "Dustin"; ## remove troubleshooting
$productPageLink=$headers_arr[5];
$productPageLink=str_replace("Location:","",$productPageLink);
#echo $productPageLink; ## troubleshooting -- seems to be getting the links
$productPageLink=trim($productPageLink);
$productPageLink=str_replace("http:","https:",$productPageLink);
#echo $productPageLink; ## troubleshooting -- still seems to have links
$ch = curl_init();
#echo $ch; ##troubleshooting -- prints out "resouce id"
curl_setopt($ch, CURLOPT_URL, $productPageLink);
#echo $ch; ##troubleshooting -- prints out "resouce id"
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');
$headers = array();
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
$headers[] = 'Accept-Language: en-US,en;q=0.5';
$headers[] = 'Upgrade-Insecure-Requests: 1';
$headers[] = 'Connection: keep-alive';
$headers[] = 'Te: Trailers';
#echo $headers; ##troubleshooting -- ## Troubleshooting -- prints out "Array"
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
#echo $result; ### troubleshooting - doesnt have any data
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch); # it end inside this if statement, however no error is printed
}
curl_close($ch);
preg_match_all('/<h2 class="product-title__brand" itemprop="brand" data-component="clickable brand link">(.*?)<\/h2>/s', $result, $output_array_brand);
#echo "<pre>"; ###
#print_r($output_array_brand);####
$brand=trim(strip_tags($output_array_brand[1][0]));
preg_match_all('/<h1 class="product-title__title">(.*)<\/h1>/', $result, $output_array);
$productTitle=$output_array[1][0];
$productTitle=$brand." ".$productTitle;
preg_match_all('/<span class="price__dollars">(.*?)<\/span>/s', $result, $output_array_price);
preg_match_all('/<span class="price__cents">(.*)<\/span>/', $result, $output_array_cent);
#echo "<pre>";
#print_r($output_array_price);
$price=trim(strip_tags($output_array_price[1][0]));
$cent=trim(strip_tags($output_array_cent[1][0]));
if($cent!="" OR $cent!=0)
{
$price=$price.".".$cent;
}
$line.=$row[0].",";
$line.='"'.$productTitle.'",';
$line.=$row[2].",";
$line.=$price.",";
$totalPrice=$row[2]*$price;
$line.=$totalPrice;
$file .= $line . PHP_EOL;
}
# echo "<br>";
$loop=$loop+1;
#echo "<br>";
if($loop>4)
{
if($check!="Location")
{
$line.=$row[0].",";
$line.=',';
$line.=$row[2].",";
$line.=",";
$line.="";
$file .= $line . PHP_EOL;
break;
}
}
}
while($check!="Location");
}
}
fclose($handle);
header('Content-Type: application/csv');
$output=$_REQUEST['output'];
header('Content-disposition: attachment; filename='.$output.'.csv');
echo $file;
#header('Content-disposition: attachment; filename='.$output.'1.csv');
#echo $file1;
exit;
?>
There is a school locator script at https://www.ocps.net/parents/pages/FindaSchool.aspx When we input any address it returns schools located in that locality. For example: Use these details and submit the form Street Number : 3902 Street Name : Bobolink Street Type : Lane City : Orlando It returns a row having three schools Elementary,Middle and High school On clicking the more button it takes to the respective school details So for each school I need the respective school names Audobon Elementary Glenridge Middle Winter Park High Please suggest some ways to achieve this functionality Thanks _________________________________________________ ____________________ http://travelinfo.techserveglobal.com/ http://blog.techserveglobal.com/ It's very possible, and I've seen a website that's done it (http://www.zybez.net/radio/) - They are able to include the recent songs played on there website. The information can be obtained from right he http://68.168.100.60:7942/played.html But I'm not sure if thats where they get it from. If I login to my Admin section, there will be an option that says get XML stats. But I'm not sure if that shows the recent songs played like it would at http://68.168.100.60:7942/played.html (Everytime I refresh the page, the stats/data/info in the XML change) Code: [Select] 1250001522Rockhttp://srbuckey.listen2myradio.comMusicPEAK - RockKings of Leon - Radioactive N/AN/AN/A226153140audio/mpeg1.9.86201901701119000000004013207.144.125.60NSPlayer/11.0.6001.7006 WMFSDK/11.0026101521287587284Kings of Leon - Radioactive 1287586919Three Days Grace - Riot 1287586542Alice In Chains - Here Comes The Rooster 1287586281Green Day - Boulevard of Broken Dreams 1287586277down 1287586075Green Day - American Idiot 1287586044Metallica - Enter Sandman 1287585836ACDC Highway To Hell.mp3 1287585449Metallica - 04 The Unforgiven I.MP3 1287585448[HTTP/1.1 200 OK] http://www.ventsi.com/Music/MetallicA/1991 - Black Album/Metallica - 04 The Unforgiven I.MP3 Hello guys, I have intermediate level PHP and I am currently working on a website. I need to write a function that will get me some specific information from a website. I need this function to go write the users name on the search bar over the http://competitive.eune.leagueoflegends.com/ladders/eune/current/rankedsolo5x5 after it founds which page the user is on I want the information of that user registered on my database. But I am not sure where to start.. An example would be like http://competitive.eune.leagueoflegends.com/ladders/eune/current/rankedsolo5x5?summoner_name=&page=4 page 5: rank 121 Sokoren 42 22 1877 I already know the name, I just need the other numerical information and get them into my database. I only want this function to work when a certain nick name is entered. Any help would be appreciated. Hi, I'm trying to retrieve/scrape some information from a website using the class name and the tag name.
Below is the example in VB:
Dim htmL_cat As HTMLDocument
Dim objTableL_cat As Object, objDatL_cat As Object, objItemL_cat As Object, objKeyL_cat As Object
Dim intRowL_cat As Long
Set htmL_cat = New HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "http://www.lelong.com.my/Auc/List/BrowseAll.asp", False
.send
htmL_cat.body.innerHTML = .responseText
End With
With htmL_cat
Set objTableL_cat = .getElementsByClassName("CatLevel1") 'Find elements with class name first
For Each objDatL_cat In objTableL_cat
Set objKeyL_cat = objDatL_cat.getElementsByTagName("a") 'Next, find elements with tag name
For Each objItemL_cat In objKeyL_cat
Sheets("Analytics").Range("E6").Offset(intRowL_cat, 0) = objItemL_cat.innerText
intRowL_cat = intRowL_cat + 1
Next
Next
End With
Set htmL_cat = Nothing
Set objTableL_cat = Nothing
Set objKeyL_cat = Nothing
How do I do the same using PHP? Thanks.
I'm a beginner to PHP and am stuck as to how I can fetch data from a MYSQL database and output that information in html to the website.
Currently I get the information from the database using normal php:
$result = mysql_query("SELECT * FROM truffleProducts");
Then I assign each field from the database to a php variable in a while loop:
while ($row = mysql_fetch_array($result)) {
$id = $row{'id'};
$name = $row{'name'};
$price = $row{'Price_per_kg'};
$season = $row{'Season'};
$country = $row{'Country'};
$image = $row{'image'};
$review = $row{'review'};
}
Then I just print each variable in a table using simple html and php.
My problem is that it only outputs the latest element or row from my database.
I want to output the database within a HTML template so that it is professional and tidy, but i cant figure out how to do that.
Any help?
Edited by louiscb, 22 October 2014 - 07:58 AM. Hi,
I need to log into a website to check for the latest data, and I want to automate this with php script that will use a cronjob to log in, retrieve the data and email it to me.
Currently I'm stuck on the logging in part (I'm testing the script below on my squirrelmail email login page). I've tried using the snoopy php class (sourceforge.net/projects/snoopy/) but it seems the POST variables aren't being recognised (because the email program is directing me to the log in page, as opposed to telling me that my username/password is wrong).
(is it even possible for a servers to accept POST variables in this way??? I'm kind of surprised but lots of poeple seem to use snoopy to login via userforms)
Please could someone point me in the right direction?
Thanks,
Stu
/ load the snoopy class and initialize the object / $snoopy = new Snoopy(); / set some values / $login_form['login_username'] = 'MY_USERNAME'; $login_form['secretkey'] = 'MY_PASSWORD'; $snoopy->cookies['vegetable'] = 'carrot'; $snoopy->cookies['something'] = 'value'; / submit the data and get the result / $snoopy->submit('http://webmail.MY_WE...rc/redirect.php' $p_data); / output the results / echo '<pre>' . htmlspecialchars($snoopy->results) . '</pre>'; Edited by stubarny, 14 June 2014 - 11:27 AM. Hi,
I've got a "data.txt" file with the following content:
One1:One2:One3:One4:One5:One6 Two1:Two2:Two3:Two4:Two5:Two6 Three1:Three2:Three3:Three4:Three5:Three6Now I want to be able to take each data and put on a specific location of a htlm code. Each line for its own. So for the first line, it should look something like this: <html> <head> <title></title> <head> <body> <h1>One2</h1> <h2>One4 some other Text One5</h2> <img src="One6.jpg"> </body> </html>Unforturtunately I don't have a clue how to do that. Can anybody help me out or does someone know a good and easy tutorial? Thanks a lot |
|||||||