|
PHP - Help Setting Up Cron Job To Pull Http Request And Create/cache File From Results
First thanks for any help that you can provide. I am somewhat new to PHP and this is the first "Web Service" I have had to create.
THE GOAL: I need to pull XML data from another server. This company's API is setup so that you must give an IP and so you can only pull data from server to server instead of client to server. The data is pulled from the API using HTTP requests...very similar to YQL. (in essence the structured query is located in the URL). This API also requires that my server only ping's their server every 10-15 mins, in order to cut down on server requests. The logical thought in my head was to setup: a cron job to run a PHP script every 10 mins. The PHP scripts would then do the following: 1. Make the HTTP request 2. Open an existing file or create one (on my server) 3. Take the returned XML data from the API and write to the newly opened file. 4. Convert that XML to JSON. 5. Save the JSON 6. Cache the JSON file 7. STOP My thought was to use curl and fopen for the first 3 steps. I found a basic script that will do this on PHP.net (as shown below). After that I am pretty much lost on how to proceed. Code: [Select] <?php $ch = curl_init("http://www.example.com/"); $fp = fopen("example_homepage.txt", "w"); curl_setopt($ch, CURLOPT_FILE, $fp); curl_setopt($ch, CURLOPT_HEADER, 0); curl_exec($ch); curl_close($ch); fclose($fp); ?> I would REALLY appreciate any help on this. Also, if you have the time to please comment and explain what you are doing in any code examples and why....that would HELP out a lot. I truly want to learn not just grab a snippet and run. So, your comments are vital for me. thank!!! Similar TutorialsHey! I am trying to get a database table of users but am running into the error: Warning: file_get_contents(http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=68583) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 500 in get_user_from_parameter(nabble:utilities.naml:890) - <n.get_user_from_parameter.as_user_page.do/> - public v in C:\xampp\htdocs\website4js\stanford\loadUsernames.php on line 32 I have looked it up and it may be a security thing...The urls I am getting are http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=68583 but the error adds an nodes& part that screws it up. Any way around this? Code: [Select] <?PHP $maxPage = 0; $mainPage = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136"; $mainContent = file_get_contents($mainPage); $pattern = "/(?<=\"Page )\d\d+/"; preg_match_all($pattern, $mainContent, $pageNumb); //find the max page for($i=0;$i<sizeof($pageNumb[0]);$i++) { if($pageNumb[0][$i] > $maxPage) { $maxPage = $pageNumb[0][$i]; } } //echo('Max page is: '.$maxPage.'\n'); //Get an array of all the pages $pages = array(); for($i=1;$i<$maxPage;$i++) { $pages[$i] = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136&i=".($i*20); } //print_r($userPages); //Get personal page urls and add to MySQL mysql_connect("localhost" , "root") or die("Cant connect"); mysql_select_db("protegeusers") or die("Cant find database"); foreach($pages as $url) { $urlContents = file_get_contents($url); $pattern = "/http:\/\/protege-ontology-editor-knowledge-acquisition-system\.136\.n4\.nabble\.com\/template\/NamlServlet\.jtp\?macro=.+;user=\d+/"; preg_match_all($pattern, $urlContents, $personalPage); foreach($personalPage as $user) { for($i=0; $i<sizeof($user);$i++) { [color=green]$userContents = file_get_contents($user[$i]);[/color] $pattern1 = "/user\/SendEmail\.jtp\?type=user.+;user=\d+/"; $pattern2 = "/(?<=\">Send Email to ).+(?=<)/"; preg_match_all($pattern1, $userContents, $userEmail); preg_match_all($pattern2, $userContents, $username); [color=green]print_r($username); print_r($email);[/color] //$query = "INSERT INTO users (username, userurl) values ('$userName','$userUrl')"; //mysql_query($query); } } } ?> Hi, I don't know very much about php. I don't know any at all, actually. I play a game called Roblox where you get to use the programming languange, "Lua" to script your own games. What I am requesting has been done before on this game, but I only know Lua. I was also told PHP cURL is needed. Anyways, let me get to the point. I am essentially trying to create a system on my webserver (You can send HTTP Requests to external sites in this game via script) that will essentially log into an account on Roblox, to perform a task. My current idea would be to have the Lua script send a request to my webserver with an generated code that would access the username and password of my roblox account (Which would be on the webserver). It would then perform a task. Is this a good way to go about this? If not, I am open to suggestions. If you can find the time to actually help me set this up on my webserver, I would greatly appreciate it! I'm also sorry if this is the wrong place for this post, I'm new here. This topic has been moved to Linux. http://www.phpfreaks.com/forums/index.php?topic=317079.0 Hi Guys,
Total PHP Noob here. I’ve run into a brick-wall and would appreciate some words of wisdom. I can do this with a browser: http://86.60.161.24 // will echo hello World I can’t do this with my server: <?php $url = 'http://86.60.161.24'; $contents = file_get_contents($url); echo ($contents); ?> It looks like the server-side HTTP -request never “touches” my Arduino-chip. No network activity detected. Obviously there is going to be a lot more data transfer happening but I need to solve this issue first. Is there anything I can do to make this work? Any and all input greatly appreciated. Cheers!!
It seems to me that this should be easy, but I can't find it in the manual or the on the web, and the values in $_SERVER don't suggest anything. Context: script A loads script B using HTTPS. Now script B has to load resources (images, style sheets, etc). Browsers often object to HTTP references from a script run by an HTTPS request, so script B should generate URLs that use HTTPS. Thus it has to know that it was loaded with HTTPS. I can do this by brute force if I have to (defining a "protocol" constant in the script itself), but I'd rather make the test automatic. I'm trying to integrate w/ a few APIs, and getting started is causing me fits. I'm using PHP locally to add common headers and as a contact form, and those are working fine. The Code: <?php require('HTTP/Request.php'); $url = "http://example.com"; $request =& new HTTP_Request($url); $request->sendRequest(); $response = $request->getResponseBody(); ?> The Result: Warning: require(HTTP/Request.php) [function.require]: failed to open stream: No such file or directory in /home/stodgy5/public_html/trackstart/start.php on line 3 Warning: require(HTTP/Request.php) [function.require]: failed to open stream: No such file or directory in /home/stodgy5/public_html/trackstart/start.php on line 3 Fatal error: require() [function.require]: Failed opening required 'HTTP/Request.php' (include_path='.:/usr/lib/php:/usr/local/lib/php:/home/stodgy5/php') in /home/stodgy5/public_html/trackstart/start.php on line 3 I'm assuming this is a directory issue, as my cPanel says Pear is installed. Any advice? I am testing a url for http post request. It connects to the site then it is disconnected. In fire fox it is giving error that the "connection was reset". In google chrome it is giving error that "Unable to load the webpage because the server sent no data". Following is the script. I am new to curl and also new to http request response. The provider of this site tells me that everything is ok and it is sending data but i am not getting any data or any header of this site. <?php $data = '<?xml version="1.0" encoding="UTF-8"?><Request><Source ClientID="test_xml" Password="1234" /><RequestDetails Language="En"><SearchHotelPriceRequest><ServiceDestination DestinationType="city" DestinationCode="477" /><ImmediateConfirmationOnly>1</ImmediateConfirmationOnly><PeriodOfStay><CheckInDate>2012-03-01</CheckInDate><Duration>3</Duration></PeriodOfStay><Rooms> <Room NumberOfRooms="1" NumberOfAdults="2" /> </Rooms></SearchHotelPriceRequest></RequestDetails></Request>'; $ch = curl_init(); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_URL, 'https://62.218.28.13:8443/monWebService/Request/v2'); curl_setopt($ch, CURLOPT_POST, $data); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); $data = curl_exec($ch); curl_close ($ch); echo $data; ?> Hello I am a total beginner and don't know first thing about php, but still trying to figure this out. So what I am trying to achieve is, send a http request from a software to my website with a ticket number, is it possible in php to receive that request and show it on a different webpage of the same site? It would be helpful if you can point me to some related articles. Hello, If i am not posting this question at right location then i apologize for it. My problem is: I need to send HTTP get request to the webservice end point which must include the following http headers: GET / HTTP/1.1 Host: apis.live.net Accept: application/json Content-Type: application/json Authorization: WRAP access_token=AuthToken (here AuthToken is the Access Token String) Note: I am using OAuth for getting an access Token but for getting contents I need to send an HTTP get request. Any kind of help will be appreciated. Thanks Hey guys, every now and then some of my scripts that consistently access headers seem to fail to open the stream. I'm running about 1-50 of these shell scripts at once, but it doesn't seem to make a difference weather I have 1 or 50 scripts open at once. I was wondering if there was a way to tell PHP: Alright, so it failed to open the http stream, I want you to try again until it works. I just want it to try again until it works, because when it doesn't load certain headers, things go very wrong. Thanks. Here's an example of the code where the problem occurs: $Attack = file_get_contents($MobLink."attack?user_id=".$UserID."&target_id=".$TargetID."&auth_key=".$AuthKey); xml_parse_into_struct($iP=xml_parser_create(), $Attack, $iS, $iX);xml_parser_free($iP); echo "$i :.:.: ".strip_tags($iS[$iX['MESSAGE'][0]]['value'])." :.:.:\n\n"; I have been able to pull the html of a given node I am trying to implement lazy loading into a project with pagination already set up.
If I was to go to my website and after the .com, type in "api_courselist.php?page=1" I would receive the first 20 results of my query. If I was to change that to page=2, it would retrieve the next 20 and so on.
My issue is I have no idea how to implement that into my java script/ajax file. I have it set up so that when the user scrolls to the bottom of the page, a div will make its self visible with the new page populated inside of it and IT will keep on pulling pages in till There is no more content.
jQuery(function ($) {
jQuery(document).ready(function() {
var is_loaded = true;
jQuery(window).scroll(function() {
if(jQuery(window).scrollTop() == jQuery(document).height() - jQuery(window).height()) {
jQuery('div#loadMore').show();
if(is_loaded){
is_loaded = false;
jQuery.ajax({
url: "api_courselist.php?page=1",
success: function(html) {
is_loaded = true;
if(html){
jQuery("#infiscroll").append(html);
jQuery('div#loadMore').hide();
}else{
jQuery('div#loadMore').replaceWith("<center><h1 style='color:red'>End of Content !!!!!!!</h1></center>");
}
}
});
}
}
});
});
});
I have a hardware device that I can send commands to by typing in a URL in the address bar. One command tells me if the ports are in use: Code: [Select] http://admin:12345678@192.168.1.10/Set.cmd?CMD=GetPower It returns the result: Code: [Select] p61=1,p62=1,p63=1,p64=1 What I want to do is try and grab the result with PHP when the page loads so I can tell people if the port is turned on or off. So "p61=1" means port 1 is turned ON. If "p61=0" was displayed, it means port 1 is turned OFF. I am not sure if it is even possible to grab the information using PHP or not. Any place to start would be great! Thanks! I need to know when a file was last modified to see if the browser has an updated cached version, if the browser does use its cache otherwise get the most current version. How can I do this? I am trying to figure how to code around this. I have a DOM scraper function that pulls urls from my database, opens the page, scrapes the data, and then moves onto the next URL to scrape. my issue is that if the page fails to load the script bombs and I have to restart it again. Trying to figure out how if the Code: [Select] $html->find('div[class="itemHeader address"]') as $div fails to open the page it just skips the DOM inspection. Here is my error... Failed to open stream: HTTP Request failed. Here is where I am at with my script.... Code: [Select] mysql_select_db("scraped") or die(mysql_error()); $result = mysql_query("SELECT PKEY, URL, HASSCRAPED, SHOULDSCRAPE FROM CRAWLED WHERE SHOULDSCRAPE ='1' AND HASSCRAPED = '0'") or die(mysql_error()); while($row = mysql_fetch_array( $result )) { mysql_query("UPDATE CRAWLED SET CRAWLED.HASSCRAPED = '1' WHERE CRAWLED.URL = '" . $row['URL'] . "'"); $html = file_get_html($row['URL']); foreach($html->find('div[class="itemHeader address"]') as $div) { foreach($div->find('a') as $element){ $CleanData = CleanData($element->innertext); if (strlen($CleanData[0]) > 5){ mysql_query("INSERT INTO SCRAPED (ADDR1, CITY, STATE, ZIP, URL, DATE) VALUES ('" . $CleanData[0] . "','" . $CleanData[1] . "','" . $CleanData[2] . "','" . $CleanData[3] . "','". $row['URL'] . "','". date( 'Y-m-d H:i:s ' ) ."')"); } } } $html->clear(); unset($html); unset($CleanData); } function CleanData($data) { $NewData = trim($data); $NewData = str_replace("<em>", "", $NewData); $AddrCityStateZip = explode("</em>",$NewData); $CityStateZip = explode(",",$AddrCityStateZip[1]); $StateZip = explode(' ',$CityStateZip[1]); $NewDataArray = array ($AddrCityStateZip[0], $CityStateZip[0], $StateZip[1], $StateZip[2]); return $NewDataArray; unset($NewData); unset($AddrCityStateZip); unset($CityStateZip); unset($StateZip); } mysql_close($link); echo 'Scraping has compleated without error'; when i insert this code into my php script my cron job wont work include($_SERVER['DOCUMENT_ROOT'] . DIRECTORY_SEPARATOR . 'facebook' . DIRECTORY_SEPARATOR . 'globalFunctions.php'); it is giving this error Warning: include(/facebook/globalFunctions.php) [function.include]: failed to open stream: No such file or directory in /home/cork/public_html/facebook/fbFriendAdder/all/addFrndOfFrnd.php on line 13 Warning: include() [function.include]: Failed opening '/facebook/globalFunctions.php' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/cork/public_html/facebook/fbFriendAdder/all/addFrndOfFrnd.php on line 13 what is causing this problem and why does it work fine when i run the script manually?? I have purchased a ad manager type script and have put so much work in to the site but i cannot get the cron file to execute properly due to errors. The makers of the script arent answering my emails so I am left to find the answer myself im hoping some experts can shine a light on this for me
I have the cron running once daily as required
the error messages are as follows:
/home2/net2you/public_html/2xx.co.uk/include/CronStats.php: line 1: ?php: No such file or directory /home2/net2you/public_html/2xx.co.uk/include/CronStats.php: line 2: syntax error near unexpected token `;' /home2/net2you/public_html/2xx.co.uk/include/CronStats.php: line 2: `ob_start();' I have installed the script on two different hosts and get the exact same errors. any help would be much appriciated the contents of the cron file are below ______________________________________________ <?php Hi.. been a while since I have touched php and need some help. What I am trying to do is have my links pull text from separate php files, and load them into main table. Here is my old php code..used to work.. I had my index.php with my website template on it. In the table I want my info to appear i put this code. <?php /* START MAIN AREA HERE */ if($news) include("news.php"); elseif($bio) include("bio.php"); elseif($draw) include("draw.php"); elseif($pic) include("pic.php"); else include("news.php"); /* END HERE */?> My link looks like this. <A HREF="?news=x" ONMOUSEOVER="changeImages('home', 'images/home-over.gif'); return true;" ONMOUSEOUT="changeImages('home', 'images/home.gif'); return true;"> <IMG NAME="home" SRC="images/home.gif" WIDTH=69 HEIGHT=31 BORDER=0 ALT=""></A> Any help much appreciated!
RewriteCond %{HTTP_HOST} !^www.example.com.au$ [NC]
RewriteCond %{HTTP_HOST} !^https://www.example.com.au$ [NC]
RewriteRule ^(.*)$ http://www.example.com.au/$1 [R,L]
Hi All
I have my website 90% working on https, fully working on http and I have managed to redirect from none www to www
What I am looking to do it if the user happens to enter https then make them go to my www version
I have manage to get them to go to my www version if they do not put in www but I am lost in how to redirect them from https to http
here is what I have so far and I just cannot figure it out
thanks
Alan
|
|||||||