|
PHP - Need Help Parsing / Extracting Links From Log Files
I am looking for some help with extracting links from log files, as it is a pain to do this manually (which I do right now). I basically have some log files which I need to check for ERROR messages and copy and paste the found URL's into another text file.
My log file format looks like this: Code: [Select] INFO <11 Feb 2012 00:00:23,822> <index> <D2> <Processing URL : http://www.domain1.com/> INFO <11 Feb 2012 00:00:23,842> <index> <D4> <Indexed: http://www.domain2.com/> <Time:146 msecs> INFO <11 Feb 2012 00:00:23,842> <index> <D4> <Processing URL : http://www.domain3.com/> ERROR <11 Feb 2012 00:00:23,924> <index> <D1> <http://www.domain4.org/operas/2003-2004/mourning/composer.aspx: > org.apache.commons.httpclient.HttpRecoverableException: org.apache.commons.httpclient.HttpRecoverableException: Error in parsing at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1965) at org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMethodBase.java:2659) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1093) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:674) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:529) at com.searchblox.scanner.http.HTTPScanner.b(Unknown Source) at com.searchblox.scanner.http.HTTPScanner.scan(Unknown Source) at com.searchblox.scanner.http.HTTPScanner.work(Unknown Source) at com.searchblox.scanner.Scanner.run(Unknown Source) at java.lang.Thread.run(Unknown Source) INFO <11 Feb 2012 00:00:23,968> <index> <D5> <Indexed: http://domain6.com/~cdobie/kearnsindex.htm> INFO <11 Feb 2012 00:00:23,968> <index> <D5> <Indexed: http://domain7.com/~cdobie/kearnsindex.htm> INFO <11 Feb 2012 00:00:32,988> <index> <D1> <Processing URL : http://www.domain8.com/> INFO <11 Feb 2012 00:00:33,072> <index> <D5> <Indexed: http://www.domain9.com/> <Time:128 msecs> INFO <11 Feb 2012 00:00:33,072> <index> <D5> <Processing URL : http://www.domain10.com/> ERROR <11 Feb 2012 00:00:33,116> <index> <D2> <http://www.domain11.com/: Connection timeout> org.apache.commons.httpclient.HttpConnection$ConnectionTimeoutException at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:736) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:661) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:529) at com.searchblox.scanner.http.HTTPScanner.b(Unknown Source) at com.searchblox.scanner.http.HTTPScanner.scan(Unknown Source) at com.searchblox.scanner.http.HTTPScanner.work(Unknown Source) at com.searchblox.scanner.Scanner.run(Unknown Source) at java.lang.Thread.run(Unknown Source) INFO <11 Feb 2012 00:00:33,154> <index> <D1> <Indexing http://www.domain12.com/ ...> INFO <11 Feb 2012 00:00:33,159> <index> <D1> <http://www.domain13.com/ - Last-Modified date: Sat Feb 11 00:00:33 CET 2012> ERROR <11 Feb 2012 00:00:33,207> <index> <D6> <http://www.domain14.com/: Connection timeout> Now what I am after is some piece of code which basically saves the http://domain.com/ part to a text file IF the line starts with ERROR. There are many different error reasons, so the strings are all different at the start and at the end, so maybe you know a way to open a log file, look out for the word ERROR at the beginning of a line and if that's the case, either save the whole line to another text file or if possible just the domain part (which would be even more great) If possible, please post a fully functional code block, as I am extremely bad with anything that has to do with regex, opening and closing files etc. Your help would be greatly appreciated I attached a sample log file to this post in case it helps (same as the lines above) Similar TutorialsIs it possible to extract all links from a website (not a single webpage) by php? I am asking about the general idea, as I wish to customize: e.g. from a specified directory and certain domains only. Thanks my issue is each time a user selcts a category or sub category, the excel file stored in server is fetched, read and output its contents and then closed again.....its quite cumbersome cos what if multiple users were logged into the system doing this same process....server load wud b definitely high......so,i would like to parse excel files and store it as an array so that data can be retrieved from the array rather than opening the file,fetching data and closing it, then again opening it,fetching it and closing and so on.....i am thinking of reading the data into an array and then unset the array if a different category or subcategory is selected....is this solution even possible? Thanks to Zane I am to where I am now. But I need some more help. I have this code which parses out a div from within the page gotten. Works great. Code: [Select] $html = file_get_dom('http://steamcommunity.com/profiles/76561198010758659/games?tab=all'); echo $html('div#game_440 h5', 0)->getPlainText(); Now I need the code to parse from multiple links, instead of using one link like it does now I would like to be able to post the links, one per line and have it figure it out If I could have it grab the links from a text file, it would be even better. I finally, also would like to output the link of which the parsed details came from with the parsed details. This topic has been moved to PHP Regex. http://www.phpfreaks.com/forums/index.php?topic=317036.0 Hi Everyone, I'm semi-new to php and trying to create a site where I have only one includes file for my navigation (main_nav.inc) and 2-3 includes files (main_template.inc) that act as templates. I then call the the nav and the template in my index.php file. Everything was working fine until I decided I should use php have my links appear active depending on which page the user is on. Everything works totally fine if I insert the code for my nav (including both the html and php) into my main_template.inc file but when I make that code into it's own includes file (main_nav.inc), I get an error. I believe the issue is with using both single and double quotes in my code and wrapping that code in single quotes in my main_nav.inc file. Here is the code from my main_nav.inc file: <?php $main_nav = ' <div id="mainNav"><ul id="mainNav"> <li><a href="index.php" <?php if ($page=='index.php') { ?> class="active" <?php } ?>>Welcome</a></li> <li><a href="healthcare.php" <?php if ($page=='healthcare.php') { ?> class="active" <?php } ?>>Healthcare</a></li> </ul></div> '; // end main_nav ?> I think that since the code is wrapped in single quotes, I can only use double quotes in my code. This is the only difference I can figure out between inserting the code into the template directly (which works perfectly fine) and inserting it as a separate inc file. When I use the main_nav.inc file I get the following error: Parse error: syntax error, unexpected T_STRING in /home/content/k/a/l/kalilaks3/html/includes/main_nav.inc on line 11 I tried changing the quotes around ($page=='index.php') to ($page=="index.php") but that causes the browser to literally read the name of the link a class="active">Welcome I also tried variations of escaping quotes such as ($page==\"index.php\") and even escaping the other quotes and wrapping everything in double quotes as opposed to single quotes. Here is my code for my main_template.inc file <?php $page = basename($_SERVER['SCRIPT_NAME']); $page_title = "Welcome"; include_once ("includes/main_nav.inc"); // sets variable include_once ("home_template.inc"); // sets all variables ?> And here is my index.php file: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta content="healthcare" name="keywords"></meta> <meta content="healthcare" name="description"></meta> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title><?php echo $page_title; ?></title> <link rel = "stylesheet" href = "_css/layout.css"></link> <script type="text/javascript" src="scripts/script.js"></script> </head> <body> <div id = "header"><div id = "globalHeader"> <?php echo $main_nav; ?> </div></div> </body> </html> Any help or suggestions would be greatly appreciated!!! I would love to get this figured out so that I only have to create my navigation once for all my sites! Hello I have a simple question about file handling... Is it possible to list all files in directories / subdirectories, and then read ALL files in those dirs, and put the content of their file into an array? Like this: array: [SomePath/test.php] = "All In this php file is being read by a new smart function!"; [SomePath/Weird/hello.txt = "Hello world. This is me and im just trying to get some help!";and so on, until no further files exists in that rootdir. All my attempts went totally crazy and none of them works... therefore i need to ask you for help. Do you have any ideas how to do this? If so, how can I be able to do it? Thanks in Advance, pros So far I have managed to create an upload process which uploads a picture, updates the database on file location and then tries to upload the db a 2nd time to update the Thumbnails file location (i tried updating the thumbnails location in one go and for some reason this causes failure) But the main problem is that it doesn't upload some files Here is my upload.php
<?php
include 'dbconnect.php';
$statusMsg = '';
$Title = $conn -> real_escape_string($_POST['Title']) ;
$BodyText = $conn -> real_escape_string($_POST['ThreadBody']) ;
// File upload path
$targetDir = "upload/";
$fileName = basename($_FILES["file"]["name"]);
$targetFilePath = $targetDir . $fileName;
$fileType = pathinfo($targetFilePath,PATHINFO_EXTENSION);
$Thumbnail = "upload/Thumbnails/'$fileName'";
if(isset($_POST["submit"]) && !empty($_FILES["file"]["name"])){
// Allow certain file formats
$allowTypes = array('jpg','png','jpeg','gif','pdf', "webm", "mp4");
if(in_array($fileType, $allowTypes)){
// Upload file to server
if(move_uploaded_file($_FILES["file"]["tmp_name"], $targetFilePath)){
// Insert image file name into database
$insert = $conn->query("INSERT into Threads (Title, ThreadBody, filename) VALUES ('$Title', '$BodyText', '$fileName')");
if($insert){
$statusMsg = "The file ".$fileName. " has been uploaded successfully.";
$targetFilePathArg = escapeshellarg($targetFilePath);
$output=null;
$retval=null;
//exec("convert $targetFilePathArg -resize 300x200 ./upload/Thumbnails/'$fileName'", $output, $retval);
exec("convert $targetFilePathArg -resize 200x200 $Thumbnail", $output, $retval);
echo "REturned with status $retval and output:\n" ;
if ($retval == null) {
echo "Retval is null\n" ;
echo "Thumbnail equals $Thumbnail\n" ;
}
}else{
$statusMsg = "File upload failed, please try again.";
}
}else{
$statusMsg = "Sorry, there was an error uploading your file.";
}
}else{
$statusMsg = 'Sorry, only JPG, JPEG, PNG, GIF, mp4, webm & PDF files are allowed to upload.';
}
}else{
$statusMsg = 'Please select a file to upload.';
}
//Update SQL db by setting the thumbnail column to equal $Thumbnail
$update = $conn->query("update Threads set thumbnail = '$Thumbnail' where filename = '$fileName'");
if($update){
$statusMsg = "Updated the thumbnail to sql correctly.";
echo $statusMsg ; }
else
{ echo "\n Failed to update Thumbnail. Thumbnail equals $Thumbnail" ; }
// Display status message
echo $statusMsg;
?>
And this does work on most files however it is not working on a 9.9mb png file which is named "test.png" I tested on another 3.3 mb gif file and that failed too? For some reason it returns the following Updated the thumbnail to sql correctly.Updated the thumbnail to sql correctly. Whereas on the files it works on it returns REturned with status 0 and output: Retval is null Thumbnail equals upload/Thumbnails/'rainbow-trh-stache.gif' Failed to update Thumbnail. Thumbnail equals upload/Thumbnails/'rainbow-trh-stache.gif'The file rainbow-trh-stache.gif has been uploaded successfully. Any idea on why this is? Folks, I want to extract certain portion form URL. Exmaple: Quote http://abc.com/this-is-test.html Output should be Quote this-is-test Another Example Quote http://abc.com/this-is-yet-another-test.html Output should be Quote this-is-yet-another-test I am not sure how it can be done with preg_match() and regex or something like that... Can someone help me with this please? Cheers Natasha I have an RSS feed, with alot of data of lottery numbers. the feed itself is : http://www.alllotto.com/rss/NY/latest.rss The item i am interested in is: Quote <item> <title>Take 5</title> <description>2011-08-27: 4, 10, 16, 17, 31</description> <guid isPermaLink="false">USA-New-York-Take-5-2011-08-27</guid> </item> How could I output those to varibles? thanks Joe I'm *attempting* to write a script that will take a paste of data from a user and when it's submitted it will drop certain parts of the data into databases and be available for recall later. I have the MySQL sorta out I think but I can't test until I get this part done. Basically people will paste a copied paste (CTRL A, CTRL C and go to my form and just hit CTRL V). It will contain a bunch of data I want to drop in the database but in different areas, like the following example the items in bold are the items I want to grab. First Name: Lincoln Last Name: Coe Address: 1234 Easy St, Perfectville, PV 00000 Telephone: 0000000000 HI guys, I'm quite new to php, and I'm really struggling to get this right. I just cant get it to work properly. Im trying to extract email from a list of URLS. I have currently got it to work with 1 URL at a time, but I am needing to as to how I can pass multiple URLS at once, either from a csv or just pasting them into the input. any help would be much appriciated. here is my current code: Quote <?php $the_url = isset($_REQUEST['url']) ? htmlspecialchars($_REQUEST['url']) : ''; ?> <form method="post"> Please enter full URL of the page to parse (including http://):<br /> <textarea name="url" cols="100" rows="10"><?php echo $the_url; ?></textarea> <br /> <input type="submit" value="Get Emails" /> </form> <?php if (isset($_REQUEST['url']) && !empty($_REQUEST['url'])) { // fetch data from specified url $text = file_get_contents($_REQUEST['url']); } elseif (isset($_REQUEST['text']) && !empty($_REQUEST['text'])) { // get text from text area $text = $_REQUEST['text']; } // parse emails if (!empty($text)) { $res = preg_match_all( "/[a-z0-9]+([_\\.-][a-z0-9]+)*@([a-z0-9]+([\.-][a-z0-9]+)*)+\\.[a-z]{2,}/i", $text, $matches ); if ($res) { foreach(array_unique($matches[0]) as $email) { echo $email . "<br />"; } } else { echo "No emails found."; } } ?> Hey guys i have a array i made and im wanting to grab a random value out of the array. The only way i can see to do this looking at the list of array functions is i need to random a key first? and then return the value assigned to the random key i just generated? I only see a function that lets me do the opposite. array_search() so now that i have generated a random key how do I grab just the value from the array using the random key. or is there a function i can just random value? $explode_names = explode(" ", $planet_names); $random_key_name = (array_rand($explode_names, 1)); echo $random_key_name; $example "Q: Example? A:" Let's assume that I'd like to display $example but exclude "Q:" and "A:", how would I do that? hey guys i need to extract all the IP's of a string and loop them for more operations but for some reason i only get the first one <?php $string = '80.37.14.13 80.37.14.14 80.37.14.15'; preg_match("/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/", $string, $matches); foreach ($matches as $ip){ echo "$ip<br>"; } ?> The string is not really seperated by spaces ... it can actualy be messy and have ips rapped arround a lot of code. The regex works because i do get the first one ... What did i miss? I usually only do design work, but a client wanted a log in system to his website, so I decided to do it. I set everything up correctly, and users can sign up, login, and log out. However, he wants to be informed when a user logs into his site. So say (user x) wants logs in, my client wants to receive an email with all of (user x)'s account information. How do I pull a row from mysql based on the login information provided? My database is has 7 columns: user, pass, name, address, state, phone, email Hi there I need to extract data from some XML. I have found a few sites that explain that part to me, however there is a section of the data which I need to extract and am not sure how to go about it. Below is an extract of the XML, the section I am trying to extract is highlighted in red (it is essentially a text message being sent I am trying to get the contents of the sms/text message): <usa_smpp_host>THTTP</usa_smpp_host> <short_message>id:660352946 sub:001 dlvrd:001 submit date:1102081032 done date:1102081035 stat:DELIVRD err:000 text:Test SMS message</short_message> <dest_addr_npi>1</dest_addr_npi> Any assistance would be appreciated. Have played around quite a bit but have not managed to figure it out yet. thanks Hi! I have .swf files that have images in them. I can view them with swf decompiler on my computer but i need to have this functionality on my website. Is there any way to extract images from .swf file with php? I've only been studying PHP for a week now and have come across this problem which I'm sure there is a simple answer to, but I just can't figure it out and would appreciate some help. I've spent far too long on this minor issue already! I have a table which contains a list of products, in this case books, which stores the date when each new item is added. I have a query that then searches through this table and extracts the 6 most recent additions. Here is the code I have so far: Code: [Select] $sql = mysql_query("SELECT * FROM products ORDER BY date_added DESC LIMIT 6"); $productCount = mysql_num_rows($sql); // count the output amount if ($productCount > 0) { while($row = mysql_fetch_array($sql)){ $id = $row["id"]; $title = $row["title"]; $author = $row["author"]; $price = $row["price"]; $date_added = strftime("%b %d, %Y", strtotime($row["date_added"])); $dynamiclist .= //My table showing the products } } else { $dynamicList = "There are currently no Books listed in this store"; } This works well when I need to display the 5 most recent products in a normal table fashion, one below another, on the page. However, I want to display the products in a more personalised order. For example display the newest item in one section of the page and 3rd newest in another. What is the best way to select an individual row from a query? To extract the details of the 2nd newest item to display in the header, for example. Thanks for your help. hey guys im wondering how you would add or takeaway a hour from a timestamp in a database when trying to select it please?
thank you
|
|||||||