PHP - Super Slow Query & Super High Cpu
Hello again,
I got a problem with one of my pages that uses MySql. I didn't expect my site to shoot up into the 1000's per hour but it has but has destroyed the loading time of one of my pages. Below is the function that is super slow! Code: [Select] function get_top($timeframe='today', $time=10, $friends=true, $limit=5, $unique=true, $me_only=false){ if($friends===false){$q='';}else{ $q=' AND (username IN ('; // This array can have upto 500 items in it. global $friend_id_array, $me;$tarray=$friend_id_array; $tarray[]=$me['id']; $x= implode(',',$tarray);$q.=$x; $q.= '))'; } if($me_only){ $q=' AND username = \''.$me['id'].'\' '; } if($timeframe=='today'){ $q_timeframe='AND DATE(thedate) = DATE(NOW()) AND YEAR(thedate) = YEAR(NOW())'; // Checking the date could be slow }else if($timeframe=='week'){ $q_timeframe='AND WEEK(thedate) = WEEK(NOW()) AND YEAR(thedate) = YEAR(NOW())'; }else if($timeframe=='month'){ $q_timeframe='AND MONTH(thedate) = MONTH(NOW()) AND YEAR(thedate) = YEAR(NOW())'; }else if($timeframe=='all'){ $q_timeframe=''; } if($unique){$q_unique='GROUP BY username';} $query="SELECT MAX(score) as highscore, username, timeframe, thedate FROM other_click WHERE (timeframe = '$time') $q $q_timeframe $q_unique ORDER BY highscore DESC LIMIT 0, $limit"; // A very long query $result=@mysql_query($query)or die(mysql_error()); if(@mysql_num_rows($result)>0){ while($row=@mysql_fetch_array($result)){ // Maybe resorting the array makes it super slow $id_array[]=$row; } return $id_array;}else{return false;} } I've commented the lines which I think maybe making this page load slowly. I also thought the way the page is loaded. The page is called 6 times per user but with different settings eg, Friends only and only scores added today. Please help because this is crippling my site. Thank-you all Paul Similar TutorialsOk so I have a single mySQL database on a dedicated server hosted through hostgator.. In this database I have 1 table that is a very large table it has well over 900,000 rows on it and we anticipate another 8.1 million records to be added within another 6 months and from there expect the table to grow regularly. Right now my issue is and can only foresee it getting worse as the size gets larger. Is that my queries are running slower and slower. Sometimes even resulting in the DB crashing. So what I am trying to figure out is how do I speed this up. I've tried 101 ways to query the DB to speed it up and can't get it to load 10 results 10.. in less than 30 seconds. I know some of my issue is the fact that I need distinct results or unique ones rather. As there are several duplicate entries but I only need to pull out one of each for this particular function which is a search. So that said is there any advice one can give me to speed up my query results. I would offer up a sample of my current queries but its just your common query using distinct. I've got my own proprietary framework. I call it Brain. It uses composer, and installation is done with a single composer file, which is handy. Here's an example:
{ "name": "skunkbad/brain-test", "description": "My Awesome App", "license": "proprietary", "minimum-stability": "dev", "repositories": [ { "type": "hg", "url": "ssh://hg@bitbucket.org/skunkbad/brain" } ], "require": { "skunkbad/brain": "default" }, "scripts": { "post-install-cmd": "Brain\\Statics\\BrainInstallScript::run" } }Brain has it's own dependencies, which aren't all really dependencies, but I've added them all to its composer.json file so I get everything I want for a new project. Here is it's composer.json: { "name": "skunkbad/brain", "version": "1.0.0", "type": "framework", "license": "Proprietary", "authors": [ { "name": "Robert B Gottier", "email": "xXxXxXxXx@gmail.com", "homepage": "http://brianswebdesign.com", "role": "Developer" } ], "require": { "pimple/pimple" : "v2.0.0", "swiftmailer/swiftmailer": "v5.2.0", "dompdf/dompdf" : "0.6.1", "phpseclib/phpseclib" : "0.3.6", "stripe/stripe-php" : "v1.15.0", "firephp/firephp-core" : "dev-master", "doctrine/dbal" : "v2.4.2" }, "autoload": { "psr-4": { "Brain\\": "/", "App\\": "../../../App/" }, "classmap": [ "Statics/" ] } }In Windows terminal, I can run composer install or composer install --prefer-dist, and it seems that especially when the doctrine packages are being downloaded or cloned (whatever it does), its super slow. Some of them hang for so long that occasionally the terminal will give me a message indicating a particular package timed out (5 mins). So I was wondering if this is just the norm. Feels like I'm on dial-up, but my internet connection is Verizon FiOS (fiber optic). Here is an example of the time-out message I get from Composer: Failed to download doctrine/common from source: The process "git clone --no-checkout "git://github.com/doctrine/common.git" "C:\XXXxxx" && cd /D "C:\XXXxxx" && git remote add composer "git://github.com/doctrine/common.git" && git fetch composer" exceeded the timeout of 300 seconds. I timed my install, and composer/common actually hung for about 7 mins. Total time was 9.5 mins. Quote '31.100.x.x is not allowed to connect to this MySQL server in /home/member/insert.php on line 13 I asked a question about this a few weeks ago, and someone solved it saying that the server was blocking connections. Its confusing the hell out of me. I did as they said, and contacted my admin. they told me to fork over money, and it would be solved. I am a broke student, and dont have the cash anyway. What gets me is they said I had to be a paid member to have a sql database and connect to it with scripts on the server. But I can run a phpBB3 forum. it runs sql. I know it does. WTF? I know they are just trying to get money from me, but no matter how much I try i cant figure out how the forums connect. mine is Code: [Select] $username="a162_bento"; $password="Password123"; $database="a162_siteDB"; $localhost="host.com"; mysql_connect($localhost,$user,$password) I wish could figure out how xampp worked. If someone has time to help me I would really appreciate it. Im trying really hard but getting kick every time i get on the stage. Hi I'm currently experiencing problems with my super global outputs.
I'm using
$_SERVER['HTTP_X_MXIT_NICK']; Using: $file = fopen("file1.txt", "r") or exit("Unable to open file!"); I have multiple files (file1.txt, file2.txt, etc.) how can I edit this so it just takes ANY file that ends in .txt and opens it? Hi guys, I am trying out this code but it does not work, any idea what I am doing wrong? <?php include 'connect.php'; $tbl_name="users"; $sql="SELECT * FROM $tbl_name"; $result=mysql_query($sql); $count=mysql_num_rows($result); ?> <table width="400" border="0" cellspacing="1" cellpadding="0"> <tr> <td><form name="form1" method="post" action=""> <table width="400" border="0" cellpadding="3" cellspacing="1" bgcolor="#CCCCCC"> <tr> <td bgcolor="#FFFFFF"> </td> <td colspan="4" bgcolor="#FFFFFF"><strong>Delete multiple rows in mysql</strong> </td> </tr> <tr> <td align="center" bgcolor="#FFFFFF">#</td> <td align="center" bgcolor="#FFFFFF"><strong>Id</strong></td> <td align="center" bgcolor="#FFFFFF"><strong>Firstname</strong></td> <td align="center" bgcolor="#FFFFFF"><strong>Lastname</strong></td> <td align="center" bgcolor="#FFFFFF"><strong>Location</strong></td> <td align="center" bgcolor="#FFFFFF"><strong>Email</strong></td> <td align="center" bgcolor="#FFFFFF"><strong>Website</strong></td> </tr> <?php while($rows=mysql_fetch_array($result)){ ?> <tr> <td align="center" bgcolor="#FFFFFF"><input name="checkbox[]" type="checkbox" id="checkbox[]" value="<? echo $rows['id']; ?>"></td> <td bgcolor="#FFFFFF"><? echo $rows['userID']; ?></td> <td bgcolor="#FFFFFF"><? echo $rows['firstname']; ?></td> <td bgcolor="#FFFFFF"><? echo $rows['lastname']; ?></td> <td bgcolor="#FFFFFF"><? echo $rows['location']; ?></td> <td bgcolor="#FFFFFF"><? echo $rows['email']; ?></td> <td bgcolor="#FFFFFF"><? echo $rows['website']; ?></td> </tr> <?php } ?> <tr> <td colspan="5" align="center" bgcolor="#FFFFFF"><input name="delete" type="submit" id="delete" value="Delete"></td> </tr> <? if($delete){ for($i=0;$i<$count;$i++){ $del_id = $checkbox[$i]; $sql = "DELETE FROM $tbl_name WHERE id='$del_id'"; $result = mysql_query($sql); } if($result){ echo "<meta http-equiv=\"refresh\" content=\"0;URL=showusers2.php\">"; } } mysql_close(); ?> </table> </form> </td> </tr> </table> Hey all, I'm well aware of associative arrays (key/value pairs) and how you can index them in an array like this: $character = array (name=>"John", occupation=>"Programmer", age=>22, "Learned language "=>"PHP" ); Then it makes sense to iterate through the key/value pairs of the array using foreach: foreach ( $character as $key=>$val ){ print "$key = $val<br>"; } However, I was watching a video tutorial where he created a $_SESSION array and then created an array associative array: $_SESSION['cart'] = array(); [/PHP] Then he added a function add_to_cart: function add_to_cart($id){ if(isset($_SESSION['cart'][$id])){ $_SESSION['cart'][$id]++; return true; } else { $_SESSION['cart'][$id] = 1; return true; } return false; } Now that $id variable holds an id converted to integer from the dataabse. So if first item is clicked, id holds a value of integer 1. When I see this: $_SESSION['cart'][$id] I see an array that holds two indexes, each index containing an associative array: [[array() => ''], [1 => '']]. So at index 0 of $_SESSION is [array() => '']. But then he uses the foreach iterator like this: function total_items($cart){ $items = 0; if(is_array($cart)){ foreach($cart as $id => $qty){ $items += $qty; d } } return $items; } Now I'm very confused. As you can see in that foreach method, it says that $id (and its corresponding value) is an associative array of $cart array, not $_SESSION array. I don't see how that happened. I thought $cart and id$ were distinct indexes of $_SESSION. I don't see how $id is a key of the $cart array. Thanks for any explanation to clear my confusion. Hi, I currently have an if, elseif, else program that starts off with Code: [Select] if( $v_name == "" || $v_msg == "" ) echo "something" how do I turn this into a switch? but the very next elseif I have Code: [Select] elseif( strcspn( $_REQUEST['msg'], '0123456789' ) == strlen( $_REQUEST['msg'] ) ) echo "something" is it possible to turn this into a switch with 2 different strings like that to evaluate? Hi, I am having a problem with a part of my query: "SELECT SQL_CALC_FOUND_ROWS jobs.id AS jobId, enquiries.id AS profId, users.username, entity_details.name AS entityName, branches.name AS branch, orderNumber, entity_vat_numbers.vatCode, entity_account_numbers.accountNumber, (SELECT symbol FROM parts_trading, currencies WHERE parts_trading.enquiryRef = enquiries.id AND parts_trading.sellingCurrency = currencies.id LIMIT 1) AS symbol, (SELECT SUM(quantity) FROM parts_trading WHERE enquiryRef = enquiries.id) - COALESCE((SELECT SUM(quantity) FROM invoices_out_reference WHERE jobRef = jobs.id), 0) AS differentPartsCount, (SELECT SUM(quantity*sellingNet) FROM parts_trading WHERE parts_trading.enquiryRef = enquiries.id) - COALESCE( (SELECT SUM(invoices_out.net) FROM invoices_out, invoices_out_reference WHERE invoices_out_reference.invoiceRef = invoices_out.id AND invoices_out_reference.jobRef = jobs.id ) , 0) AS netRemaining, (SELECT SUM(quantity*sellingVat) FROM parts_trading WHERE parts_trading.enquiryRef = enquiries.id) - COALESCE( (SELECT SUM(invoices_out.vat) FROM invoices_out, invoices_out_reference WHERE invoices_out_reference.invoiceRef = invoices_out.id AND invoices_out_reference.jobRef = jobs.id ) , 0) AS vatRemaining, (SELECT SUM(quantity*(sellingNet+sellingVat)) FROM parts_trading WHERE parts_trading.enquiryRef = enquiries.id) - COALESCE( (SELECT SUM(invoices_out.net+invoices_out.vat) FROM invoices_out, invoices_out_reference WHERE invoices_out_reference.invoiceRef = invoices_out.id AND invoices_out_reference.jobRef = jobs.id ) , 0) AS totalRemaining FROM jobs, enquiries, users, branches, entity_details LEFT JOIN entity_account_numbers ON entity_details.id = entity_account_numbers.entityRef LEFT JOIN entity_vat_numbers ON entity_details.id = entity_vat_numbers.entityRef WHERE enquiries.traderRef = users.id AND jobs.enquiryRef = enquiries.id AND users.branchRef = branches.id AND (SELECT SUM(quantity) FROM parts_trading WHERE enquiryRef = enquiries.id) - COALESCE((SELECT SUM(quantity) FROM invoices_out_reference WHERE jobRef = jobs.id), 0) > 0 AND enquiries.entityRef = entity_details.id" I know for certain that it is the part: "AND (SELECT SUM(quantity) FROM parts_trading WHERE enquiryRef = enquiries.id) - COALESCE((SELECT SUM(quantity) FROM invoices_out_reference WHERE jobRef = jobs.id), 0) > 0" That is causing the problem. Unfortunately this is a vital part of my query used for not displaying rows that have quantities matching the sum of what is in the other table . Is there are more efficient way to get the same result than that part of the query? This topic has been moved to MySQL Help. http://www.phpfreaks.com/forums/index.php?topic=342696.0 Hi All, I have a form that enables the user to know events that occur in a specific city. Most of the time they get the results in a few seconds, despite the fact that many rows in various tables might be searched for detailed infos about the events in that city. However, this form partly relies on a table against which a MySQL event regularly does some operations. If the user uses the form while the event is being executed, they have to wait up to 30 seconds before getting the results, and sometimes only part of the html page is generated. Is there a way to avoid this lengthy waiting time ? I am wondering whether or not the best would be putting the site on maintenance while the event is being executed, with a "please come back in a few minutes" message. Thanks in advance for your help. would Java handle it faster? can PHP be made to handle it as fast? I really need help. check out my website and search cars for sale, then any distance and then 37919 zip. it takes too long and there are only 3351 cars on there now. before, when it only had about 2200 cars it was really fast. only about a 1000 cars made it slow. what could it be? nabacar.com It was returning results in about 15 seconds or less with 2200 cars. Now it is over a minute with only 1000 cars on it. I will have 150,000 on it when I get this sorted. http://nabacar.com some java was written to speed things up, but it didn't seem to help. Is it the server or the code causing this? I try to get a high score system to work. The problem is to write data to the txt file: The file adder.php contains: <?php if(array_key_exists('name',$_GET)==true and array_key_exists('score',$_GET)==true) { $name=$_GET['name']; $score=$_GET['score']; $file=fopen('highscore.txt','a'); fwrite($file,$name.';'.$score.';'); fclose($file); } header("Location: index.php"); // Replace index.php with the name (and, if not in the same folder, path) // of the page that will display the scores (see below) ?> I use this website: http://members.multimania.nl, so it looks like http://members.multimania.nl/mine/adder.php?name=winner&score=99 What am I doing wrong? I successfully load a page by simple_html_dom.php (developed in simplehtmldom.sourceforge.net) as $html = file_get_html('externalpage'); But sometimes this make a high load on CPU and the page does not load for a long time (probably due to the external site server). How can I skip the process when it is not normal to avoid high CPU usage? I have just did this php code for the high-low game but the $tries variable always goes down to 0 when I refresh..Can someone help me to return the $tries to browser with the url? Thank you
<html>
<?php function High_Low($guess){
ok so i want to order by high to low on the field price but i don't know how to do this. i can order low to high by using ORDER BY `Price` ASC but how can i order high to low? Hello, I have what I hope to be a very simple question.. I have built a game in flash and for my high score table I simply want to request the high scores in reverse.. i.e the lowest score first.. here is my php.. please help, thanks in advance !! :- <?php include("sqldialog.php"); $table = $_POST['tab']; $recept = SelectDatas($table, "name, score, dategame", "id > 0", "score desc"); if ($recept) { $i = 0; foreach ($recept as $rec) { $i++; if ($i==1) { $result .= "pseudo$i=".$rec['name']."&score$i=".$rec['score']."&dategame$i=".$rec['dategame']; } else { $result .= "&pseudo$i=".$rec['name']."&score$i=".$rec['score']."&dategame$i=".$rec['dategame']; } } echo $result; } else { echo "BAD, $pseudo, $score, $dategame,$ipclient"; } ?> I think I've coded myself into a corner. I have numerous items entered with measurement extremes - highest and lowest. Now I want somebody to be able to search for an item that could match a specific length. So I could have several items that are entered into a database like this: Code: [Select] item_id | length | high_low_param -----------+-----------+---------------------- 1234 | 12 cm | high -----------+-----------+--------------------- 1234 | 6 cm | low If somebody searches for an item that is 8 cm long, I want the results to be item 1234 and any other items that span that range. I'm not sure how to do this? array_fill ? I don't know. I don't want to slow down the database too much, and I don't want to have to put an entry for every single length each item could be. Any suggestions? Hi, i'm new here and have a straightforward question. On a server I use I have this script and we use it to force the download of mp3 files on a single click of a link. (Avoid opening in an internet audio buffering plug in or application) Code: [Select] <?php $filename = $_GET['file']; // required for IE, otherwise Content-disposition is ignored if(ini_get('zlib.output_compression')) ini_set('zlib.output_compression', 'Off'); // addition by Jorg Weske $file_extension = strtolower(substr(strrchr($filename,"."),1)); if( $filename == "" ) { echo "<html><title>Download Script</title><body>ERROR: download file NOT SPECIFIED. USE force-download.php?file=filepath</body></html>"; exit; } elseif ( ! file_exists( $filename ) ) { echo "<html><title>Download Script</title><body>ERROR: File not found. USE force-download.php?file=filepath</body></html>"; exit; }; switch( $file_extension ) { case "pdf": $ctype="application/pdf"; break; case "exe": $ctype="application/octet-stream"; break; case "zip": $ctype="application/zip"; break; case "doc": $ctype="application/msword"; break; case "xls": $ctype="application/vnd.ms-excel"; break; case "ppt": $ctype="application/vnd.ms-powerpoint"; break; case "gif": $ctype="image/gif"; break; case "png": $ctype="image/png"; break; case "mp3": $ctype="audio/mpeg3"; break; case "jpeg": case "jpg": $ctype="image/jpg"; break; default: $ctype="application/force-download"; } header("Pragma: public"); // required header("Expires: 0"); header("Cache-Control: must-revalidate, post-check=0, pre-check=0"); header("Cache-Control: private",false); // required for certain browsers header("Content-Type: $ctype"); // change, added quotes to allow spaces in filenames, by Rajkumar Singh header("Content-Disposition: attachment; filename=\"".basename($filename)."\";" ); header("Content-Transfer-Encoding: binary"); header("Content-Length: ".filesize($filename)); readfile("$filename"); exit(); ?> I recently got an email from my service provider quoting the following Quote Hello, I apologize, but I was forced to suspend the script /home/mezerik/********/forcedownload.php as it was causing a high load on the server, and due to it affecting all of the other accounts on the system, I forced to take immediate action for the health of the server. Unfortunately I do not have any specific recommendations for this script, however, in general, adding some sort of caching mechanism, where the script does not need to generate a new page with every request, helps to lower the over load that a script will cause. Likely the original author or support group of the software that you are using will be able to help you to understand how to add something of this nature. If you reply back to this with your IP address (http://www.******.com/ip.shtml) we will be more than happy to go ahead enable HTTP access for you, so that you can safely work on the script without it causing further issues. Please let us know how you would like to proceed. I am not sure what is wrong with the script and if it is insecure to the server and should be edited or removed. I have a class that I created, halfway down my page I call a function from the class which prints some code and also changes a variable inside the class. It's a search class basically it prints out the search results and saves the total amount of results into the variable. I want to display the amount of results above the results like so:- echo $Search->results; $Search->GetResults('foo'); The problem is that $Search->results will be equal to NULL until the variable is updated in the function GetResults. Hope that makes sense, am I going about this wrong? |