PHP - Getting Script To Insert 40,000 Rows Without Stopping

Hi, I have an app that access a MySQL database via a php script.
I noticed that INSERT was returning error but the data was being recorded successfully.
I finally traced the problem to be related with the line 
echo $conn->affected_rows;   

For some reason when it is an SQL INSERT it returns -11 but as I said the INSERTS executes successfully.

The app requesting the service sends in sequence:

char* txtSQL[]={"INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)",
                            "INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)",
                            "INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)",
                            "INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)",
                            "UPDATE activity set value=333 where value=130",
                            "SELECT sum(value) from activity where mac='a9c4952de6b4'",
                            "DELETE from activity WHERE value=333"};

I set a monitor to check what was being returned and got:

query=INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)
HttpResponse:  -   httpResponseCode: -11
query=INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)
HttpResponse:  -   httpResponseCode: -11
query=INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)
HttpResponse:  -   httpResponseCode: -11
query=INSERT INTO activity (mac,jd,date,time,area,type,value) VALUES ('a9c4952de6b4',2458454,'2018-12-01','10:22','Area0002','h',130)
HttpResponse:  -   httpResponseCode: -11
query=UPDATE activity set value=333 where value=130
HttpResponse: 4 *   httpResponseCode: 201   -------------------------------------->As can be seem the INSERTS above returned error but worked OK
query=SELECT sum(value) from activity where mac='a9c4952de6b4'
HttpResponse: 1379 *   httpResponseCode: 200
query=DELETE from activity WHERE value=333
HttpResponse: 4 *   httpResponseCode: 201

The  PHP script do ing the job i s as be low :

<?php
include('connection.php');

//these are just in case setting headers forcing it to always expire 
header('Cache-Control: no-cache, must-revalidate');

error_log(print_r($_POST,TRUE));

if( isset($_POST['query']) && isset($_POST['key']) ){                                   //checks if the tag post is there and if its been a proper form post
  header('Content-type: application/x-www-form-urlencoded');
  if($_POST['key']==$SQLKEY){                                                           //validates the SQL key
    $query=urldecode($_POST['query']);
    if(get_magic_quotes_gpc()){     //check if the worthless pile of crap magic quotes is enabled and if it is, strip the slashes from the query
      $query=stripslashes($query);
    }
    $conn = new mysqli($DB_ADDRESS,$DB_USER,$DB_PASS,$DB_NAME);    //connect

    if($conn->connect_error){                                                           //checks connection
      header("HTTP/1.0 400 Bad Request");
      echo "ERROR Database Connection Failed: " . $conn->connect_error, E_USER_ERROR;   //reports a DB connection failure
    } else {

      $result=$conn->query($query);                                                     //runs the posted query                        

      if($result === false){
        header("HTTP/1.0 400 Bad Request");                                             //sends back a bad request error
        echo "Wrong SQL: " . $query . " Error: " . $conn->error, E_USER_ERROR;          //errors if the query is bad and spits the error back to the client
      } else {
        if (strlen(stristr($query,"SELECT"))>0) {                                       //tests if it's a SELECT statement
          $csv = '';                                                                                // bug fix Undefined variable: csv
          while ($fieldinfo = $result->fetch_field()) {
            $csv .= $fieldinfo->name.",";
          }
          $csv = rtrim($csv, ",")."\n";
//********************************          echo $csv;                                                                    //prints header row
          $csv = '';

          $result->data_seek(0);
          while($row = $result->fetch_assoc()){
            foreach ($row as $key => $value) {
              $csv .= $value.",";
            }
            $csv = rtrim($csv, ",");                                                 //."\n";
          }
          echo $csv;                                                                    //prints all data rows
        } else {
          header("HTTP/1.0 201 Rows");
          echo $conn->affected_rows;       //if the query is anything but a SELECT, it will return the number of affected rows (INSERT IS  RETURNING -11)
        }
      }
      $conn->close();                                          //closes the DB
    }
  } else {
     header("HTTP/1.0 400 Bad Request");
     echo "-Bad Request";                                       //reports if the secret key was bad
  }
} else {
        header("HTTP/1.0 400 Bad Request");
        echo "*Bad Request";
}
?>

Any help will be much appreciated.
Thanks

Paulo Borges

So when I execute this it'll go for about 130k rows then give a 500 error. Any ideas how to avoid this and get it to complete?

Code: [Select]
$i=10000000;
while($i<=99999999)
  {
  $public = $i;
  $value = rand(10000000, 99999999); 
  mysql_query("INSERT INTO yummy_table (public, value) VALUES('$public', '$value' ) ") or die(mysql_error()); 
  $i++;
  }
echo "done";

Hello, is it possible to insert somehow information from textarea to mysql?
For example i have 3 lines in text area:
John Tacker is a bad guy
Tom Tacker has a big nose
Ted Tacker is the same person as John Tacker
and i want to insert them in table's different rows

Hi again all,

Why does the foreach loop im doing, put the array values from collection, into seperate table rows rather than 1 row per whole array?



    <?php
        class registration{                    
            public $fields = array("username", "email", "password");
            public $data = array();
            public $table = "users";
            public $dateTime = "";
            public $datePos = 0;
            public $dateEntryName = "date";
            public $connection;
            
            function timeStamp(){
                return($this->dateTime = date("Y-m-d H:i:s"));
            }
            
            function insertRow($collection){      
                    //HERE
                    foreach($this->fields as $row => $value){
                        mysql_query("INSERT INTO $this->table ($value)
                        VALUES ('$collection[$row]')");
                    }
                    
                    mysql_close($this->connection->connectData);
            }
            
            function validateFields(){            

                $this->connection = new connection();
                $this->connection->connect(); 
                        
                foreach($this->fields as $key => $value){
                    array_push($this->data, $_POST[$this->fields[$key]]);
                }
                $this->dateTime = $this->timeStamp();
                array_unshift($this->data, $this->dateTime);                
                array_unshift($this->fields, $this->dateEntryName);   
                
                foreach($this->data as $value){
                    echo "$value";
                }
                
                $this->insertRow($this->data);
            }
        }
        
        $registration = new registration();
        $registration->validateFields();
    ?>




I end up with 3 rows

row 1: username
row 2: email
row 3 : password

rather than row 1:username email password.

Hi i have this upload script which works fine it uploads image to a specified folder and sends the the details to the database.
but now i am trying to instead make a modify script which is Update set so i tried to change insert to update but didnt work can someone help me out please
this my insert image script which works fine but want to change to modify instead
Code: [Select]
<?php
mysql_connect("localhost", "root", "") or die(mysql_error()) ;
mysql_select_db("upload") or die(mysql_error()) ;

// my file the name of the input area on the form type is the extension of the file
//echo $_FILES["myfile"]["type"];

//myfile is the name of the input area on the form 
$name = $_FILES["image"] ["name"]; // name of the file
$type = $_FILES["image"]["type"]; //type of the file
$size = $_FILES["image"]["size"]; //the size of the file
$temp = $_FILES["image"]["tmp_name"];//temporary file location when click upload it temporary stores on the computer and gives it a temporary name
$error =array(); // this an empty array where you can then call on all of the error messages
$allowed_exts = array('jpg', 'jpeg', 'png', 'gif'); // array with the following extension name values
$image_type = array('image/jpg', 'image/jpeg', 'image/png', 'image/gif'); // array with the following image type values
$location = 'images/'; //location of the file or directory where the file will be stored
$appendic_name = "news".$name;//this append the word [news] before the name so the image would be news[nameofimage].gif

// substr counts the number of carachters and then you the specify how how many you letters you want to cut off from the beginning of the word example drivers.jpg it would cut off dri, and would display vers.jpg
//echo $extension = substr($name, 3);

//using both substr and strpos, strpos it will delete anything before the dot in this case it finds the dot on the $name file deletes and + 1 says read after the last letter you delete because you want to display the letters after the dot. if remove the +1 it will display .gif which what we want is just gif
$extension = strtolower(substr($name, strpos ($name, '.') +1));//strlower turn the extension non capital in case extension is capital example JPG will strtolower will make jpg
// another way of doing is with explode
// $image_ext strtolower(end(explode('.',$name))); will explode from where you want in this case from the dot adn end will display from the end after the explode

$myfile = $_POST["myfile"];

     if (isset($image)) // if you choose a file name do the if bellow
       {
       
       // if extension is not equal to any of the variables in the array $allowed_exts error appears
        if(in_array($extension, $allowed_exts) === false )
       {
         $error[] = 'Extension not allowed! gif, jpg, jpeg, png only<br />'; // if no errror read next if line
       }
        // if file type is not equal to any of the variables in array $image_type error appears
        if(in_array($type, $image_type) === false)
       {
          $error[] = 'Type of file not allowed! only images allowed<br />';     
       }
       
        // if file bigger than the number bellow error message
        if($size > 2097152)
       {
          $error[] = 'File size must be under 2MB!';     
       }
       
       // check if  folder exist in the server
        if(!file_exists ($location))
       {
          $error[] = 'No directory ' . $location. ' on the server Please create a folder ' .$location;     
       }
            
     }
     // if no error found do the move upload function
       if (empty($error)){
           if (move_uploaded_file($temp, $location .$appendic_name))
           {
             // insert data into database first are the field name teh values are the variables you want to insert into those fields appendic is the new name of the image
mysql_query("INSERT INTO image (myfile ,image)
 VALUES ('$myfile', '$appendic_name')") ;
           exit();
           }
         }
       else
          {
        foreach ($error as $error)
           {
               echo $error;
           }
        }
      
     
  
//echo $type;
?>

Hi,   I am developing a web application which will be comprised of three columns: (1) Search form, (2) Image gallery, (3) Image details. The site is going to be more like html framesets, but research reveals that framesets are not favourable due to many reasons. I however found an alternative that I want to use instead of html framesets http://www.ironspide...smartframes.htm   I thought it would be easy for me to add rows which should serve as a page header, but I was wrong. I tried adding     document.write('<frameset rows="17%,*" frameborder="no">');   to this part of the code, but row apeared in a wrong place at the bottom:

// Writes frameset

document.write('<frameset rows="17%,*" frameborder="no">');
document.write('<frameset cols="25%,75%" frameborder="no">');
document.write('<frame src="' + menuURL + '" name="menu">');
document.write('<frame src="' + contentURL + '" name="content" marginheight="15" marginwidth="50" scrolling="yes">');
// -->

The original script for the index.html page is as follows:

<!--
/* SmartFrames - Author: Robert Darrell - http://www.ironspider.ca/ */
// Defines variables
// REPLACE menu.htm WITH YOUR MENU PAGE
// REPLACE page1.htm WITH YOUR INITIAL CONTENT PAGE
menuURL = "menu.htm";
contentURL = "page1.htm";
reloadURL = parent.document.URL;
orphanURL = reloadURL.substring(reloadURL.indexOf('?')+1, reloadURL.indexOf('&'));
frameName = reloadURL.substring(reloadURL.indexOf('&')+1, reloadURL.length);

// Reassigns variable according to frame name passed by orphan
if (frameName == "menu") {
menuURL = orphanURL; 
} else if (frameName == "content") {
contentURL = orphanURL;
}

// Writes frameset
document.write('<frameset cols="25%,75%" frameborder="no">');
document.write('<frame src="' + menuURL + '" name="menu">');
document.write('<frame src="' + contentURL + '" name="content" marginheight="15" marginwidth="50" scrolling="yes">');
// -->
</script>

<!-- Use the noframes section to help search engines index your site -->
<noframes>THIS WEBSITE IS ABOUT...<br><br>
<a href="menu.htm">Site Menu</a>
</noframes>

</frameset>

I hope someone will help.   Regards.   joseph  

Ok I'm trying to insert multiple rows by using a while loop but having problems. At the same time, need to open a new mysql connection while running the insert query, close it then open the previous mysql connection.

I managed to insert multiple queries before using a loop, but for this time, the loop does not work? 
I think it is because I am opening another connection... yh that would make sense actually?

Here is the code:


$users = safe_query("SELECT * FROM ".PREFIX."user");
       while($dp=mysql_fetch_array($users)) { 
          
   $username = $dp['username'];
 $nickname = $dp['nickname'];
 $pwd1 = $dp['password']; 
 $mail = $dp['email'];
 $ip_add = $dp['ip'];
 $wsID = $dp['userID'];
 $registerdate = $dp['registerdate'];
 $birthday = $dp['birthday'];
 $avatar = $dp['avatar'];
 $icq = $dp['icq']; 
 $hp = $dp['homepage'];
 
echo $username." = 1 username only? :(";
      

// ----- Forum Bridge user insert -----

$result = safe_query("SELECT * FROM `".PREFIX."forum`");
$ds=mysql_fetch_array($result);
$forum_prefix = $ds['prefix'];
define(PREFIX_FORUM, $forum_prefix);
define(FORUMREG_DEBUG, 0);

$con = mysql_connect($ds['host'], $ds['user'], $ds['password']) or system_error('ERROR: Can not connect to MySQL-Server');
$condb = mysql_select_db($ds['db'], $con) or system_error('ERROR: Can not connect to database "'.$ds['db'].'"');
                
include('../_phpbb_func.php');
$phpbbpass = phpbb_hash($pwd1);
$phpbbmailhash = phpbb_email_hash($mail);
$phpbbsalt = unique_id();

safe_query("INSERT INTO `".PREFIX_FORUM."users` (`username`, `username_clean`, `user_password`, `user_pass_convert`, `user_email`, `user_email_hash`, `group_id`, `user_type`, `user_regdate`, `user_passchg`, `user_lastvisit`, `user_lastmark`, `user_new`, `user_options`, `user_form_salt`, `user_ip`, `wsID`, `user_birthday`, `user_avatar`, `user_icq`, `user_website`) VALUES ('$username', '$username', '$phpbbpass', '0', '$mail', '$phpbbmailhash', '2', '0', '$registerdate', '$registerdate', '$registerdate', '$registerdate', '1', '230271', '$phpbbsalt', '$ip_add', '$wsID', '$birthday', '$avatar', '$icq', '$hp')");
if (FORUMREG_DEBUG == '1') {
echo "<p><b>-- DEBUG -- : User added: ".mysql_affected_rows($con)."<br />";
       echo "<br />-- DEBUG -- : Query used: ".end($_mysql_querys)."</b></p><br />";
      
       $result = safe_query("SELECT user_id from ".PREFIX_FORUM."users WHERE username = '$username'");
       $phpbbid = mysql_fetch_row($result);
safe_query("INSERT INTO `".PREFIX_FORUM."user_group` (`group_id`, `user_id`, `group_leader`, `user_pending`) VALUES ('2', '$phpbbid[0]', '0', '0')");
safe_query("INSERT INTO `".PREFIX_FORUM."user_group` (`group_id`, `user_id`, `group_leader`, `user_pending`) VALUES ('7', '$phpbbid[0]', '0', '0')");
mysql_close($con);
}

         include('../_mysql.php');        
         mysql_connect($host, $user, $pwd) or system_error('ERROR: Can not connect to MySQL-Server');
mysql_select_db($db) or system_error('ERROR: Can not connect to database "'.$db.'"');
       
       }



So I need to be able to insert these rows using the while loop.. how can I do this?

I really appreciate any help.

Not sure if this is the right forum, but I have a database that I will need to populate with a large number of rows (2000+).

I have written a PHP script that uploads individual entries.

Is it possible to use something like a spreadsheet where I can set out the rows/columns as they will appear in the database, and then upload in one go rather than uploading each row individually?

Thanks for any observations and/or help.

I'm new to PHP/MySQL and have been searching high and low for a solution to this...

When using my PHP script to insert a row into the database it errors out but does not give me a specific error. When I use the MySQL command line client I can insert the first, second, third, or any number of rows without a problem.

What I don't understand is, once I have inserted a row using the command line client, my PHP script will insert rows without issue.

So my question is, what is it about the first row being inserted that makes it so special? One of the columns in the table is an AUTO_INCREMENT field, so i'm guessing that has something to do with it?

I've used this on like three other scripts I don't know why it won't insert now 

<?php
date_default_timezone_set('America/Los_Angeles');
$timestamp = date('H:m:s m.d');  
$admin = '24.68.214.97';
$visitor_ip = $_SERVER['REMOTE_ADDR'];
$host="mysql"; // Host name
$username="15557_test"; // Mysql username
$password="**********"; // Mysql password
$db_name="15557_test"; // Database name
$tbl="announce"; // Table name
mysql_connect("$host", "$username", "$password")or die("cannot connect to server");
mysql_select_db("$db_name")or die("cannot select DB"); 
$body = $_POST["body"];

if ($visitor_ip == $admin) {
mysql_query("INSERT INTO $tbl (body, date) VALUES ('$body', '$timestamp')"); //right here, i can echo $body and see what I wrote, but I can't insert into $tbl
}
else {
die("no");
}
?>
<meta http-equiv="REFRESH" content="0;url=http://testchan.dcfilms.org/board-b.php">

i have got this script which displays all folders within a folder but what i want to do now is get it to display a random picture for each of the folders that it is showing in the grid. i know how to display a random image using array_rand() but i don't know how i would implement it.

Code: [Select]
<form name="Gallery" method="post">
<?php
$path = "gallery_files/gallery/";
$dir_handle = @opendir($path) or die("Unable to open folder");
 
echo "<table cellspacing='15'>";
echo "<tr>";
while (false !== ($folder = readdir($dir_handle))) {
if($folder == "pictures.php")
continue;
if($folder == ".")
continue;
if($folder == "..")
continue;

echo ($x % 6 == 0) ? "</tr><tr>" : "";
echo "<td><a href='pictures/?folder=$folder'><img src='path/to/random/image' style='height:auto;width:110px;' alt='$folder'></a><br /><a href='pictures/?folder=$folder'>$folder</a></td>";
  $x++;
}
echo "</tr>";
echo "</table>";
closedir($dir_handle);
?>
</form>

 Hello,

I'm trying to write my first database using SQL and PHP, without much luck, I'm afraid.  After inserting a new record (for which I didn't have a problem), I wanted to automatically generate a username which is the person's first name (they'll then be able to change it later).  Trying to use Dreamweaver as a model, I've come up with this:

 $updateSQL = sprintf("UPDATE users SET username=%s",
                       GetSQLValueString($_POST['first_name'], "text"))

Well...it does update the username, but it does so on ALL of the records as opposed to the one that was just inserted.  Any help would be appreciated...

Thank you.

Well the subject line is pretty explicit.  I found this script that uploads a picture onto a folder on the server called images, then inserts the the path of the image on the images folder onto a VACHAR field in a database table. 
Code: [Select]




<?php

//This file inserts the main image into the images table.

//address error handling

ini_set ('display_errors', 1);
error_reporting (E_ALL & ~E_NOTICE);





//authenticate user
//Start session
session_start();

       
        //Connect to database
require ('config.php');

//Check whether the session variable id is present or not. If not, deny access.
if(!isset($_SESSION['id']) || (trim($_SESSION['id']) == '')) {
header("location: access_denied.php");
exit();
}

        else{ 







// Check to see if the type of file uploaded is a valid image type

function is_valid_type($file)

{

     // This is an array that holds all the valid image MIME types

     $valid_types = array("image/jpg", "image/jpeg", "image/bmp", "image/gif");

 

     if (in_array($file['type'], $valid_types))

         return 1;

     return 0;

}

 

// Just a short function that prints out the contents of an array in a manner that's easy to read

// I used this function during debugging but it serves no purpose at run time for this example

function showContents($array)

{

     echo "<pre>";

     print_r($array);

     echo "</pre>";

}

 

// Set some constants

 

// This variable is the path to the image folder where all the images are going to be stored

// Note that there is a trailing forward slash

$TARGET_PATH = "images/";

 

// Get our POSTed variable

$image = $_FILES['image'];

 

// Sanitize our input

$image['name'] = mysql_real_escape_string($image['name']);

 

// Build our target path full string.  This is where the file will be moved to

// i.e.  images/picture.jpg

$TARGET_PATH .= $image['name'];

 

// Make sure all the fields from the form have inputs

if ( $image['name'] == "" )

{

     $_SESSION['error'] = "All fields are required";

     header("Location: member.php");

     exit;

}

 

// Check to make sure that our file is actually an image

// You check the file type instead of the extension because the extension can easily be faked

if (!is_valid_type($image))

{

     $_SESSION['error'] = "You must upload a jpeg, gif, or bmp";

     header("Location: member.php");

     exit;

}

 

// Here we check to see if a file with that name already exists

// You could get past filename problems by appending a timestamp to the filename and then continuing

if (file_exists($TARGET_PATH))

{

     $_SESSION['error'] = "A file with that name already exists";

     header("Location: member.php");

     exit;

}

 

// Lets attempt to move the file from its temporary directory to its new home

if (move_uploaded_file($image['tmp_name'], $TARGET_PATH))

{

     // NOTE: This is where a lot of people make mistakes.

     // We are *not* putting the image into the database; we are putting a reference to the file's location on the server

     $sql = "insert into images (member_id, image_cartegory, image_date, image) values ('{$_SESSION['id']}', 'main', NOW(), '" . $image['name'] . "')";

     $result = mysql_query($sql) or die ("Could not insert data into DB: " . mysql_error());

     header("Location: images.php");

echo "File uploaded";

     exit;

}

else

{

     // A common cause of file moving failures is because of bad permissions on the directory attempting to be written to

     // Make sure you chmod the directory to be writeable

     $_SESSION['error'] = "Could not upload file.  Check read/write persmissions on the directory";

     header("Location: member.php");

     exit;

}







        } //End of if session variable id is not present.
 


?> 


The script seems to work fine because I managed to upload a picture which was successfully inserted into my images folder and into the database.
Now the problem is, I can't figure out exactly how to write the script that displays the image on an html page. I used the following script which didn't work.


Code: [Select]

//authenticate user
//Start session
session_start();

       
        //Connect to database
require ('config.php');

$sql = mysql_query("SELECT* FROM images WHERE member_id = '".$_SESSION['id']."' AND image_cartegory = 'main' ");
$row = mysql_fetch_assoc($sql);
$imagebytes = $row['image'];
header("Content-type: image/jpeg");
print $imagebytes;







Seems to me like I need to alter some variables to match the variables used in the insert script, just can't figure out which.  Can anyone help??

Hello, I am very new to both PHP and MYSQL but I am eager to learn.

What I want to do is write a PHP script that will parse an XML file and insert the results into a MYSQL database. I've read a bunch of guides online and I think I have the parser structure down. I created a parser using xml_parser_create(). As I understand it, I need to create functions to handle each of the three events, startTag, endTag, and contents. This is where I am not sure what to do. What do I need to tell these functions to do when these events are triggered? I know I need to connect to the MYSQL database at some point but I do not know when.

If anyone could point me in the right direction, I'd appreciate it.

Hi Guys, having one of them "why wont it work" nightmares whilst doing some updates on a web page someone else has coded, its a simple php and mysql insert string that has some validation on but no matter what I do it wont add a new property, it updates fine, i have only added three new fields to the original source files, property_county, property_size, property_status, it should have been a simple job but has got me stumped, anyone got any suggestions?

The script uses three files,
properties.php - which is the page which holds the form
class.properties.php - which contains the functions - this is where i think the problem is....
and
property_form.php - which is just the form, this is fine, just the above two that seem to be causing the problem!!

any suggestions would be greatly appreciated.
Thanks
Matt