|
PHP - Csv Import Script.. Improvements?
I've written the script below to import a csv file into the database. I've tried to make this as secure as possible by sanitizing and validating the data, but is there anything else I could do to improve it? I'm happy with the way it works so far.
All I can think to add is a meme check via php but I can't find a way to check whether it's specifically a csv file (other than just checking the extension but that's not secure)
<?php
if ($_FILES[csv][size] > 0) {
//get the csv file
$file = $_FILES[csv][tmp_name];
$handle = fopen($file,"r");
//set the row counter
$csv_row = 0;
//loop through the csv file and insert into database
do {
// start the import
if ($data[0]) {
//add 1 the row counter
$csv_row++;
//clear errors
$errors = '';
//sanitize the inputs
$firstName = addslashes($data[0]);
$lastName = addslashes($data[1]);
$email = addslashes($data[2]);
//validate the inputs - set $errors as 'YES' if any fail
if(!ctype_alpha($firstName)){
$errors = 'YES';
}
if(!ctype_alpha($lastName)){
$errors = 'YES';
}
if(!filter_var($email, FILTER_VALIDATE_EMAIL)){
$errors = 'YES';
}
//check whether there were any errors in the validation
if($errors!=='YES'){
//insert into database
$stmt = $db->prepare("INSERT INTO test (firstName,lastName,email) VALUES (?,?,?)");
$stmt->bind_param('sss',$firstName,$lastName,$email);
$stmt->execute();
}
//there was an error, store the affected row number in the error array
else
{
$error_report[] = "There was an error on row $csv_row";
} //end error check
} //end single row import
} while ($data = fgetcsv($handle));
if($csv_row>0){
$success = 'YES';
}
} // end csv import
?>
Edited by paddyfields, 11 June 2014 - 01:06 PM. Similar Tutorialsinclude "connect.php"; if(isset($_POST['submit'])) { $filename=$_POST['filename']; $handle = fopen("$filename", "r"); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $import="UPDATE isc_products(prodavailability,prodinvtrack,prodcurrentinv) values('$data[1]','$data[2]','$data[3]') where vendor_id = '($data[0])' "; mysql_query($import) or die(mysql_error()); } fclose($handle); print "Import done"; } else { print "<form action='inv_update.php' method='post'>"; print "Type file name to import:<br>"; print "<input type='text' name='filename' size='20'><br>"; print "<input type='submit' name='submit' value='submit'></form>"; } ?> I am building this script to import some data via a csv that has three fields. The where statement gives me this error - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(prodavailability,prodinvtrack,prodcurrentinv) values('35','1','35') where vendo' at line 1. I'm know my syntax is flawed but can't find a solution to this.... any one point me in the right direction? I know just enough php to get myself in trouble!! lol I am looking for a way to upload a csv file to a website and have it populate a database. Does anyone know of a way to do this. The users of the site are novice website users and I am a novice php coder. Any help would be hugely appreciated. Thanks Hi! I hope somebody can help me what im do wrong. i have checked that the data from the file is in $source_file but nothing imports to the database Code: [Select] <?php include('config.php'); include('opendb.php'); if(isset($_POST['upload'])) { $source_file = @$_POST['userfile']; //$source_file = fopen('http://localhost/test/upload/test.csv', 'r'); $target_table = 'foretag'; function csv_file_to_mysql_table($source_file, $target_table, $max_line_length=10000) { if (($handle = fopen("$source_file", "r")) !== FALSE) { $columns = fgetcsv($handle, $max_line_length, ","); foreach ($columns as $column) { $column = str_replace(".","",$column); } $insert_query_prefix = "INSERT INTO $target_table (".join(",",$columns).")\nVALUES"; while (($data = fgetcsv($handle, $max_line_length, ";")) !== FALSE) { while (count($data)<count($columns)) array_push($data, NULL); $query = "$insert_query_prefix (".join(",",quote_all_array($data)).");"; mysql_query($query); } fclose($handle); } } function quote_all_array($values) { foreach ($values as $key=>$value) if (is_array($value)) $values[$key] = quote_all_array($value); else $values[$key] = quote_all($value); return $values; } function quote_all($value) { if (is_null($value)) return "NULL"; $value = "'" . mysql_real_escape_string($value) . "'"; return $value; } } include('closedb.php'); echo "<br>done<br>"; ?> I have the following form (attached to this post as a Thumbnail).
I want to make it user-friendly, specifically, bring it up to the modern web standards, namely, I am looking for things that that I think are modern and are user friendly and are great when considered by themselves, but together they may even conflict. And I need some help figuring out what will work best for my situation and what is technically possible.
Right now the user enteres fields into the form, into the editable fields on the left. Then user can save the fields by clicking Save, or click Del to delete the row. User can Add a new row at the bottom.
The things I think will improve the form a * implement a double-click feature on the editable fields to make them editable, and save them automatically upon `blur` event * user can use `Tab` key to navigate from one `<input>` field to the next, normally, as before, and edit values inside. * user can save all fields entered so far, and restore them via some mechanism (i.e. either to web page or to clipboard, and restore via direct paste for example, or file upload..) * Maybe I can use DataTables, but I have never used it so I don't know what could be done Can you help suggest form or UI improvements for the current form that I have? I may also have some questions on first steps or first examples of implementing particular ones (possibly making a separate post for those) Attached Files form.png 7.94KB 0 downloads All criticism/suggestions/improvements appreciated Registration.php Code: [Select] <?php $con = mysql_connect("localhost","","") or die(mysql_error()); mysql_select_db('Users'); if(isset($_COOKIE['ID_my_site'])) { $cookie_username = mysql_real_escape_string(filter_input(INPUT_COOKIE, 'ID_', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $cookie_password = sha1($_COOKIE['Key_']); $cookie_check = mysql_query("SELECT * FROM Users WHERE username = '$cookie_username'") or die(mysql_error()); $cookie_results = mysql_fetch_array($cookie_check); if ($cookie_password == $cookie_results['Password']) { echo "<div id=\"login_msg\">You are already logged on. Redirecting...</div><br />" && header("location:/index.php"); } } if(isset($_POST['submit'])) { $Username = mysql_real_escape_string(filter_input(INPUT_POST, 'Username', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $Email = mysql_real_escape_string(filter_input(INPUT_POST, 'Email', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $Password = sha1($_POST['Password']); $Password2 = sha1($_POST['Password2']); if (!$Username | !$Email | !$Password | !$Passord2) { echo "<div id=\"error_msg\">You did not complete all of the required fields, please try again.</div><br />"; } if ($Password != $Password2) { echo "<div id=\"error_msg\">Your passwords do not match, please try again.</div><br />"; } $check_username = mysql_query("SELECT * FROM Users WHERE (Username = $Username)"); $result_username = mysql_fetch_row($check_username); $check_email = mysql_query("SELECT * FROM Users WHERE (Email = $Email)"); $result_email = mysql_fetch_row($check_email); if ($result_username == true) { echo "<div id=\"error_msg\">The Username: '$Username', already exists. Please enter another username.</div><br />"; } if ($result_email == true) { echo "<div id=\"error_msg\">The Email Adress: '$Email', is already in our Database.</div><br />"; } $sql = "INSERT INTO Users (Id, Username, Email, Password) VALUES ('', '$Username','$Email','$Password')"; $add_member = mysql_query($sql) or die(mysql_error()); if (mysql_query($add_member)) { $week = time() + 604800; setcookie(ID_, $_POST['Username'], $week); setcookie(Key_, $_POST['Password'], $week); echo "<div id=\"login_msg\">Successfully added to our Database.</div><br />" && header ("location:/Login.php"); } else { echo "<div id=\"error_msg\">Invalid input.</div><br />"; } } ?> Login.php Code: [Select] <?php include("db.php"); if(isset($_COOKIE['ID_my_site'])) { $cookie_username = mysql_real_escape_string(filter_input(INPUT_COOKIE, 'ID_', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $cookie_password = sha1($_COOKIE['Key_']); $cookie_check = mysql_query("SELECT * FROM Users WHERE username = '$cookie_username'") or die(mysql_error()); $cookie_results = mysql_fetch_array($cookie_check); if ($cookie_password == $cookie_results['Password']) { echo "<div id=\"login_msg\">You are already logged on. Redirecting...</div><br />" && header("location:/index.php"); } } if(isset($_POST['submit'])) { $Username = mysql_real_escape_string(filter_input(INPUT_POST, 'Username', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $Password = sha1($_POST['Password']); if (!$Username | !$Password) { echo "<div id=\"error_msg\">You did not complete all of the required fields, please try again.</div><br />"; } $sql = "SELECT * FROM Users WHERE (Username, Password) = ('$Username', '$Password')"; $db_check = mysql_num_rows($sql) or die(mysql_error()); if (mysql_query($db_check)) { $week = time() + 604800; setcookie(ID_, $cookie_username, $week); setcookie(Key_, $cookie_password, $week); echo "<div id=\"login_msg\">Successfully Logged In.</div><br />" && header ("location:/index.php"); } elseif (($Username | $Password) != $db_check) { echo "<div id=\"error_msg\">Invalid username or password, please try again.</div><br />"; } } ?> Logout.php Code: [Select] <?php include("db.php"); if(isset($_COOKIE['ID_my_site'])) { $cookie_username = mysql_real_escape_string(filter_input(INPUT_COOKIE, 'ID_', FILTER_SANITIZE_FULL_SPECIAL_CHARS)); $cookie_password = sha1($_COOKIE['Key_']); $cookie_check = mysql_query("SELECT * FROM Users WHERE username = '$cookie_username'") or die(mysql_error()); $cookie_results = mysql_fetch_array($cookie_check); if ($cookie_password != $cookie_results['Password']) { header("location:/login.php"); } else { $past = time() - 604800; setcookie(ID_, gone, $past); setcookie(Key_, gone, $past); echo "<div id=\"error_msg\">Sucessfully logged out. Good Bye!</div><br />" && header ("location:/login.php"); } } ?> Hello, I'm having trouble with the following code $file = file_get_contents("database.sql"); $queries = explode(";",$file); for ($i=0,$c=count($queries);$i<$c;$i++){ mysql_query($queries[$i],$mlink); } the problem is the sql file I'm trying to import has ; alllll over the place, not just at the end of the queries, so the queries are not getting inputted correctly. I could see a way to fix this if each query was on a single line, but some of them are over 100 lines each. How do I make sure it only explodes using the ; at the end of query? and not the ; that are actually in the tables etc. thanks Having a find and replace nightmare and dumbfounded by a way to correct this. I am exporting a large inner join SQL statement in CSV format. I then want to be able to edit and replace values as needed from the other tables. ex: Orlando = 1 Miami = 2 Jacksonville = 3 Well if I want to go through and replace everything 'Orlando' with the id 1, so my inner joins dont break. I am doing this in the following method. // read the file $file = file_get_contents('data.csv'); // replace the data $result = $db->query('SELECT id_baseprices, baseprices FROM baseprices'); while ($line = $db->fetchNextObject($result)) { $file = str_replace(',"'.$line->baseprices.'",', ',"'.$line->id_baseprices.'",', $file); } $result = $db->query('SELECT id_cities, cities FROM cities'); while ($line = $db->fetchNextObject($result)) { $file = str_replace(',"'.$line->cities.'",', ',"'.$line->id_cities.'",', $file); } // write the file file_put_contents('revamped.csv', $file); Problem with the above is it replaces "Orlando" everywhere with 1. I really only want this to happen inside column 8. I am lost and google must hate me tonight, so thank you for any thoughts or comments. I have a table I am trying to import to mySQL I am having trouble assigning the right variable which represents a column header to the correct column header in the mySQL table. <?php /********************************/ /* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/ /* Edit the entries below to reflect the appropriate values /********************************/ $databasehost = "xxx"; $databasename = "xxx"; $databasetable = "names"; $databaseusername ="xxx"; $databasepassword = "xxx"; $fieldseparator = "\t"; $lineseparator = "\n"; $csvfile = "AddEnt2.txt"; $con = @mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error()); @mysql_select_db($databasename) or die(mysql_error()); $fcontents = file ('AddEnt2.txt'); # expects the csv file to be in the same dir as this script for($i=0; $i<sizeof($fcontents); $i++) { $line = trim($fcontents[$i],'\t'); // '\t' for tab delimeted $arr = explode('\t', $line); // '\t' for tab delimeted $name=implode("','",$arr); $PA=implode("','",$line); # if your data is comma separated # instead of tab separated # change the '\t' above to ',' $sql = "insert into $databasetable values ('$PA','$name')"; $sql = str_replace("''", mysql_escape_string("NULL"), $sql); $sql = str_replace("' '", mysql_escape_string("NULL"), $sql); mysql_query($sql); echo $sql ."<br>\n"; if(mysql_error()) { echo mysql_error() ."<br><b><font color=red>\n</b>"; } } ?> Hi, i have a script to import a csv into a database but some of my product descriptions have ' in the names, this causes mysql to error, how can i get around this? here is my code. Code: [Select] if(isset($_POST['submit'])) { $i=0; $fname = $_FILES['sel_file']['name']; $chk_ext = explode(".",$fname); if(strtolower($chk_ext[1]) == "csv") { mysql_query("truncate table products") ; $filename = $_FILES['sel_file']['tmp_name']; $handle = fopen($filename, "r"); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { if($i > 0) { $sql = "INSERT into products(prod_id,prod_name,prod_description,prod_cat,prod_sub_cat,tax,prod_price,active) values('$data[0]','$data[1]','$data[9]','$data[14]','$data[15]','$data[19]','$data[35]','$data[65]')"; mysql_query($sql) or die(mysql_error()); } $i++; } fclose($handle); echo "Successfully Imported<br>"; } else { echo "Invalid File"; } } Hi, I have two xml files, one has 750 products, and the other one has 900 products (including product informations, stock information, sizes and photos) so shortly one product in xml file is like this;
product_name: x t-shirt And my tables are like this:
Product (product_id (numerical), product_code (alphanumerical), name) (So that I need to insert one product in my db, and take the id, and insert the xml-photos-data with the correct product_id to photos table, same goes with the stocks table) So I m thinking about the easiest and most healthy way to import this amount of data, and I need to update it daily or weekly (planning to do it by cron jobs for now). I have found some solutions but since I have not done this before could not decide which is the most secure way. Need to add that I can have other xml files from different companies to import later on, so I think I should code for long term. Thank you in advance for your help. Edited December 17, 2018 by RommeoHello all - Currently I have functions that will parse a csv file and insert the values into a database. That code is working. However, I want to take it a step further and ONLY insert selected fields from the csv file. For example, I have 14 rows in the csv and of those I only want to import values into the database of the rows that I define. So, gName, fName and id -> take those and pump those values into the database. I'm sure this a possible but not sure of the direction I should take. Any help on my problem would be excellent! Here's the code: Code: [Select] csv_file_to_mysql_table('tim 3.csv', 'growers'); function csv_file_to_mysql_table($source_file, $target_table, $max_line_length = 10000) { if (($handle = fopen("$source_file", "r")) !== FALSE) { $columns = fgetcsv($handle, $max_line_length, ","); foreach ($columns as &$column) { $column = str_replace(".", "", $column); } $insert_query_prefix = "INSERT INTO $target_table (" . join(",", $columns) . ")\nVALUES"; while (($data = fgetcsv($handle, $max_line_length, ",")) !== FALSE) { while (count($data) < count($columns)) array_push($data, NULL); $query = "$insert_query_prefix (" . join(",", quote_all_array($data)) . ");"; mysql_query($query) or die(mysql_error()); } fclose($handle); } } function quote_all_array($values) { foreach ($values as $key => $value) if (is_array($value)) $values[$key] = quote_all_array($value); else $values[$key] = quote_all($value); return $values; } function quote_all($value) { if (is_null($value)) return "NULL"; $value = "'" . mysql_real_escape_string($value) . "'"; return $value; } Here's some sample data: Code: [Select] Long,Lat,id,gName,fId,fName,fldId,fldName,fldAcers,featureID,objID,fInsu,fFSA,fBid -82.38306422,40.38439870,2,Norris| Tim,3,Hallinan,4,H1 - 10.0,9.900,1,1, , ,29 -82.22279060,40.42760230,2,Norris| Tim,4,Holt,5,Ho1,11.50,1,1, , ,30 -82.21917211,40.42838107,2,Norris| Tim,4,Holt,6,Ho2,15.10,1,1, , ,31 -82.21710436,40.42454375,2,Norris| Tim,4,Holt,7,Ho3,17.90,1,1, , ,32 -82.21833571,40.42367314,2,Norris| Tim,4,Holt,7,Ho3,17.90,2,2, , ,32 -82.21595345,40.42200315,2,Norris| Tim,4,Holt,9,Ho4,9.100,1,1, , ,34 -82.36538195,40.37711617,2,Norris| Tim,5,Home,10,H1 - 36.3A,36.20,1,1, , ,35 -82.36159625,40.37804250,2,Norris| Tim,5,Home,11,H2 - 3.9A,3.900,1,1, , ,36 -82.36196265,40.38085335,2,Norris| Tim,5,Home,12,H3 - 6A,7.000,1,1, , ,37 -82.41030962,40.38997625,2,Norris| Tim,2,Kenyon,13,K10-17A,17.00,1,1, , ,38 -82.38584288,40.35998635,2,Norris| Tim,2,Kenyon,14,K11-14A,14.52,1,1, , ,39 -82.41644710,40.37927258,2,Norris| Tim,2,Kenyon,15,K3 - 18.2A,18.20,1,1, , ,40 -82.40744700,40.37788250,2,Norris| Tim,2,Kenyon,16,K4 - 26.2A,26.12,1,1, , ,41 -82.40390048,40.37467350,2,Norris| Tim,2,Kenyon,17,K5 - 8.9A,8.874,1,1, , ,42 -82.38605720,40.36920760,2,Norris| Tim,2,Kenyon,18,K6 - 19.3A,18.22,1,1, , ,43 -82.39960597,40.38844915,2,Norris| Tim,2,Kenyon,19,K7 - 18.4A,18.37,1,1, , ,44 -82.39515150,40.39498212,2,Norris| Tim,2,Kenyon,20,K8 - 7.3A,7.324,1,1, , ,45 -82.38817795,40.39458225,2,Norris| Tim,2,Kenyon,21,K9 - 40.4A,40.28,1,1, , ,46 -82.38836722,40.39172487,2,Norris| Tim,2,Kenyon,21,K9 - 40.4A,40.28,2,2, , ,46 -82.40294059,40.35565598,2,Norris| Tim,10,Lane,2,1,6.900,1,1, , ,47 -82.40579843,40.35399913,2,Norris| Tim,10,Lane,22,2,7.200,1,1, , ,48 -82.38322795,40.37619695,2,Norris| Tim,6,Leach,23,L1 - 1.5,1.500,1,1, , ,49 -82.38334655,40.38060737,2,Norris| Tim,6,Leach,24,L2 - 17.6,17.60,1,1, , ,50 -82.38032235,40.38354262,2,Norris| Tim,7,Robinson,25,R1 - 7.5,7.400,1,1, , ,51 -82.39919331,40.35353322,2,Norris| Tim,9,Shorey,26,SH1,5.757,1,1, , ,52 -82.40033216,40.35715336,2,Norris| Tim,9,Shorey,28,SH2,13.39,1,1, , ,54 -82.37072642,40.31789197,2,Norris| Tim,8,Stream,27,S1,17.80,1,1, , ,53 I have mysql exported in the following sql format: Code: [Select] - CREATE TABLE IF NOT EXISTS `pma_history` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `username` varchar(64) COLLATE utf8_bin NOT NULL DEFAULT '', `db` varchar(64) COLLATE utf8_bin NOT NULL DEFAULT '', `table` varchar(64) COLLATE utf8_bin NOT NULL DEFAULT '', `timevalue` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `sqlquery` text COLLATE utf8_bin NOT NULL, PRIMARY KEY (`id`), KEY `username` (`username`,`db`,`table`,`timevalue`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='SQL history for phpMyAdmin' AUTO_INCREMENT=3761 ; -- -- Dumping data for table `pma_history` -- INSERT INTO `pma_history` (`id`, `username`, `db`, `table`, `timevalue`, `sqlquery`) VALUES (2, 'root', 'information_schema', 'FILES', '2010-03-04 19:26:33', 'SELECT * FROM `FILES`') Just as an example anyway. What I need to be able to do, is import this using PHP. Does anyone know of a suitable class for doing this? I could make my own, but I cant quite work out the best way to seperate the queries out? Thanks Hi all So... I am creating an import script for putting contacts into a database. The script we had worked ok for 500kb / 20k row CSV files, but anything much bigger than that and it started to run into the max execution limit. Rather than alter this I wish to create something that will run in the background and work as efficiently as possible. So basically the CSV file is uploaded, then you choose if the duplicates should be ignored / overwritten, and you match up the fields in the CSV (by the first line being a field title row), to the fields in the database. The field for the email address is singled out as this is to be checked for duplicates that already exist in the system. It then saves these values, along with the filename, and puts it all into an import queue table, which is processed by a CRON job. Each batch of the CRON job will look in the queue, find the first import that is incomplete, then start work on that file from where it left off last. When the batch is complete it will update the row to give a pointer in the file for the next batch, and update how many contacts were imported / how many duplicates there were So far so good, but when checking for duplicity it is massively slowing down the script. I can run 1000 lines of the file in 0.04 seconds without checking, but with checking that increases to 14-15 seconds, and gets longer the more contacts are in the db. For every line it tries to import its doing a SELECT query on the contact table, and although I am not doing SELECT * its still adding up to a lot of DB activity. One thought was to load every email address in the contacts table into an array before hand, but this table could be massive so thats likely to be just as inefficient. Any ideas on optimising this process? I have a problem with my project. i want to import Excel sheet directly into mysql database for PHP language without any conversion of Excel sheet into another format. i am very fed about this from last two weeks. So Please help me How I Do it. any expert please help me I am trying to import csv file to wordpress using database.the files are uploaded in database but its not coming in wordpress . The code is as below: <?php if(isset($_POST['submit'])) { $fname=$_FILES["file"]["name"]; $datetime=$_FILES["file"][""]; $chk_ext = explode(".",$fname); if(strtolower(($chk_ext[1]) == "csv")) { $filename = $_FILES['file']['tmp_name']; $handle = fopen($filename, "r"); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $fname=addslashes($data[0]); $datetime=addslashes($data[1]); $products=addslashes($data[2]); $preset=addslashes($data[3]); $format=addslashes($data[4]); $sql = "INSERT into ".$wpdb->prefix ."facebook(filename,datetime,products,preset,format) values('$fname','$datetime','$products','$preset','$format')"; mysql_query($sql) or die(mysql_error()); } fclose($handle); echo "Successfully Imported"; } else { echo "Invalid File"; } } ?> <form enctype="multipart/form-data" action="" method="post" name="uploadDetails"> <table cellpadding="4" cellspacing="0" border="1"> <tr> <td><label for="upload_a_file">Upload File:</label></td> <td><input type="file" name="file" id="upload_a_file" /></td> </tr> <tr> </tr> </table> <input type="hidden" name="fd_upload" value="1" /> <input type="submit" value="Submit" class="allbutton" name="submit" /> </form> Hope you will help me This topic has been moved to MySQL Help. http://www.phpfreaks.com/forums/index.php?topic=358188.0 My primary method of working with my database is Sequel Pro (Mac OS X), and I'm trying to update a column in a table via a CSV import. I should be able to do it via Sequel's Import, but I keep getting errors.
I've made sure I'm Updating and not Inserting.
I've mapped my column (just one column), and it's just a one column CSV file.
I get this error for each row:
[ERROR in row 1] You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE `email_acct`='\naccs.k12.in.us'' at line 1 I messed with what each is terminated by, escaped by, enclosed by...etc...nothing changes. I get the same error, just at the end it's a different email account location. So I'll try to do it via a query, but I can't find a reliable solution to make it work. I could create a separate table and update via that way, but I can use this as a learning experience. Hey guys, I'm having a nightmare at the moment.
I am trying to import a CSV file into a MySQL table, some of the data is interested properly into the rows however some of the CSV row do not get imported.
Here is my PHP script:
<?php
$databasehost = "localhost";
$databasename = "name";
$databasetable = "import";
$databaseusername="username";
$databasepassword = "password";
$fieldseparator = ",";
$lineseparator = "\n";
$enclosedbyquote = '"';
$csvfile = "db-core/feed/csv/test.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$pdo->exec("TRUNCATE TABLE `$databasetable`");
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." REPLACE INTO TABLE `$databasetable`
FIELDS OPTIONALLY ENCLOSED BY ".$pdo->quote($enclosedbyquote)."
TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator)."
IGNORE 1 LINES");
echo "Loaded a total of $affectedRows records from this csv file.\n";
?>
As you can see the script is importing the CSV into the table however only 40 of the rows are added to the SQL table when there are actually 80 rows in the CSV. I am also telling the script to IGNORE 1 LINES if that has anything to do with the issue.
Here is one of the rows from the CSV file:
"AETA83919","AETV34458584","SD57VUP","BLACK","Diesel","2007","76778","Hatchback","3","VAUXHALL","CORSA","1.3 CDTi [90] SXi 3dr [AC]","1200","3989","","Manual","http://images.autoexposure.co.uk/AETA83919/AETV34458584_1b.jpg","0","3","","0","2007 57 reg VAUXHALL CORSA 1.3 CDTi [90] SXi 3dr [AC], Manual, Hatchback, Diesel, 76,778 miles. BLACK, Comprehensive Warranty Included In Price. HPI Checked! Pre Delivery Health Check!, 5 seats, LOVELY VAUXHALL CORSA SXI 1.3CDTI SXI WITH AIR CONDITIONING!! VERY WELL CARED FOR AND MAINTAINED AND JUST SERVICED BY DRIVEN CAR SALES LTD AT 76K!! GREAT COMBINATION OF PERFORMANCE AND ECONOMY!! VIEWING AND TEST DRIVING IS RECOMMENDED!! EACH CAR COMES WITH A MININIUM 9 MONTHS MOT. FULL MECHANICAL HEALTH CHECK GUARANTEES CAR IS PREPARED TO THE HIGHEST STANDARD. COMPREHENSIVE RAC WARRANTY INCLUDED. DRIVEN CAR SALES WELCOME ANY INSPECTION. BUY WITH CONFIDENCE AND REASSURANCE AS DRIVEN CAR SALES LTD ARE OFFICIAL MEMBERS OF RETAIL MOTOR INDUSTRY FEDERATION. VIEW OUR REVIEWS ON TRUSTPILOT. 3,989","","0","1","","UK","0","","0","0","","","",""As far as I can see everything looks fine with the row, all of the rows in the CSV end like this. One thing I have noticed is it looks like the rows that are missing are in fact being placed in the last column of every other row here is an example: So as you can see it looks like one row is being inserted into the last column of some rows and I am so confused to why this might be happening? Any ideas? i was working on a mini project that imports csv file to the database through ajax and it's working fine
<?php
if(!empty($_FILES["marks_file"]["name"]))
{
$connect = mysqli_connect("localhost", "root", "", "dbname");
$output = '';
$allowed_ext = array("csv");
$extension = end(explode(".", $_FILES["marks_file"]["name"]));
if(in_array($extension, $allowed_ext))
{
$file_data = fopen($_FILES["marks_file"]["tmp_name"], 'r');
fgetcsv($file_data);
while($row = fgetcsv($file_data))
{
$name = mysqli_real_escape_string($connect, $row[0]);
$Physics = mysqli_real_escape_string($connect, $row[1]);
$Maths = mysqli_real_escape_string($connect, $row[2]);
$Chemistry = mysqli_real_escape_string($connect, $row[3]);
$Biology = mysqli_real_escape_string($connect, $row[4]);
$SST = mysqli_real_escape_string($connect, $row[5]);
$query = "
INSERT INTO csv
(name, Physics, Maths, Chemistry, Biology, SST)
VALUES ('$name', '$Physics', '$Maths', '$Chemistry', '$Biology' , '$SST')
";
mysqli_query($connect, $query);
}
$select = "SELECT * FROM csv ORDER BY id DESC";
$result = mysqli_query($connect, $select);
$output .= '
<table class="table table-bordered">
<tr>
<th width="25%" >name</th>
<th width="15%" >Physics</th>
<th width="15%" >Maths</th>
<th width="15%" >Chemistry</th>
<th width="15%" >Biology</th>
<th width="15%" >SST</th>
</tr>
';
while($row = mysqli_fetch_array($result))
{
$output .= '
<tr>
<td>'.$row["name"].'</td>
<td>'.$row["Physics"].'</td>
<td>'.$row["Maths"].'</td>
<td>'.$row["Chemistry"].'</td>
<td>'.$row["Biology"].'</td>
<td>'.$row["SST"].'</td>
</tr>
';
}
$output .= '</table>';
echo $output;
}
else
{
echo 'errorx';
}
}
else
{
echo "errory";
}
?>
however the imported csv files inserts null values in the tables because the format of all csv files assigned to me are in the exact same format: ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,Fields,Physics~75,Maths~50,Chemistry~65,Bio~85,SST~100 ,,,Name1,10,25,35,42,62 ,,,Name2,80,45,45,45,25 ,,,Name3,63,25,63,36,36 ,,,Name4,82,36,75,48,42 ,,,Name5,45,45,78,25,24 ,,,Name6,36,36,15,75,36 ,,,Name7,99,45,24,24,45 ,,,Name8,45,85,85,85,96 i changed my code a bit modified the functions to espace blank spaces and not return null values in the data tables
while (($row = fgetcsv($file_data, 1000, ",")) !== FALSE)
{
if ((string) $row[0] != '0' and empty($row[0]))
{
continue;
}
$name = mysqli_real_escape_string($connect, $row[0]);
$Physics = mysqli_real_escape_string($connect, $row[1]);
$Maths = mysqli_real_escape_string($connect, $row[2]);
$Chemistry = mysqli_real_escape_string($connect, $row[3]);
$Biology = mysqli_real_escape_string($connect, $row[4]);
$SST = mysqli_real_escape_string($connect, $row[5]);
$query = "
INSERT INTO csv
(name, Physics, Maths, Chemistry, Biology, SST)
VALUES ('$name', '$Physics', '$Maths', '$Chemistry', '$Biology' , '$SST')
";
mysqli_query($connect, $query);
}
But it doesn't work with the specified csv files i think the ajax call breaks and that's why nothing happens.However it works fine with csv files without blank spaces Edited March 19, 2019 by zetastreak I am a developer in PHP but have run into an obstacle and was hoping for some insight. I am attempting to do something a little complicated I fear but perhaps there is a solution. A client of mine is offering CSV templates for customers to use to BATCH UPLOAD item's to their account. This is self explanatory in the fact that I know how to import a .CSV with PHP and I know how to properly insert it into the database. My question is this: What if we wanted to offer a way for the customer to insert an image into the .CSV template. By the time the .csv reached my PHP code would the image still be in a format where it could be uploaded to a directory on the server? I have not found anything on the web where anyone has tried this. Again, the customer will insert an image from their hard drive into a row within the .CSV template. My code will parse the .CSV row data and if the image is still in tact...will then upload the image to a director with more code. Thanks for any help. Joe |
|||||||