PHP - Tip Needed: How To Stream A Compressed (bz2,bzip2,gzip) File Through Xmlreader
Is there a way I can readily stream-process a compressed XML file in PHP? Something like this NON WORKING example:
Code: [Select] $reader = new XMLReader(); $reader->open(bzopen($planet_file,"r")); while ($reader->read()) { ... See also http://php.net/manual/en/xmlreader.open.php Similar TutorialsI have spent all day trying to get any of these functions to work... I have downloaded heaps of expample and NONE! work.. In the case of Zip Archive the CREATE function will not produce a zip file archive to strat with.. Error message says it's created ok... but a search finds no such file. GZip has the same problem. Bzip2 at least creates a blank zip file (17bytes) but I can't add any files to the Bzip2. I have tried on the localhost with dll's installed and on a comercial web server... Nothing gets created... I have read endless examples and documents on the web... None work.. Any help would be much apreciated. I am being sent an XML feed which needs to be downloaded in a 'compressed way'.
The example I have been given is this:
$url = 'http://myurl.com'; $headers[] = "Accept-Encoding: gzip,deflate"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_BINARYTRANSFER, true); $data = curl_exec($ch); but the output I am getting is jargon like this with loads of black diamond question marks:� +T��sƒ���-}#v���]K���������HB�$&G�����Y�����'߷s�,�.4֫'?'���9;*9^��>�j~�ǫ}z|hq�J��Գ".o2)~b���U~I| How to a convert it back into XML? Many thanks Edited by samjsharples, 30 September 2014 - 03:49 PM. Hi guys, I did read allot of documentation on the internet about reading/writing/parsing an XML file. I ended up using the following code, because I really have large files (some about 200MB) and regular dom does not work: while ($xml->read()) { switch ($xml->nodeType) { case (XMLReader::ELEMENT): if ($xml->localName == "job") { $node = $xml->expand(); $dom = new DomDocument(); $n = $dom->importNode($node,true); $dom->appendChild($n); $job = simplexml_import_dom($n); The problem I have is a special character error in the xml file, error returned on this line: "$node = $xml->expand();" I am literally banging my head to the wall to find a simple solution to this. I already have a cleaning function, but that can be applied only after the code above. As the file is large, to clean it, I would have to use the same code above to work on partial content at once, so I would have the same special character problem when I would try to read and split the file. I bet I am not the first one to be in this situation, but after about 5 hours of searching on the internet, I cannot do it no more. And I am not a php expert to come up with a new idea. One other thing to do probably would be to split the file into multiple files, and read them after that, without using the XMLReader. But this would ask for a different application. If, for example, on a file where I have an error, I do the reading with simplexml, without using the XMLReader, I don't get the error. But I cannot use simplexml on the files, since file size is variable. I have to use a reliable method that works for all situations. Hopefully someone has an idea to this STUPID situation! Thanks. Hi guys, I'll try to be concise. When I use this code (zlib): $xmlFile = 'test.xml'; $data = implode("", file($xmlFile)); $gzdata = gzencode($data, 9); $gzFile = "test.gz"; $fp = fopen($gzFile, "w"); fwrite($fp, $gzdata); fclose($fp); It produces test.gz but when I extract the .gz file I get "test" file (without extension) I need to gzip the xml file (keeping the extension) not only the data (when I extract the gz I should get test.xml back) Is there a way I can do this ? For security reasons I can't use exec() function. Please help! Thanks in advance, Alex The other day I noticed someone had post something like this, well this was actually the solution to the problem. But it got me thinking about a project of mine that I am working on and the need to pump out a downloadable file. We are storing most of the files in a database using base64 encoding the 2 key types of files we are storing are images and PDF's mostly pdf's anyway where I am wanting to go with this is, is there anyway to take the base64 encoded file and get it to download through this, or am I tackling the idea in the wrong way? <?php $filename = 'somename.txt'; //this would obviously be changed according to the file type we would output $data =<<<DATA I know my data would go here for the given file once decoded. DATA; header('Content-Description: File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename=' . $filename); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . strlen($data)); echo $data; ?> I'm trying to copy an existing file using a php script, run on the command line. I've tried copy() and also a script from the comments on copy() http://www.php.net/manual/en/function.copy.php#102320 No matter how I try to open the file I get " failed to open stream: No such file or directory". I tried changing the name of the file, and the ext, and both. The file does exist, and if I run cp old new, the file gets copied fine. Current code Code: [Select] <?php $source = '/www/documents/myfile.pdf'; $destination = '/www/documents/mypdf.pdf.bak'; copy($source, $destination); ?> Warning: copy(/www/documents/myfile.pdf): failed to open stream: No such file or directory in /www/cron/historic.php on line 3 Code: [Select] ~$ ls /www/hua/compucom113dev/client/documents/ myfile.pdf ~$ cp /www/documents/myfile.pdf /www/documents/myfile.pdf.bak ~$ ls /www/hua/compucom113dev/client/documents/ myfile.pdf myfile.pdf.bak I had chmod in the script at one point to make sure the directory had the permissions needed but it didn't make a difference so I removed it. I can add it back if someone can help explain what to set it to. I was doing 0777 then back to 0755 at the end. Hello, this is my first post I hope you guys can help me out here, basically whats going on is I declare a variable "$diree" then I insert it in a function but its not getting recognized, here is the code: $institutea "some directory"; $dire = str_replace(" ","-",$institutea); # SETTINGS $max_width = 110; $max_height = 130; $per_page = 5; $page = $_GET['page']; $has_previous = false; $has_next = false; function getPictures() { global $page, $per_page, $has_previous, $has_next; if ( $handle = opendir($diree.'/') ) { // done changes here $lightbox = rand(); ?><table border="1"><tr><td> <form action="pro.php" method="post"> <?php echo "<ul id='pictures'>"; $count = 1; $skip = $page * $per_page; if ( $skip != 0 ) $has_previous = true; while ( $count < $skip && ($file = readdir($handle)) !== false ) { if ( !is_dir($file) && ($type = getPictureType($file)) != '' ) $count++; } $count = 1; while ( $count < $per_page && ($file = readdir($handle)) !== false ) { if ( !is_dir($file) && ($type = getPictureType($file)) != '' ) { if ( ! is_dir($dire .'/') ) { // done changes here mkdir($dire .'/'); // done changes here } if ( ! file_exists($dire .'/'.$file) ) { makeThumb( $file, $type ); } echo '<li>'; echo '<img src="'.$dire .'/'.$file.'" alt="" / width="110" height="110"><br/><input type="checkbox" name="food[]" value='.$file.'>'; // done changes here echo '</li>'; $count++; } } echo '</ul>'; ?> </td></tr><tr><td><div align="center"><input type="submit" value="Add"></div></td></tr> </tr></table></form> <?php while ( ($file = readdir($handle)) !== false ) { if ( !is_dir($file) && ($type = getPictureType($file)) != '' ) { $has_next = true; break; } } } } function getPictureType($file) { $split = explode($dire .'/', $file); $ext = $split[count($split) - 1]; if ( preg_match('/jpg|jpeg/i', $ext) ) { return 'jpg'; } else if ( preg_match('/png/i', $ext) ) { return 'png'; } else if ( preg_match('/gif/i', $ext) ) { return 'gif'; } else { return ''; } } function makeThumb( $file, $type ) { global $max_width, $max_height; if ( $type == 'jpg' ) { $src = imagecreatefromjpeg($file); } else if ( $type == 'png' ) { $src = imagecreatefrompng($file); } else if ( $type == 'gif' ) { $src = imagecreatefromgif($file); } if ( ($oldW = imagesx($src)) < ($oldH = imagesy($src)) ) { $newW = $oldW * ($max_width / $oldH); $newH = $max_height; } else { $newW = $max_width; $newH = $oldH * ($max_height / $oldW); } $new = imagecreatetruecolor($newW, $newH); imagecopyresampled($new, $src, 0, 0, 0, 0, $newW, $newH, $oldW, $oldH); if ( $type == 'jpg' ) { imagejpeg($new, $dire .'/'.$file); } else if ( $type == 'png' ) { imagepng($new, $dire .'/'.$file); } else if ( $type == 'gif' ) { imagegif($new, $dire .'/'.$file); } imagedestroy($new); imagedestroy($src); } ?> Hey all, I viewed this tutorial and the guy used $_SERVER['DOCUMENT_ROOT'] to reference a view layout for his specific index file. So I tried to imitate and did exactly what he did, but for me I got the following error: Code: [Select] Warning: include(/Users/jmerlino/Sites/mark/public/diet/views/layouts/shop.php) [function.include]: failed to open stream: No such file or directory in /Users/jmerlino/Sites/diet/index.php on line 22 Warning: include() [function.include]: Failed opening '/Users/jmerlino/Sites/mark/public/diet/views/layouts/shop.php' for inclusion (include_path='.:/usr/lib/php') in /Users/jmerlino/Sites/diet/index.php on line 22 This is the php code that is causing this error: include($_SERVER['DOCUMENT_ROOT'].'/'.'diet/views/layouts/'.$controller.'.php'); I'm not sure why it's going to: /Users/jmerlino/Sites/mark/public/diet/views/layouts/shop.php instead of: /Users/jmerlino/Sites/diet/views/layouts/shop.php Thanks for any response My images generator comprises
an array of image names extracted form an images table from a database using a select statement
a random number generator,
and a string that builds the correct pathname for the selected file.
To select one of the images for display, i generate a random number between 1 and the length of the array.
Though the generator is working, I noticed one of the random numbers is throwing up this error:
Warning: getimagesize(images/): failed to open stream: No such file or directory in
An inspection of the array reveals 2 array elements (representing my number of images) but one array element is NULL ( the first entry in the banner table
image_generator.php
require_once('connection.inc.php'); $sql = 'SELECT `filename` FROM banner'; $result = $mysqli->query($sql, MYSQLI_STORE_RESULT) or die(mysqli_error()); $row = $result->fetch_array(MYSQLI_ASSOC);//an array of image names $count = $result->num_rows; for ($i = 1; $i <= $count; ++$i) { $row[$i] = $result->fetch_array(MYSQLI_ASSOC); } $i = rand(1, $count); //a random number generator, //The random number is used in the final line to build the correct pathname for the selected file. $selectedImage = "images/{$row[$i]['filename']}"; if (file_exists($selectedImage) && is_readable($selectedImage)) { $imageSize = getimagesize($selectedImage); }var_dump($row) array (size=1) 'filename' => string 'ginsomin2.jpg' (length=13) nullRANDON IMAGE DISPLAY require_once 'image_generator.php'; <div id="banner" class="wrapper clearfix"> <img src="<?php echo $selectedImage; ?>" alt="banner"> </div>Kindly advice how i may proceed from here? Thanks. Using PHP 5.4.17 on Windows XP, and MySQL 5.6.17 on Windows Server 2003.
In phpinfo(), I have: mysqlnd 5.0.10 - 20111026
I have a need to capture/analyze the MySQL TCP packets as they pass between client computer and database server. I'm using WireShark 1.12.1 to do it.
I understand that compression of the payload will happen if both client and server can do it.
(It seems that the MySQL dissector in WireShark does not yet automatically deflate the payload - according to a bugtracker entry.)
So, is there a config file statement I can use, either in php.ini's [mysqlnd] section, or the server's my.cnf file, that will force zero compression?
Hey! I am trying to get a database table of users but am running into the error: Warning: file_get_contents(http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=68583) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 500 in get_user_from_parameter(nabble:utilities.naml:890) - <n.get_user_from_parameter.as_user_page.do/> - public v in C:\xampp\htdocs\website4js\stanford\loadUsernames.php on line 32 I have looked it up and it may be a security thing...The urls I am getting are http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=68583 but the error adds an nodes& part that screws it up. Any way around this? Code: [Select] <?PHP $maxPage = 0; $mainPage = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136"; $mainContent = file_get_contents($mainPage); $pattern = "/(?<=\"Page )\d\d+/"; preg_match_all($pattern, $mainContent, $pageNumb); //find the max page for($i=0;$i<sizeof($pageNumb[0]);$i++) { if($pageNumb[0][$i] > $maxPage) { $maxPage = $pageNumb[0][$i]; } } //echo('Max page is: '.$maxPage.'\n'); //Get an array of all the pages $pages = array(); for($i=1;$i<$maxPage;$i++) { $pages[$i] = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136&i=".($i*20); } //print_r($userPages); //Get personal page urls and add to MySQL mysql_connect("localhost" , "root") or die("Cant connect"); mysql_select_db("protegeusers") or die("Cant find database"); foreach($pages as $url) { $urlContents = file_get_contents($url); $pattern = "/http:\/\/protege-ontology-editor-knowledge-acquisition-system\.136\.n4\.nabble\.com\/template\/NamlServlet\.jtp\?macro=.+;user=\d+/"; preg_match_all($pattern, $urlContents, $personalPage); foreach($personalPage as $user) { for($i=0; $i<sizeof($user);$i++) { [color=green]$userContents = file_get_contents($user[$i]);[/color] $pattern1 = "/user\/SendEmail\.jtp\?type=user.+;user=\d+/"; $pattern2 = "/(?<=\">Send Email to ).+(?=<)/"; preg_match_all($pattern1, $userContents, $userEmail); preg_match_all($pattern2, $userContents, $username); [color=green]print_r($username); print_r($email);[/color] //$query = "INSERT INTO users (username, userurl) values ('$userName','$userUrl')"; //mysql_query($query); } } } ?> Ive gzipped some static html files and want to send it to the browser but not sure how to approach it So i have : http://site.com/cache/file.html (14gb) http://site.com/cache/file.html.gz (1kb) I did this but do i still need a header?? <?php if(file_exists('cache/file.html')){ @readfile("cache/file.html"); } ?> I have just googling in order to find out how to do GZIP so that user can load web page faster. n i found some article which stated that i have to put this script only. <?php ob_start( 'ob_gzhandler' ); ?> my question is...is it doing the zip? if it is correct where should i put that script, in every page which is needed to be compressed or inside index page only? thanks in advance hey all now im new to gzip got not much knowledge on it. i was testing my site in seositecheckup and according to the test im asked to compress my html, css and js files and i found this bit of coding on google <?php if(isset($_SERVER['HTTP_ACCEPT_ENCODING']) && substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')) ob_start('ob_gzhandler'); else ob_start(); ?> i included this file on top of my pages and then it seemed to compress my my file but it seem we need to set header expire on gzip so how could i set expire on it and could someone tell me what exactly is this coding above compressing i mean is it compressing image and all type of files? I had Tom Cat Java installed on my Linux sever and I need to call a java file, which isn't working. Someone told me to verify Java is in fact installed on my server using this command: Code: [Select] echo exec('which java'); but when I run it, nothing displays! The java command I'm trying to run is this: Code: [Select] exec('java -Xmx1024m -jar "/home/account/public_html/java -cp b4a_c2dm.jar anywheresoftware.b4a.c2dm.C2DM send $php_device_id \"test\""'); The .jar file is there. My path to java is: /usr/local/jdk/bin/java Thanks for any help... Hi guys <?php sleep(00); $handle = fopen("http://www.myurl.com/?bla=bla=bla" , "r"); fclose($handle); ?> How do i prevent this from outputing to a file on my server. I only need the url to be accessed to add a new row of data to my DB. Any Ideas? Regards Mark Hi. I have text files in the directory that contain rapidshare.com links. Now what I need to do is take them one by one and check links inside them one by one using rapidshare API. Everything works fine apart from one thing. It reads files and deletes the ones containing dead links but the problem is that it only deletes the file if ALL links are dead, but not when only one link is dead or some of them. Code: Code: [Select] <?php function rs_check($url){ if (!$url) return false; $files_pattern= '/\/files\/([^\/]*)\//'; $filename_pattern= '/\/files\/.*\/(.*)/'; preg_match($files_pattern, $url, $matches_id); preg_match($filename_pattern, $url, $matches_name); $res= file_get_contents("http://api.rapidshare.com/cgi-bin/rsapi.cgi?sub=checkfiles_v1&files={$matches_id[1]}&filenames={$matches_name[1]}"); list($fId,$fName,$fSize,$fServerId,$fStatus,$fShortHost,$fmd5) =explode(',',$res); return ($fStatus==1 || $fStatus==2); } $directory = "download/"; $linksfiles = glob("" . $directory . "*.txt"); foreach($linksfiles as $linkfile) { $lf = file_get_contents($linkfile); $purefilename = str_replace($directory, "" , $linkfile); if($purefilename != "5100c00fdb545e12e8cb4f9c5f69ec21.txt"){ if(rs_check($lf) != 1){ unlink($directory . $purefilename); echo("<br />"); } } } ?> Hi! I hope somebody can help me what im do wrong. i have checked that the data from the file is in $source_file but nothing imports to the database Code: [Select] <?php include('config.php'); include('opendb.php'); if(isset($_POST['upload'])) { $source_file = @$_POST['userfile']; //$source_file = fopen('http://localhost/test/upload/test.csv', 'r'); $target_table = 'foretag'; function csv_file_to_mysql_table($source_file, $target_table, $max_line_length=10000) { if (($handle = fopen("$source_file", "r")) !== FALSE) { $columns = fgetcsv($handle, $max_line_length, ","); foreach ($columns as $column) { $column = str_replace(".","",$column); } $insert_query_prefix = "INSERT INTO $target_table (".join(",",$columns).")\nVALUES"; while (($data = fgetcsv($handle, $max_line_length, ";")) !== FALSE) { while (count($data)<count($columns)) array_push($data, NULL); $query = "$insert_query_prefix (".join(",",quote_all_array($data)).");"; mysql_query($query); } fclose($handle); } } function quote_all_array($values) { foreach ($values as $key=>$value) if (is_array($value)) $values[$key] = quote_all_array($value); else $values[$key] = quote_all($value); return $values; } function quote_all($value) { if (is_null($value)) return "NULL"; $value = "'" . mysql_real_escape_string($value) . "'"; return $value; } } include('closedb.php'); echo "<br>done<br>"; ?> |