PHP - Some Simple Html, Help

Guys I m trying to make a crawler and the crawler works fine but the problem is that after some random amount of time the crawler crashes with the message "PHP Fatal error: Maximum function nesting level of '100' reached, aborting! in ....."  . Please help me out . Is it my code or there is a problem with simple html dom ??

I have a quick question to ask, as i cant see it in their docs. Maybe you can help me.
I am using http://simplehtmldom.sourceforge.net/

Right, i have written a Simple HTML Dom script to collect data from this page, as an example:
http://www.visualdesign.ie/_dev/myscraper/simplehtmldom/dev-env/scraping/daily/daily.html

Script executed by running a cron job on this file (which generates the XML):
http://www.visualdesign.ie/_dev/myscraper/simplehtmldom/dev-env/scraping/daily/daily.php

The data is collected, and written to this file in XML file:
http://www.visualdesign.ie/_dev/myscraper/simplehtmldom/dev-env/scraping/daily/daily.xml

Thats is fine, and the script essentially scrapes the entire page, with no conditional IF's for any sections.

A sample of my code for one segment is below:
http://pastebin.com/JLb8f92N 

What i would like to do now, and i am hoping you could help out is...
Scrap this same page, but produce seperate XML files, based on the date.
So if you view that page, i want to scrape the data in the table Saturday 4th February 2012 and produce XML for for that day only.
The i want to scrape the data in the section for Sunday 5th February 2012, and another for Tuesday 7th February 2012. I think its self explanatory.

The table date sections are separated by this HTML element: bg color = " #CCCCCC "
But bare in mind that the table rows on each date section will change when the site is updated, and there may be additional or less football game records.

Anyway, i would appreciate any help you can show me, on how to place in any conditionals in the code, and only scrape the date sections needed.
Separate script files would be fine if it was needed.

Many thanks for your time,
Darren.

The following code is relatively short, but censors any HTML tags inside the XML:

function object2array($object) // from php.net 
{  
    $return = NULL;  
        
    if(is_array($object))  
    {  
        foreach($object as $key => $value)  
            $return[$key] = object2array($value);  
    }  
    else  
    {  
        $var = get_object_vars($object);  
            
        if($var)  
        {  
            foreach($var as $key => $value)  
                $return[$key] = ($key && !$value) ? NULL : object2array($value);  
        }  
        else return $object;  
    }  

    return $return;  
} 

$bla=simplexml_load_file($xml_file);
$bla=object2array($bla);

This one keeps HTML but turns everything into one giant string:

$bla=$bla->asXML();

So how can I easily preserve HTML? But better yet, can I somehow just tell PHP which tags to convert? For example, only <this> and <that> in:

Code: [Select]
<this>
  <that>Text <foo>and</foo> test and <whatever>something</whatever>.</that>
</this>
thus creating:

Code: [Select]
Array
(
    [this] => Array
            [0] => Array
                (
                    [that] => Text <foo>and</foo> test and <whatever>something</whatever>.
                 )

}

All

some help would be welcome. I'm having troubles with this simple email script.

the idea: would like to make a mailing from a form. This works just fine. I'm having troubles with the mailing it self.
What i posted he I edited my code to try without concatenating strings (*$_Post varible within xml). This however doesn't work eitherway.

tests:

all variables get set properly (tested with echo's  so I cut them out to save you the reading :-))
connection gets established just fine
All email adresses from the database pass by just fine. (also by echo :-))

I'm thinking semantic error but can't find the problem. Can anyone help me? 

<?php
    
    //make connection with database
    $con = mysql_connect("XXXXXXXXX","XXXXXXXXXX","XXXXXXXX");
    if (!$con){
        die('Could not connect: ' . mysql_error());
      }
    
    mysql_select_db("dejuistestudiek", $con);

    //select all emails from mailing table
    if($con){
        $sql = "SELECT * FROM mailing";    
        $result = mysql_query($sql);
        
        //prepare text for html mail
        $text = trim($_POST['TEXT']);
        $text = nl2br($text);
        $text = stripslashes($text);


        if($result){
            
            while($row = mysql_fetch_array($result)){        
                
                // single recipient
                $to  = $row['email'];
                
                // subject
                $subject = 'StuWay - Nieuwsbrief';
                
                // message
                $message = '<html>
                            <head>
                              <title>Birthday Reminders for August</title>
                            </head>
                            <body>
                              <p>Here are the birthdays upcoming in August!</p>
                              <table>
                                <tr>
                                  <th>Person</th><th>Day</th><th>Month</th><th>Year</th>
                                </tr>
                                <tr>
                                  <td>Joe</td><td>3rd</td><td>August</td><td>1970</td>
                                </tr>
                                <tr>
                                  <td>Sally</td><td>17th</td><td>August</td><td>1973</td>
                                </tr>
                              </table>
                            </body>
                            </html>';
                                            
                // To send HTML mail, the Content-type header must be set
                $headers  = 'MIME-Version: 1.0' . "\r\n";
                $headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
                
                // Additional headers
                $headers .= 'From: Do-Not-Reply@dejuistestudiekeuze.be' . "\r\n";
                // Mail it
                $bool = mail($to, $subject, $message, $headers);
                
            }
        }
    }

?>

Hello, can any1 show me how to insert the following in db
Code: [Select]

<?php
include('dbconnect.php');
include('simple_html_dom.php');
$html = file_get_dom('test3.php');
foreach($html->find('h2') as $heading)
{
//echo $heading;
foreach ($heading->find('a') as $link)
{
                                       $item_title = md5($link);
echo $link."<br/>";
$sql = "INSERT INTO articles(article_id, article_link) VALUES($item_title, $link)";
$result = mysql_query($sql);
}
}
?>

what i want to do is insert every link in the db, but this is not working. The output is ok but the insert doesnt work. Any help please??

hello dear community,

i am currently wroking on a approach to parse some sites that contain datas on Foundations in Switzerland
with some details like goals, contact-E-Mail and the like,,,

See http://www.foundationfinder.ch/  which has a dataset of 790 foundations. All the data are free to use - with no limitations copyrights on it.

I have tried it with PHP Simple HTML DOM Parser - but , i have seen that it is difficult to get all necessary data -that is needed to get it up and running.

Who is wanting to jump in and help in creating this scraper/parser.   I love to hear from you.

Please help me - to get up to speed with this approach?


regards
Dilbertone

 
Hi Everyone,

I have just started using Simple HTML DOM today and I have spent 4 hours not getting what I want.

I want to be able to extract the following information:

Code: [Select]
<div class="listing_content">
<span class="serialNumb" style="line-height: 21px;">77777</span>
<br />
444 ASDF, Alpha, Tango, Beta
<br />
77777 Director:99999
              <div>
<img title='web' src='http://cpgimg.com/images/icon_sm_web.gif' alt='web'/>&nbsp;&nbsp;<a href='javascript:void(0)' onClick="window.open('/redir.jsp?p_url=http:%2f%2fwww.cnn.com&p_cid=2707304&p_hid=279E00&p_ct=3527&p_pr=KO&p_fr=U');" class='listing_link'>website</a>&nbsp;&nbsp;<img title='email' src='http://cpgimg.com/images/icon_sm_mail.gif' alt='email'/>&nbsp;&nbsp;<a class='listing_link' href="javascript:void(0)" onclick="popupEmail('/email.jsp?lang=0&p_cid=2707304');(new Image()).src='/redir.jsp?p_url=&p_cid=2707304&p_hid=279E00&p_ct=3527&p_pr=ON&p_fr=E&msec='+(new Date()).getMilliseconds()">E-mail</a>&nbsp;&nbsp;
               </div>
</div>                                  
The content I need to pull separately from above include:
1- serialNumb = 77777
2- 444 ASDF, Alpha, Tango, Beta
3- 77777 Director:99999
4- www.cnn.com

I want all the data to recorded to different variables so I can upload them to MySQL.

Any help with this is much appreciated. I don't have to use Simple DOM HTML but per my search it seems to be the best tool (however, I am not so lucky with it.)

***Not to forget that this page is full of <div>, <br />, <img>, and other tags. The quoted part is just one excerpt but this part is unique and used once in the page "style="line-height: 21px;". Also the "('/redir.jsp?p_url" is also unique for the URL portion.

Thanks again.

Hi All,

I am using the PHP Simple HTML DOM parser to connect to a financials website, parse out a companies financial information (Income statement in this case) and then insert the scrapped data into a mysql database that I can then later use to run automated calculations.

Here is the code I have so far:

Code: [Select]
<?php
include_once 'simple_html_dom.php';

//Connect to financial Website and Create DOM from URL
$income_statement = file_get_html('http://www.WEBSITE.com/finance?etc..etc...etc...etc...');

//PULL FINANCIAL DATA
foreach($income_statement->find('td[class]' ) as $lines=>$data) {

echo $data->plaintext . "<br/>";

}

// clean up memory
    $html->clear();
    unset($html);
?>

So far I am able to get output that looks like this:

Code: [Select]
Revenue
336.57
331.52
324.32
319.29
320.40
Other Revenue, Total
-
-
-
-
-
Total Revenue
336.57
331.52
324.32
319.29
320.40
etc.............................
But being a newb I do not understand how I can break each $ value and each - into their own variables and then insert them to their corresponding mysql table fields. During the database insert I would like to ignore field headings from insertion (i.e Revenue, Total Revenue, etc....

Any help would be absolutely amazing, as I have been reading, scripting and searching for information like crazy, but just can't seem to figure it out.

Hi,
I am learning PHP now, so pardon my silly question which I am not able to resolve from a week. I have created a simple web form where in I display the values entered by a user.


<form action ="reply.php" id="myForm" method="post" >
      Name: <input type="text" name="name" size="25" maxlength="50" /> <br> </br>
      Description: <textarea name="editor1"> </textarea>
      <input type="submit" value="Submit"  />
</form>

and reply.php contains:
<?php
    echo "In reply page";
   
    foreach($_POST as $field => $value)
   {
        echo "$field = $value";
   }
?>

When I click on the submit button, I just get a blank page without any values from the form. Can anyone please let me know what am I missing?

Set up:
I am using Netbeans with PHP bundle added on to it. When i run only a simple php proj it displays that page in localhost/nameofproj, but when I run a php file along with a jsp file, it runs in localhost:8080/nameofproj.  Is this the reason(localhost:8080 instead of just localhost ) for showing a blank page, not even a simple echo stmt, when i click on submit?
I even re installed netbeans, still no luck.

Thanks in advance.

Hello dear Community,

i have a document i need to parse it and spit out only this part of the table:

see http://schulnetz.nibis.de/db/schulen/schule.php?schulnr=67003&lschb=

how to i parse the stuff!? With perl or php?

Note i have the xpaths (see below) Sad that i cannot apply them on Simple DOM Parser since this Dom Parser does not work with Xpaths but with  CSS-Selectors:

Well i want to get all the data with that are within the table that name is called  class="fliess"

How to dump all the results?
BTW - thinking about the most elegant way, i think it is the most pretty way would be to do it with perl - So i can try it with HTML::TableExtract or....

Well what do you suggest - Which way to choose to do this [very] simple thing?


Look forward to hear from you!

see the xpaths:

Schule:  /html/body/center/table/tbody/tr[2]/td[1]
Stasse: /html/body/center/table/tbody/tr[3]/td[1]
Ort:     /html/body/center/table/tbody/tr[4]/td[1]
Tel:     /html/body/center/table/tbody/tr[5]/td[1]
Schulgliederungen:  /html/body/center/table/tbody/tr[6]/td[1]
Besonderheite: /html/body/center/table/tbody/tr[7]/td[1]
E-Mail: /html/body/center/table/tbody/tr[8]/td[1]
Schulnummer: /html/body/center/table/tbody/tr[9]/td[1]

good day dear community,


this is a big issue. I have to decide: between native PHP DOM Extension or of simple DOM html parser

well i want to parse the site he http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

I will suggest to use the native PHP "DOM" Extension instead of "simple html parser", since it will be much faster and easier

What do you think about this one here...:

Code: [Select]
$doc = new DOMDocument
@$doc->loadHTMLFile('...URL....'); // Using the @ operator to hide parse errors
$contents = $doc->getElementById('content')->nodeValue; // Text contents of #content


look  forward to hear from you

best regards
db1

Hello dear friends,

first of all : merry merry Xmas!!!


i want to parse with the simple Simple HTML DOM Parser,


well i am pretty new to php and to the Simple HTML DOM Parser.

My example: http://schulen.bildung-rp.de/gehezu/startseite/einzelanzeige.html?tx_wfqbe_pi1[uid]=60119

I want to collect the data in the block:

I have investigated the sourcecode - and found out that the attribute of interest should be this one: class="content"div class="content"><!-- TYPO3SEARCH_begin -->


here the code is: - my trails.

 

// inculde the Simple HTML DOM Parser 
include_once('simple_html_dom.php');

// get the file we want to parse right now,create a DOM 
$html = file_get_html('');

// simple_html_dom::find() creates a new 
// simple_html_dom-Objekt, that consists out of
// corresponding childelements 

foreach($html->find('class: content ') as $h3) {

  // simple_html_dom::get the text in a tag
  // den Text innerhalb eines Tags
  if($h3->innertext == 'Text of a H3 Tag') {
    break;
  }
}

// simple_html_dom::next_sibling() gives the
// next   Element
$table = $h3->next_sibling();



but believe me - it  gives me not back what is aimed.

what  have id done wrong...? 

dbone

I'm using PHP 5.2 Server and Simple HTML DOM 1.5. This script scrape or extract data from a football site, its fully working on PHP 5.9 Server but I need to know how I can fix it for PHP 5.2 server. Can someone give me a hint on how can I fix the error? Thanks in advance.

My PHP 5.2 Server script output shows:
++++++++++++++++
Object id #599 Object id #604 Object id #609 Object id #614 Object id #619
Object id #627 Object id #632 Object id #637 Object id #642 Object id #647
Object id #655 Object id #660 Object id #665 Object id #670 Object id #675
Object id #683 Object id #688 Object id #693 Object id #698 Object id #703
Object id #711 Object id #716 Object id #721 Object id #726 Object id #731
++++++++++++++++

while PHP 5.9 Server says
++++++++++++++++
Rk Player Team POS OPPONENT
1 Aaron Rodgers GB QB at CAR
2 Tom Brady NE QB vs. SD
3 Matt Schaub HOU QB at MIA
4 Michael Vick PHI QB at ATL
++++++++++++++++

I did applied the bug solution listed on https://sourceforge.net/tracker/index.php?func=detail&aid=3107230&group_id=218559&atid=1044037 but it is still not working. It says:
++++++++++++++++
Details:

I get compiler errors in PHP 5.2 when using this as an object.

The offending lines are 609 and 940, which both contain this construct:

if ($this->size>0) $this->char = $this->doc[0];

This tries to get the first character of $this->doc, but PHP 5.2 sees it as trying to access it as an array. It's easily fixed by this:

if ($this->size>0) $this->char = substr($this->doc, 0, 1);

Or you could probably use chr(ord($this->doc)) as well. Either way solves the compile error without changing functionality.
++++++++++++++++

Here are my codes:

Code: [Select]
<?php
# don't forget the library
include('simple_html_dom.php');

# this is the global array we fill with article information
$articles = array();
$source = 'http://www.athlonsports.com/columns/winning-game-plan/fantasy-football-qb-rankings';
# passing in the first page to parse, it will crawl to the end
# on its own
getArticles($source);      


function getArticles($page) {
global $articles, $descriptions;

$html = new simple_html_dom();
$html->load_file($page);

//$items = $html->find('div[class=preview]');
$items = $html->find('tbody tr');

foreach($items as $post) {
    # remember comments count as nodes
   /*$articles[] = array($post->children(3)->outertext,
                        $post->children(6)->first_child()->outertext);*/
    $articles[] = array($post->children(0), $post->children(1), $post->children(2), $post->children(3), $post->children(4));
}

# lets see if there's a next page
if($next = $html->find('a[class=nextpostslink]', 0)) {
    $URL = $next->href;
    echo "going on to $URL <<<\n";
    # memory leak clean up
   $html->clear();
    unset($html);

    getArticles($URL);
}
}

?>


<html>
<head>
</head>
<body>
<?
echo "Source: " . $source;
?>
<table cellpadding="5" cellspacing="0" border="0">
<?php
    foreach($articles as $item) {
        echo "<tr>";
        echo "<td>" . $item[0] . "</td><td>" . $item[1] . "</td><td>" . $item[2] . "</td>";
        echo "<td>" . $item[3] . "</td><td>" . $item[4] . "</td>";
        echo "<tr>";
    }
?>
</table>


</body>
</html>

Hi,

I m doing some work for my self an because of that i been reading a lot arround about PHP, and theres something that i would like to ask a bit of enlightenment. So my question is as the title says about html form's using php to insert data into mysql, i been reading tutorials arround the interwebs and even made afew successful tests, but pretty much all tutorials use 2 files to accomplish this the html file with the form and an insert.php where the actual code is stored so this made me think is this how usually it's done? in over all you will have 1 file for the form, 1 for the insert, 1 for the edit php code and 1 for delete. How do you guys usually do it?

PS: one of the tests i did was making 1 single file with all these using an switch. My interest in making this question is solo to learn how other people do it to see if i m in the right way.

Thanks in advance.