PHP - Simple Html Dom Parser 'simple_html_dom.php' Problem

Hello dear Community,

i have a document i need to parse it and spit out only this part of the table:

see http://schulnetz.nibis.de/db/schulen/schule.php?schulnr=67003&lschb=

how to i parse the stuff!? With perl or php?

Note i have the xpaths (see below) Sad that i cannot apply them on Simple DOM Parser since this Dom Parser does not work with Xpaths but with  CSS-Selectors:

Well i want to get all the data with that are within the table that name is called  class="fliess"

How to dump all the results?
BTW - thinking about the most elegant way, i think it is the most pretty way would be to do it with perl - So i can try it with HTML::TableExtract or....

Well what do you suggest - Which way to choose to do this [very] simple thing?


Look forward to hear from you!

see the xpaths:

Schule:  /html/body/center/table/tbody/tr[2]/td[1]
Stasse: /html/body/center/table/tbody/tr[3]/td[1]
Ort:     /html/body/center/table/tbody/tr[4]/td[1]
Tel:     /html/body/center/table/tbody/tr[5]/td[1]
Schulgliederungen:  /html/body/center/table/tbody/tr[6]/td[1]
Besonderheite: /html/body/center/table/tbody/tr[7]/td[1]
E-Mail: /html/body/center/table/tbody/tr[8]/td[1]
Schulnummer: /html/body/center/table/tbody/tr[9]/td[1]

good day dear community,


this is a big issue. I have to decide: between native PHP DOM Extension or of simple DOM html parser

well i want to parse the site he http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

I will suggest to use the native PHP "DOM" Extension instead of "simple html parser", since it will be much faster and easier

What do you think about this one here...:

Code: [Select]
$doc = new DOMDocument
@$doc->loadHTMLFile('...URL....'); // Using the @ operator to hide parse errors
$contents = $doc->getElementById('content')->nodeValue; // Text contents of #content


look  forward to hear from you

best regards
db1

Hello dear friends,

first of all : merry merry Xmas!!!


i want to parse with the simple Simple HTML DOM Parser,


well i am pretty new to php and to the Simple HTML DOM Parser.

My example: http://schulen.bildung-rp.de/gehezu/startseite/einzelanzeige.html?tx_wfqbe_pi1[uid]=60119

I want to collect the data in the block:

I have investigated the sourcecode - and found out that the attribute of interest should be this one: class="content"div class="content"><!-- TYPO3SEARCH_begin -->


here the code is: - my trails.

 

// inculde the Simple HTML DOM Parser 
include_once('simple_html_dom.php');

// get the file we want to parse right now,create a DOM 
$html = file_get_html('');

// simple_html_dom::find() creates a new 
// simple_html_dom-Objekt, that consists out of
// corresponding childelements 

foreach($html->find('class: content ') as $h3) {

  // simple_html_dom::get the text in a tag
  // den Text innerhalb eines Tags
  if($h3->innertext == 'Text of a H3 Tag') {
    break;
  }
}

// simple_html_dom::next_sibling() gives the
// next   Element
$table = $h3->next_sibling();



but believe me - it  gives me not back what is aimed.

what  have id done wrong...? 

dbone

I'm using PHP 5.2 Server and Simple HTML DOM 1.5. This script scrape or extract data from a football site, its fully working on PHP 5.9 Server but I need to know how I can fix it for PHP 5.2 server. Can someone give me a hint on how can I fix the error? Thanks in advance.

My PHP 5.2 Server script output shows:
++++++++++++++++
Object id #599 Object id #604 Object id #609 Object id #614 Object id #619
Object id #627 Object id #632 Object id #637 Object id #642 Object id #647
Object id #655 Object id #660 Object id #665 Object id #670 Object id #675
Object id #683 Object id #688 Object id #693 Object id #698 Object id #703
Object id #711 Object id #716 Object id #721 Object id #726 Object id #731
++++++++++++++++

while PHP 5.9 Server says
++++++++++++++++
Rk Player Team POS OPPONENT
1 Aaron Rodgers GB QB at CAR
2 Tom Brady NE QB vs. SD
3 Matt Schaub HOU QB at MIA
4 Michael Vick PHI QB at ATL
++++++++++++++++

I did applied the bug solution listed on https://sourceforge.net/tracker/index.php?func=detail&aid=3107230&group_id=218559&atid=1044037 but it is still not working. It says:
++++++++++++++++
Details:

I get compiler errors in PHP 5.2 when using this as an object.

The offending lines are 609 and 940, which both contain this construct:

if ($this->size>0) $this->char = $this->doc[0];

This tries to get the first character of $this->doc, but PHP 5.2 sees it as trying to access it as an array. It's easily fixed by this:

if ($this->size>0) $this->char = substr($this->doc, 0, 1);

Or you could probably use chr(ord($this->doc)) as well. Either way solves the compile error without changing functionality.
++++++++++++++++

Here are my codes:

Code: [Select]
<?php
# don't forget the library
include('simple_html_dom.php');

# this is the global array we fill with article information
$articles = array();
$source = 'http://www.athlonsports.com/columns/winning-game-plan/fantasy-football-qb-rankings';
# passing in the first page to parse, it will crawl to the end
# on its own
getArticles($source);      


function getArticles($page) {
global $articles, $descriptions;

$html = new simple_html_dom();
$html->load_file($page);

//$items = $html->find('div[class=preview]');
$items = $html->find('tbody tr');

foreach($items as $post) {
    # remember comments count as nodes
   /*$articles[] = array($post->children(3)->outertext,
                        $post->children(6)->first_child()->outertext);*/
    $articles[] = array($post->children(0), $post->children(1), $post->children(2), $post->children(3), $post->children(4));
}

# lets see if there's a next page
if($next = $html->find('a[class=nextpostslink]', 0)) {
    $URL = $next->href;
    echo "going on to $URL <<<\n";
    # memory leak clean up
   $html->clear();
    unset($html);

    getArticles($URL);
}
}

?>


<html>
<head>
</head>
<body>
<?
echo "Source: " . $source;
?>
<table cellpadding="5" cellspacing="0" border="0">
<?php
    foreach($articles as $item) {
        echo "<tr>";
        echo "<td>" . $item[0] . "</td><td>" . $item[1] . "</td><td>" . $item[2] . "</td>";
        echo "<td>" . $item[3] . "</td><td>" . $item[4] . "</td>";
        echo "<tr>";
    }
?>
</table>


</body>
</html>

hello dear php-experts,

i fairly new to simple_html_dom usage and methods. I know a little the parser,
 
i want to gather some information from this site:

https://europa.eu/youth/volunteering/organisations_en#open

is this possible to get the content - of let us say 10 or 20 last records on that page - and subesquently to store it in my mysql - db!?

<?php
// Report all PHP errors (see changelog)
error_reporting(E_ALL);

include('inc/simple_html_dom.php');

    //base url
    $base = 'https://europa.eu/youth/volunteering/organisations_en#open';

    //home page HTML
    $html_base = file_get_html( $base );

    //get all category links
    foreach($html_base->find('a') as $element) {
        echo "<pre>";
        print_r( $element->href );
        echo "</pre>";
    }

    $html_base->clear(); 
    unset($html_base);

?>

 

I have the above code and I'm trying to get certain elements of the page but it isn't returning anything. 

 

Is it possible that certain PHP functions might be disabled on the server to stop that?

The above code works perfectly on other sites.

 

Is there any workaround?

 

 

btw:  i have created a small snipped as a proof of concept to run this with Python and BeautifulSoup - 

 

import requests
from bs4 import BeautifulSoup
 
url = 'https://europa.eu/youth/volunteering/organisations_en#open'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')
print(soup.find('title').text)
block = soup.find('div', class_="eyp-card block-is-flex")

 

and this.... 

 

European Youth Portal
>>> block.a
<a href="/youth/volunteering/organisation/48592_en" target="_blank">"Academy for Peace and Development" Union</a>
>>> block.a.text
'"Academy for Peace and Development" Union'
 
>>> block.select_one('div > div > p:nth-child(9)')
<p><strong>PIC:</strong> 948417016</p>
>>> block.select_one('div > div > p:nth-child(9)').text
'PIC: 948417016'

 

what is aimed in the end - i want to gather the first 20 results of the page - and put them in to a sql-db or alternatively show the information in a little widget 

I want to remove empty paragraphs from an HTML document using simple_html_dom.php. I know how to do it using the DOMDocument class, but, because the HTML files I work with are prepared in MS Word, the DOMDocument's loadHTMLFile() function gives this exception "Namespaces are not defined".

This is the code I use with the DOMDocument object for HTML files not prepared in MS Word:
<?php
/* Using the DOMDocument class */

/* Create a new DOMDocument object. */
$html = new DOMDocument("1.0", "UTF-8");

/* Load HTML code from an HTML file into the DOMDocument. */
$html->loadHTMLFile("HTML File With Empty Paragraphs.html");

/* Assign all the <p> elements into the $pars DOMNodeList object. */
$pars = $html->getElementsByTagName("p");

echo "The initial number of paragraphs is " . $pars->length . ".<br />";

/* The trim() function is used to remove leading and trailing spaces as well as
* newline characters. */
for ($i = 0; $i < $pars->length; $i++){
    if (trim($pars->item($i)->textContent) == ""){
        $pars->item($i)->parentNode->removeChild($pars->item($i));
        $i--;
    }
}

echo "The final number of paragraphs is " . $pars->length . ".<br />";

// Write the HTML code back into an HTML file.
$html->saveHTMLFile("HTML File WithOut Empty Paragraphs.html");
?>

This is the code I use with the simple_html_dom.php module for HTML files prepared in MS Word:
<?php
/* Using simple_html_dom.php */

include("simple_html_dom.php");

$html = file_get_html("HTML File With Empty Paragraphs.html");

$pars = $html->find("p");

for ($i = 0; $i < count($pars); $i++) {
    if (trim($pars[$i]->plaintext) == "") {
        unset($pars[$i]);
        $i--;
    }
}

$html->save("HTML File without Empty Paragraphs.html");
?>

It is almost the same, except that that the $pars variable is a DOMNodeList when using DOMDocument and an array when using simple_html_dom.php. But this code does not work. First it runs for two minutes and then reports these errors: "Undefined offset: 1" and "Trying to get property of nonobject" for this line: "if (trim($pars[$i]->plaintext == "")) {".

Does anyone know how I can fix this?

Thank you.

I also asked on stackoverflow.

Im using some software called php html dom parser i wont to be able to keep the souce tidy

i.e before dom parser

<?php

//////////////////////SEO TOOL///////////////////////////

$title = 'Green Deal Nationwide - PB Energy Solutions Ltd';
$description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';

///////////////////////////////////////////////////////

?>
<?php include('includes/settings.php'); ?>
<?php include('includes/header.php'); ?>

    <div class="container">

        <div id="large-page-img">
        <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />
        </div>
        <div id="content-home">
     
            <div class="iedit">

after dom parser saved to file

<?php    //////////////////////SEO TOOL///////////////////////////    $title = 'Green Deal Nationwide - PB Energy Solutions Ltd';  $description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';    ///////////////////////////////////////////////////////    ?>  <?php include('includes/settings.php'); ?>  <?php include('includes/header.php'); ?>        <div class="container">            <div id="large-page-img">          <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />          </div>          <div id="content-home">                      <div class="iedit"><div class="iedit">

is there anyway i can keep it like the original fil after dom?

how should i approach the following:
a page with a products list+link to product page

i want to build a crawler that loops through all the products in the list and goes to the product page and
and parses the product page.

need help with the loop

hello dear Freaks 

 

i am currently  musing bout the portover of a python bs4 parser to php -  working with the simplehtmldom-parser / pr the DOM-selectors... (see below).

The project: for a list of meta-data of wordpress-plugins: - approx 50 plugins are of interest! but the challenge is: i want to fetch meta-data of all the existing plugins. What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp - that are updated (most) recently. It is all aobut acutality...

https://wordpress.org/plugins/participants-database ....and so on and so forth.  

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.
 

we have the following set of meta-data for each wordpress-plugin:

Version: 1.9.5.12 
installations: 10,000+    
WordPress Version: 5.0 or higher 
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags:databasemembersign-up formvolunteer
Last updated: 19 hours ago

 

the project consits of two parts:   the looping-part: (which seems to be pretty straightforward). the parser-part: where i have some issues - see below. I'm trying to loop through an array of URLs and scrape the data below from a list of wordpress-plugins. See my loop below-

as a base i think it is good starting point to work from the following target-url: 

 plugins wordpress.org/plugins/browse/popular with 99 pages of content: cf ...
 wordpress.org/plugins/browse/popular/page/1 
wordpress.org/plugins/browse/popular/page/2
wordpress.org/plugins/browse/popular/page/99

 

the Output of text_nodes:

['Version: 1.9.5.12', 'Active installations: 10,000+', 'Tested up to: 5.6 ']  

but if we want to fetch the data of all the wordpress-plugins and subesquently sort them to show the -let us say - latest 50 updated plugins. This would be a interesting task:

 

first of all we need to fetch the urls

then we fetch the information and have to sort out the newest- the newest timestamp. Ie the plugin that updated most recently

List the 50 newest items - that are the 50 plugins that are updated recently ..

 

we have the following set
 

see here the Soup_ 

 soup = BeautifulSoup(r.content, 'html.parser')
        target = [item.get_text(strip=True, separator=" ") for item in soup.find(
            "h3", class_="screen-reader-text").find_next("ul").findAll("li")[:8]]
        head = [soup.find("h1", class_="plugin-title").text]
        new = [x for x in target if x.startswith(
            ("V", "Las", "Ac", "W", "T", "P"))]
        return head + new


with ThreadPoolExecutor(max_workers=50) as executor1:
    futures1 = [executor1.submit(parser, url) for url in allin]

for future in futures1:
    print(future.result())

 

see the formal output

 

Quote

[lorem ipsum dolor sit amet', 'Version: 2.34.1', 'Last updated: 5 months ago', 'Tags: magna aliquyam erat, sed diam voluptua. At vero eos et accusam']
[consetetur sadipscing elitr', 'Version: 6.54.1', 'Last updated: 5 months ago', 'Tags: lorem ipsum dolor sit amet']
[sed diam nonumy eirmod tempor invidunt ut labore', 'Version: 7.16.1', 'Last updated: 5 months ago', 'Tags: tarifa, sevilla lisabin invidunt ut labore et dolore magna aliquyam erat']
[tempor invidunt ut taria malaga jerusalem labore', 'Version: 9.58.1', 'Last updated: 5 months ago', 'Tags: ilabore et lissabon dolore magna aliquyam erat']

 

background: https://stackoverflow.com/questions/61106309/fetching-multiple-urls-with-beautifulsoup-gathering-meta-data-in-wp-plugins

Well - i guess that we c an do this with the simple DOM Parser - here the seclector reference. 

https://stackoverflow.com/questions/1390568/how-can-i-match-on-an-attribute-that-contains-a-certain-string

 

look forward to any hint and help.

 

have a great day 

Edited May 3, 2020 by dil_bert

Everything works fine, unless I add this stupid thing to get rid of people using HTML
Code: [Select]
$text = pun_htmlspecialchars($text);
Once I add that to my function, no bbcodes work at all?  But I cant use html.. (which is good) but I need to beable to use BBCODE, and parse hackers from using html also, any help?



MY CODE absolutely destroyed the forum page

here it is:

http://pastebin.com/jv7m47kn

I am using simple_html_dom.php

I am stuck with the Code of How to parse below Content :
Quote

<div id="entry_4" class="entry clearfix "><div class="entry_title clearfix"><h1 class=" ">Smith J</h1></div><div class="full_listing"><div class="blocks"><div id="entry_4_block_0" class="block indent-level-0"><div class="share_link" wpol:entryId="719183066N00W" wpol:contactPointId="719183066N00W"><div class="save_menu"><div class="icon"></div></div><div class="share_menu"><div class="icon"></div></div><a class="screen_reader_only" rel="nofollow"
href="/mobile/send-to-mobile-accessible?entryId=719183066N00W&amp;listingId=719183066N00W&amp;searchType=R&amp;channel=WP"
name="Smith">Send this listing to your mobile</a></div><span class="phone_number ">0457 599 539</span>
<div class="address"><span class="street_line">1 Martin Pl</span><span class="locality">Sydney</span><span class="state">NSW</span><span class="postcode">2000</span></div><a rel="nofollow"
class="show_map"
name="Smith"
href="/search/where-is?locality=Sydney&streetNumber=1&streetName=Martin&streetType=Pl&state=NSW&amp;product=N00W%23719183066N00W%23Smith+J&amp;channel=WP"
onclick="return false;">Show map...</a></div></div></div></div>


I am trying
if(!$html->find('div[id=entry_' .$i.']',0)==""){ 
echo "inside0000"; 
foreach($html->find('div[id=entry_' .$i.']') as $result){ 
$resultdata[]=array( 
'name' => $result->find('h[class=" "]',0)->innertext, 
'streetLine' => $result->find('span[class=street_line]',0)->innertext, 
'locality' => $result->find('span[class=locality]',0)->innertext, 
'state' => $result->find('span[class=state]',0)->innertext, 
'postcode' => $result->find('span[class=postcode]',0)->innertext, 
'phone' => $result->find('span[phone_number ]',0)->innertext 
); 


It gets Into

inside0000

But doesn't Parse the Data.

Can anyone help me please ?

Gidday all,

My Utimate goal is to parse the data on the first row in first table and first row in second table.
from he http://www.bom.gov.au/products/IDQ60901/IDQ60901.94580.shtml

Presently I can only parse data in the last row in the last table.

I got to this point about 2 days ago, I am unable to find any info as to what I need to do to achieve what I want.
some of the info I've found I don't understand.

Need newbie help.

What do I need to add/change to parse the data in  at least the first table row?

Code: [Select]

<?php
error_reporting(E_ALL);
include_once('htmldom/simple_html_dom.php');
$url = 'http://www.bom.gov.au/products/IDQ60901/IDQ60901.94580.shtml';


// Create DOM from URL
$html = file_get_html($url);

foreach($html->find('table tr') as $weather) {
    if($weather->find('th')) {continue;} //apparently this needs to be added because there is a bug in simple_html_dom.php
    if(!$weather->find('td ', 0)) {continue;}
    
    $datetime = $weather->find('td', 0)->plaintext;
    $currentTemp = $weather->find('td', 1)->plaintext;

}
 
print_r('updated:' . '&nbsp' .$datetime);
print_r ('<br>');
print_r('CurrentTmp:' . '&nbsp' .$currentTemp);
print_r ('<br>');
?>

I successfully load a page by simple_html_dom.php (developed in simplehtmldom.sourceforge.net) as
$html = file_get_html('externalpage');

But sometimes this make a high load on CPU and the page does not load for a long time (probably due to the external site server). How can I skip the process when it is not normal to avoid high CPU usage?

hi,
i have this xml

  <m time="2012-03-09T11:14:20+00:00" timestamp="1331291660">
  <ma id="1219457" xsid="0">
  <time>2012-03-09T19:30:00+00:00</time>
  <gru id="8388">Nacional</gru>
  <ht id="2325">Teste</ht>
  <at id="8919">Teste2</at>
  <results />
  <mar did="6" name="Under">
  <ofr id="95690814" n="2" ot="0" last_updated="2012-03-09T11:13:35+00:00" flags="1" bmoid="1000095485">
  <ors i="0" time="2012-03-08T18:59:22+00:00" starting_time="2012-03-09T19:30:00+00:00">
  <a1>4</a1>
  <a2>3.5</a2>
  <a3>4</a3>
  <a4>2</a4>
  <a5>8</a5>
  </ors>
  </ofr>
  </mar>
  </ma>
  </m>

and this code:

 $DOMDocument = new DOMDocument( '1.0' , 'utf-8' );
      $DOMDocument->preserveWhiteSpace = false;
      $DOMDocument->loadXML( $xml );
      foreach ( $DOMDocument->getElementsByTagName( '*' ) as $Nodes ) {
                  foreach ( $Nodes->getElementsByTagName( '*' ) as $Node ) {
                              $Data[ $Node->parentNode->nodeName ][ $Node->nodeName ] = $Node->nodeValue;

  }

with this code i can load the value of the a1 and a2 etc but i need load the name of the mar ( Under ) and the did.

how can i do this?
thanks