PHP - Simple Html Dom Parser: Starting Points For A Very Easy Example

Hello dear Community,

i have a document i need to parse it and spit out only this part of the table:

see http://schulnetz.nibis.de/db/schulen/schule.php?schulnr=67003&lschb=

how to i parse the stuff!? With perl or php?

Note i have the xpaths (see below) Sad that i cannot apply them on Simple DOM Parser since this Dom Parser does not work with Xpaths but with  CSS-Selectors:

Well i want to get all the data with that are within the table that name is called  class="fliess"

How to dump all the results?
BTW - thinking about the most elegant way, i think it is the most pretty way would be to do it with perl - So i can try it with HTML::TableExtract or....

Well what do you suggest - Which way to choose to do this [very] simple thing?


Look forward to hear from you!

see the xpaths:

Schule:  /html/body/center/table/tbody/tr[2]/td[1]
Stasse: /html/body/center/table/tbody/tr[3]/td[1]
Ort:     /html/body/center/table/tbody/tr[4]/td[1]
Tel:     /html/body/center/table/tbody/tr[5]/td[1]
Schulgliederungen:  /html/body/center/table/tbody/tr[6]/td[1]
Besonderheite: /html/body/center/table/tbody/tr[7]/td[1]
E-Mail: /html/body/center/table/tbody/tr[8]/td[1]
Schulnummer: /html/body/center/table/tbody/tr[9]/td[1]

Hi All,

I am using the PHP Simple HTML DOM parser to connect to a financials website, parse out a companies financial information (Income statement in this case) and then insert the scrapped data into a mysql database that I can then later use to run automated calculations.

Here is the code I have so far:

Code: [Select]
<?php
include_once 'simple_html_dom.php';

//Connect to financial Website and Create DOM from URL
$income_statement = file_get_html('http://www.WEBSITE.com/finance?etc..etc...etc...etc...');

//PULL FINANCIAL DATA
foreach($income_statement->find('td[class]' ) as $lines=>$data) {

echo $data->plaintext . "<br/>";

}

// clean up memory
    $html->clear();
    unset($html);
?>

So far I am able to get output that looks like this:

Code: [Select]
Revenue
336.57
331.52
324.32
319.29
320.40
Other Revenue, Total
-
-
-
-
-
Total Revenue
336.57
331.52
324.32
319.29
320.40
etc.............................
But being a newb I do not understand how I can break each $ value and each - into their own variables and then insert them to their corresponding mysql table fields. During the database insert I would like to ignore field headings from insertion (i.e Revenue, Total Revenue, etc....

Any help would be absolutely amazing, as I have been reading, scripting and searching for information like crazy, but just can't seem to figure it out.

I'm using PHP 5.2 Server and Simple HTML DOM 1.5. This script scrape or extract data from a football site, its fully working on PHP 5.9 Server but I need to know how I can fix it for PHP 5.2 server. Can someone give me a hint on how can I fix the error? Thanks in advance.

My PHP 5.2 Server script output shows:
++++++++++++++++
Object id #599 Object id #604 Object id #609 Object id #614 Object id #619
Object id #627 Object id #632 Object id #637 Object id #642 Object id #647
Object id #655 Object id #660 Object id #665 Object id #670 Object id #675
Object id #683 Object id #688 Object id #693 Object id #698 Object id #703
Object id #711 Object id #716 Object id #721 Object id #726 Object id #731
++++++++++++++++

while PHP 5.9 Server says
++++++++++++++++
Rk Player Team POS OPPONENT
1 Aaron Rodgers GB QB at CAR
2 Tom Brady NE QB vs. SD
3 Matt Schaub HOU QB at MIA
4 Michael Vick PHI QB at ATL
++++++++++++++++

I did applied the bug solution listed on https://sourceforge.net/tracker/index.php?func=detail&aid=3107230&group_id=218559&atid=1044037 but it is still not working. It says:
++++++++++++++++
Details:

I get compiler errors in PHP 5.2 when using this as an object.

The offending lines are 609 and 940, which both contain this construct:

if ($this->size>0) $this->char = $this->doc[0];

This tries to get the first character of $this->doc, but PHP 5.2 sees it as trying to access it as an array. It's easily fixed by this:

if ($this->size>0) $this->char = substr($this->doc, 0, 1);

Or you could probably use chr(ord($this->doc)) as well. Either way solves the compile error without changing functionality.
++++++++++++++++

Here are my codes:

Code: [Select]
<?php
# don't forget the library
include('simple_html_dom.php');

# this is the global array we fill with article information
$articles = array();
$source = 'http://www.athlonsports.com/columns/winning-game-plan/fantasy-football-qb-rankings';
# passing in the first page to parse, it will crawl to the end
# on its own
getArticles($source);      


function getArticles($page) {
global $articles, $descriptions;

$html = new simple_html_dom();
$html->load_file($page);

//$items = $html->find('div[class=preview]');
$items = $html->find('tbody tr');

foreach($items as $post) {
    # remember comments count as nodes
   /*$articles[] = array($post->children(3)->outertext,
                        $post->children(6)->first_child()->outertext);*/
    $articles[] = array($post->children(0), $post->children(1), $post->children(2), $post->children(3), $post->children(4));
}

# lets see if there's a next page
if($next = $html->find('a[class=nextpostslink]', 0)) {
    $URL = $next->href;
    echo "going on to $URL <<<\n";
    # memory leak clean up
   $html->clear();
    unset($html);

    getArticles($URL);
}
}

?>


<html>
<head>
</head>
<body>
<?
echo "Source: " . $source;
?>
<table cellpadding="5" cellspacing="0" border="0">
<?php
    foreach($articles as $item) {
        echo "<tr>";
        echo "<td>" . $item[0] . "</td><td>" . $item[1] . "</td><td>" . $item[2] . "</td>";
        echo "<td>" . $item[3] . "</td><td>" . $item[4] . "</td>";
        echo "<tr>";
    }
?>
</table>


</body>
</html>

good day dear community,


this is a big issue. I have to decide: between native PHP DOM Extension or of simple DOM html parser

well i want to parse the site he http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

I will suggest to use the native PHP "DOM" Extension instead of "simple html parser", since it will be much faster and easier

What do you think about this one here...:

Code: [Select]
$doc = new DOMDocument
@$doc->loadHTMLFile('...URL....'); // Using the @ operator to hide parse errors
$contents = $doc->getElementById('content')->nodeValue; // Text contents of #content


look  forward to hear from you

best regards
db1

Im using some software called php html dom parser i wont to be able to keep the souce tidy

i.e before dom parser

<?php

//////////////////////SEO TOOL///////////////////////////

$title = 'Green Deal Nationwide - PB Energy Solutions Ltd';
$description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';

///////////////////////////////////////////////////////

?>
<?php include('includes/settings.php'); ?>
<?php include('includes/header.php'); ?>

    <div class="container">

        <div id="large-page-img">
        <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />
        </div>
        <div id="content-home">
     
            <div class="iedit">

after dom parser saved to file

<?php    //////////////////////SEO TOOL///////////////////////////    $title = 'Green Deal Nationwide - PB Energy Solutions Ltd';  $description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';    ///////////////////////////////////////////////////////    ?>  <?php include('includes/settings.php'); ?>  <?php include('includes/header.php'); ?>        <div class="container">            <div id="large-page-img">          <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />          </div>          <div id="content-home">                      <div class="iedit"><div class="iedit">

is there anyway i can keep it like the original fil after dom?

So I started learning how to code php with usercake. I then went to use different things like fileNice for directory listing and I went to make a music lister with music player running mainly (EDM) .mp3's. I have uploaded a copy of my music.php file and I would love some help on how to develop my site further!   https://github.com/3...aster/music.php   This is the part I need help developing:

<?php

  function getUploadedFiles($directory){
    $folder = opendir($directory);
    while ($file = readdir($folder)) {
    if($file !== '.' && $file !== '..')
    {
        $filename = $directory.''.$file;
        $file_array[] = array('filename' => $file);
    }
}
return $file_array;
  }

$music = getUploadedFiles('../Music/EDM');

foreach($music as $value)
{
	echo '<div class="well well-sm">';
	echo '<li class="list-group-item list-group-item-success"><div class="alert alert-success" role="alert"><a href="../Music/EDM/'.$value['filename'].'"title="Play .$value" class="sm2_button"></a> '.$value['filename'].'</div></li>';
	echo '<p><a href="../Music/EDM/'.$value['filename'].'"><button type="button" class="btn btn-sm btn-primary"><span class="glyphicon glyphicon-headphones"> </span>&nbsp; Listen</button></a>&nbsp;';
	echo '<a href="../Music/EDM/'.$value['filename'].'" download><button type="button" class="btn btn-sm btn-success"><span class="glyphicon glyphicon-cloud-download"></span>&nbsp; Download</button></p></br></a></p>';
	echo '<p><a class="btn btn-default" href="#" role="button">View details &raquo;</a></p>';
	echo '</div>';
	}

  ?>

I would love to figure out how to use "echo '<p><a class="btn btn-default" href="#" role="button">View details &raquo;</a></p>';" to show time of when song was added, so Time last modified ?  Hopefully I can learn how to set individual counters instead of them being the same for all the songs :/     Here is a link to the site with everything running: http://thebatcave.x10.mx/dev/music.php Tell me what you think ?
Edited by 321System123, 10 August 2014 - 08:45 PM.

how should i approach the following:
a page with a products list+link to product page

i want to build a crawler that loops through all the products in the list and goes to the product page and
and parses the product page.

need help with the loop

hello dear Freaks 

 

i am currently  musing bout the portover of a python bs4 parser to php -  working with the simplehtmldom-parser / pr the DOM-selectors... (see below).

The project: for a list of meta-data of wordpress-plugins: - approx 50 plugins are of interest! but the challenge is: i want to fetch meta-data of all the existing plugins. What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp - that are updated (most) recently. It is all aobut acutality...

https://wordpress.org/plugins/participants-database ....and so on and so forth.  

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.
 

we have the following set of meta-data for each wordpress-plugin:

Version: 1.9.5.12 
installations: 10,000+    
WordPress Version: 5.0 or higher 
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags:databasemembersign-up formvolunteer
Last updated: 19 hours ago

 

the project consits of two parts:   the looping-part: (which seems to be pretty straightforward). the parser-part: where i have some issues - see below. I'm trying to loop through an array of URLs and scrape the data below from a list of wordpress-plugins. See my loop below-

as a base i think it is good starting point to work from the following target-url: 

 plugins wordpress.org/plugins/browse/popular with 99 pages of content: cf ...
 wordpress.org/plugins/browse/popular/page/1 
wordpress.org/plugins/browse/popular/page/2
wordpress.org/plugins/browse/popular/page/99

 

the Output of text_nodes:

['Version: 1.9.5.12', 'Active installations: 10,000+', 'Tested up to: 5.6 ']  

but if we want to fetch the data of all the wordpress-plugins and subesquently sort them to show the -let us say - latest 50 updated plugins. This would be a interesting task:

 

first of all we need to fetch the urls

then we fetch the information and have to sort out the newest- the newest timestamp. Ie the plugin that updated most recently

List the 50 newest items - that are the 50 plugins that are updated recently ..

 

we have the following set
 

see here the Soup_ 

 soup = BeautifulSoup(r.content, 'html.parser')
        target = [item.get_text(strip=True, separator=" ") for item in soup.find(
            "h3", class_="screen-reader-text").find_next("ul").findAll("li")[:8]]
        head = [soup.find("h1", class_="plugin-title").text]
        new = [x for x in target if x.startswith(
            ("V", "Las", "Ac", "W", "T", "P"))]
        return head + new


with ThreadPoolExecutor(max_workers=50) as executor1:
    futures1 = [executor1.submit(parser, url) for url in allin]

for future in futures1:
    print(future.result())

 

see the formal output

 

Quote

[lorem ipsum dolor sit amet', 'Version: 2.34.1', 'Last updated: 5 months ago', 'Tags: magna aliquyam erat, sed diam voluptua. At vero eos et accusam']
[consetetur sadipscing elitr', 'Version: 6.54.1', 'Last updated: 5 months ago', 'Tags: lorem ipsum dolor sit amet']
[sed diam nonumy eirmod tempor invidunt ut labore', 'Version: 7.16.1', 'Last updated: 5 months ago', 'Tags: tarifa, sevilla lisabin invidunt ut labore et dolore magna aliquyam erat']
[tempor invidunt ut taria malaga jerusalem labore', 'Version: 9.58.1', 'Last updated: 5 months ago', 'Tags: ilabore et lissabon dolore magna aliquyam erat']

 

background: https://stackoverflow.com/questions/61106309/fetching-multiple-urls-with-beautifulsoup-gathering-meta-data-in-wp-plugins

Well - i guess that we c an do this with the simple DOM Parser - here the seclector reference. 

https://stackoverflow.com/questions/1390568/how-can-i-match-on-an-attribute-that-contains-a-certain-string

 

look forward to any hint and help.

 

have a great day 

Edited May 3, 2020 by dil_bert

say if I had a black page with a background or something simple and it had no content except for a link called "about us"
and I wanted that link to be located at the bottom left side of the page but since it has no content its stuck up the top unless <br> or <p> is introduced.
how could this be done without spamming <br /><p><br /> untill im happy with its location and if content is added it gets boosted down further with a massive gap ><

hi,

i am trying to filter my gps history points.

i would like to ignore points if they are too close to the previous position.

i can do this like so.

$old_lat = $old_long = "0";

                 foreach($history['data'] as $record) {

//calculate distance in meters
$distance = distance($record['latitude'], $record['longitude'], $old_lat, $old_long, "V");

if($distance >= 300) {
echo 'add(jQuery(this), number += 1, "' . date("d-m-Y @ h:i:s",$record['timestamp']) . '", "map_post.php?n=' . $name . 'u=' . $history['user'] . '", "' . $history['user'] . '", "' . $record['latitude'] . '", "' . $record['longitude'] . '");';
                
$old_lat = $record['latitude'];
$old_long = $record['longitude'];
}

the problem with the above is that if i arrive home at 6pm on friday and stay outside my house for 24hours, it will show the latest position as at 6pm on fri.

i would like it to show the 6pm friday position but with a date 'from and to' time like (28/10/11 18:00 - 29/10/11 18:00) rarther than just (28/10/11 18:00) which looks inacurate.

Possible?

Ok i have 3 columms right now i have  date  player and points and what i want is for this script to add up each players points and send it to page  as totals decending i have got the page to show them individually but when i do group it only gets one record so i tried to add in the  sum points and it gives me a  ) Warning: mysql_fetch_assoc() expects parameter 1 to be resource, boolean given in so i am kind of lost as to where adding the sum(points) would cause this below is the code where the problem is any help would be much appreciated

$r = mysql_query("select  * from player_points Sum(points) as total where date_entered > '$d' Group by player order by  total DESC");

Okay, I'm trying to make it so that when someone levels up in my game, they can use stat bonus points to increase their abilities.

However, there doesn't seem to be a break in there to ensure that someone can't spend points that they don't have. For instance, they get 3 per level, but if they spend 6, the system lets them and shows a -3 in their stat_bonus field.

Any help would be appreciated! Thanks in advance!!


<?php

include("fightfunc.php");
include("lib.php");
//include('cookies.php');
//$link = opendb();
define("PAGENAME", "Stat Bonus");
$player = check_user($secret_key, $db);
$x = $player->username;
if($_SESSION['234asdfas']){$y = $_SESSION['234asdfas'];}else{echo 'Error: You do not have a session variable set'; exit;}

$command = 1;

$choice = $_POST['choice'];
$amount = $_POST['amount'];

if (isset($_POST["statbonus"])) {
       
   include("tmpconfig.php");
$checkquery = $db->execute("SELECT * FROM `users` WHERE `id`=?", $player->id);
//$rec = mysql_fetch_array($checkquery);
extract($_POST);
$errors = 0;
$errorlist = "";
        
if ($errors == 0) { 

$query = $db->execute("UPDATE `users` SET `stat_bonus`=? WHERE `id`=?", array($player->stat_bonus - $amount, $player->id)) or die("querya failed: ". mysql_error());
if($query)
$qry = $db->execute("select * from `users` where `id`=?", array($player->id)) or die("qry failed: ".mysql_error());
$rw = $qry->fetchrow();

if($qry){
$query2 = $db->execute("UPDATE `users` SET `stat_bonus`=?, $choice=$choice+$amount WHERE `id`=?", array($player->stat_bonus - $amount, $player->id)) or die("query2 failed: ". mysql_error());
if($query2){
echo("<br><br>You have successfully used your stat bonus points.<br>");
} elseif  
($amount > $player["stat_bonus"])
{ echo "<font color='red'>Enter an amount higher than 0!</font>"; }
  elseif
  ($amount == 0)
   { echo "<font color='red'>Enter an amount higher than 0!</font>";  }
  elseif
  (!is_numeric($amount)) { 
echo "<font color='red'>Enter an amount!</font.";
        } 
else {
echo "<font color=green><b>You have succesfully spend skill points!</b></font><br /><br />";
}}}}
$player = check_user($secret_key, $db);
?>
<table width="100%">
<tr>
  <td colspan="2"><fieldset>
<form action="" method="post">
<font size='4'><b>Skill points</b></font><br />
Every time you raise a level, you get 3 skillpoints. You can spend them here.
<br />
You currently have <font color=green><b><?=$player->stat_bonus?> Skill Points.</b></font>. Please choose an attribute:<br />
<br />
<INPUT TYPE="radio" NAME="choice" VALUE="strength" CHECKED>Strength<br />
<INPUT TYPE="radio" NAME="choice" VALUE="vitality" >Vitality<br />
<INPUT TYPE="radio" NAME="choice" VALUE="agility">Agility<br />
<INPUT TYPE="radio" NAME="choice" VALUE="wisdom">Wisdom<br />
<br />
How many skill points do you want to spend? (+1 for every skill point): <br /><br />
Amount: <INPUT TYPE="text" NAME="amount" SIZE="8" MAXLENGTH="8">
<INPUT TYPE="submit" NAME="statbonus" VALUE="Submit">
</form></fieldset></td></tr>
    </table>