PHP - Native Php Dom Extension Versus Simple Dom Html Parser

Hi All,

I am using the PHP Simple HTML DOM parser to connect to a financials website, parse out a companies financial information (Income statement in this case) and then insert the scrapped data into a mysql database that I can then later use to run automated calculations.

Here is the code I have so far:

Code: [Select]
<?php
include_once 'simple_html_dom.php';

//Connect to financial Website and Create DOM from URL
$income_statement = file_get_html('http://www.WEBSITE.com/finance?etc..etc...etc...etc...');

//PULL FINANCIAL DATA
foreach($income_statement->find('td[class]' ) as $lines=>$data) {

echo $data->plaintext . "<br/>";

}

// clean up memory
    $html->clear();
    unset($html);
?>

So far I am able to get output that looks like this:

Code: [Select]
Revenue
336.57
331.52
324.32
319.29
320.40
Other Revenue, Total
-
-
-
-
-
Total Revenue
336.57
331.52
324.32
319.29
320.40
etc.............................
But being a newb I do not understand how I can break each $ value and each - into their own variables and then insert them to their corresponding mysql table fields. During the database insert I would like to ignore field headings from insertion (i.e Revenue, Total Revenue, etc....

Any help would be absolutely amazing, as I have been reading, scripting and searching for information like crazy, but just can't seem to figure it out.

Hello dear Community,

i have a document i need to parse it and spit out only this part of the table:

see http://schulnetz.nibis.de/db/schulen/schule.php?schulnr=67003&lschb=

how to i parse the stuff!? With perl or php?

Note i have the xpaths (see below) Sad that i cannot apply them on Simple DOM Parser since this Dom Parser does not work with Xpaths but with  CSS-Selectors:

Well i want to get all the data with that are within the table that name is called  class="fliess"

How to dump all the results?
BTW - thinking about the most elegant way, i think it is the most pretty way would be to do it with perl - So i can try it with HTML::TableExtract or....

Well what do you suggest - Which way to choose to do this [very] simple thing?


Look forward to hear from you!

see the xpaths:

Schule:  /html/body/center/table/tbody/tr[2]/td[1]
Stasse: /html/body/center/table/tbody/tr[3]/td[1]
Ort:     /html/body/center/table/tbody/tr[4]/td[1]
Tel:     /html/body/center/table/tbody/tr[5]/td[1]
Schulgliederungen:  /html/body/center/table/tbody/tr[6]/td[1]
Besonderheite: /html/body/center/table/tbody/tr[7]/td[1]
E-Mail: /html/body/center/table/tbody/tr[8]/td[1]
Schulnummer: /html/body/center/table/tbody/tr[9]/td[1]

I'm using PHP 5.2 Server and Simple HTML DOM 1.5. This script scrape or extract data from a football site, its fully working on PHP 5.9 Server but I need to know how I can fix it for PHP 5.2 server. Can someone give me a hint on how can I fix the error? Thanks in advance.

My PHP 5.2 Server script output shows:
++++++++++++++++
Object id #599 Object id #604 Object id #609 Object id #614 Object id #619
Object id #627 Object id #632 Object id #637 Object id #642 Object id #647
Object id #655 Object id #660 Object id #665 Object id #670 Object id #675
Object id #683 Object id #688 Object id #693 Object id #698 Object id #703
Object id #711 Object id #716 Object id #721 Object id #726 Object id #731
++++++++++++++++

while PHP 5.9 Server says
++++++++++++++++
Rk Player Team POS OPPONENT
1 Aaron Rodgers GB QB at CAR
2 Tom Brady NE QB vs. SD
3 Matt Schaub HOU QB at MIA
4 Michael Vick PHI QB at ATL
++++++++++++++++

I did applied the bug solution listed on https://sourceforge.net/tracker/index.php?func=detail&aid=3107230&group_id=218559&atid=1044037 but it is still not working. It says:
++++++++++++++++
Details:

I get compiler errors in PHP 5.2 when using this as an object.

The offending lines are 609 and 940, which both contain this construct:

if ($this->size>0) $this->char = $this->doc[0];

This tries to get the first character of $this->doc, but PHP 5.2 sees it as trying to access it as an array. It's easily fixed by this:

if ($this->size>0) $this->char = substr($this->doc, 0, 1);

Or you could probably use chr(ord($this->doc)) as well. Either way solves the compile error without changing functionality.
++++++++++++++++

Here are my codes:

Code: [Select]
<?php
# don't forget the library
include('simple_html_dom.php');

# this is the global array we fill with article information
$articles = array();
$source = 'http://www.athlonsports.com/columns/winning-game-plan/fantasy-football-qb-rankings';
# passing in the first page to parse, it will crawl to the end
# on its own
getArticles($source);      


function getArticles($page) {
global $articles, $descriptions;

$html = new simple_html_dom();
$html->load_file($page);

//$items = $html->find('div[class=preview]');
$items = $html->find('tbody tr');

foreach($items as $post) {
    # remember comments count as nodes
   /*$articles[] = array($post->children(3)->outertext,
                        $post->children(6)->first_child()->outertext);*/
    $articles[] = array($post->children(0), $post->children(1), $post->children(2), $post->children(3), $post->children(4));
}

# lets see if there's a next page
if($next = $html->find('a[class=nextpostslink]', 0)) {
    $URL = $next->href;
    echo "going on to $URL <<<\n";
    # memory leak clean up
   $html->clear();
    unset($html);

    getArticles($URL);
}
}

?>


<html>
<head>
</head>
<body>
<?
echo "Source: " . $source;
?>
<table cellpadding="5" cellspacing="0" border="0">
<?php
    foreach($articles as $item) {
        echo "<tr>";
        echo "<td>" . $item[0] . "</td><td>" . $item[1] . "</td><td>" . $item[2] . "</td>";
        echo "<td>" . $item[3] . "</td><td>" . $item[4] . "</td>";
        echo "<tr>";
    }
?>
</table>


</body>
</html>

Hello dear friends,

first of all : merry merry Xmas!!!


i want to parse with the simple Simple HTML DOM Parser,


well i am pretty new to php and to the Simple HTML DOM Parser.

My example: http://schulen.bildung-rp.de/gehezu/startseite/einzelanzeige.html?tx_wfqbe_pi1[uid]=60119

I want to collect the data in the block:

I have investigated the sourcecode - and found out that the attribute of interest should be this one: class="content"div class="content"><!-- TYPO3SEARCH_begin -->


here the code is: - my trails.

 

// inculde the Simple HTML DOM Parser 
include_once('simple_html_dom.php');

// get the file we want to parse right now,create a DOM 
$html = file_get_html('');

// simple_html_dom::find() creates a new 
// simple_html_dom-Objekt, that consists out of
// corresponding childelements 

foreach($html->find('class: content ') as $h3) {

  // simple_html_dom::get the text in a tag
  // den Text innerhalb eines Tags
  if($h3->innertext == 'Text of a H3 Tag') {
    break;
  }
}

// simple_html_dom::next_sibling() gives the
// next   Element
$table = $h3->next_sibling();



but believe me - it  gives me not back what is aimed.

what  have id done wrong...? 

dbone

I understand that I can place my HTML code outisde of my PHP tags and have it publish, OR i can put it inside the PHP tags and ECHO every line.

Am I correct in thinking that either will provide the same result?

Is there an advantage to one method over the other??

Im using some software called php html dom parser i wont to be able to keep the souce tidy

i.e before dom parser

<?php

//////////////////////SEO TOOL///////////////////////////

$title = 'Green Deal Nationwide - PB Energy Solutions Ltd';
$description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';

///////////////////////////////////////////////////////

?>
<?php include('includes/settings.php'); ?>
<?php include('includes/header.php'); ?>

    <div class="container">

        <div id="large-page-img">
        <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />
        </div>
        <div id="content-home">
     
            <div class="iedit">

after dom parser saved to file

<?php    //////////////////////SEO TOOL///////////////////////////    $title = 'Green Deal Nationwide - PB Energy Solutions Ltd';  $description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';    ///////////////////////////////////////////////////////    ?>  <?php include('includes/settings.php'); ?>  <?php include('includes/header.php'); ?>        <div class="container">            <div id="large-page-img">          <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />          </div>          <div id="content-home">                      <div class="iedit"><div class="iedit">

is there anyway i can keep it like the original fil after dom?

how should i approach the following:
a page with a products list+link to product page

i want to build a crawler that loops through all the products in the list and goes to the product page and
and parses the product page.

need help with the loop

hello dear Freaks 

 

i am currently  musing bout the portover of a python bs4 parser to php -  working with the simplehtmldom-parser / pr the DOM-selectors... (see below).

The project: for a list of meta-data of wordpress-plugins: - approx 50 plugins are of interest! but the challenge is: i want to fetch meta-data of all the existing plugins. What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp - that are updated (most) recently. It is all aobut acutality...

https://wordpress.org/plugins/participants-database ....and so on and so forth.  

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.
 

we have the following set of meta-data for each wordpress-plugin:

Version: 1.9.5.12 
installations: 10,000+    
WordPress Version: 5.0 or higher 
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags:databasemembersign-up formvolunteer
Last updated: 19 hours ago

 

the project consits of two parts:   the looping-part: (which seems to be pretty straightforward). the parser-part: where i have some issues - see below. I'm trying to loop through an array of URLs and scrape the data below from a list of wordpress-plugins. See my loop below-

as a base i think it is good starting point to work from the following target-url: 

 plugins wordpress.org/plugins/browse/popular with 99 pages of content: cf ...
 wordpress.org/plugins/browse/popular/page/1 
wordpress.org/plugins/browse/popular/page/2
wordpress.org/plugins/browse/popular/page/99

 

the Output of text_nodes:

['Version: 1.9.5.12', 'Active installations: 10,000+', 'Tested up to: 5.6 ']  

but if we want to fetch the data of all the wordpress-plugins and subesquently sort them to show the -let us say - latest 50 updated plugins. This would be a interesting task:

 

first of all we need to fetch the urls

then we fetch the information and have to sort out the newest- the newest timestamp. Ie the plugin that updated most recently

List the 50 newest items - that are the 50 plugins that are updated recently ..

 

we have the following set
 

see here the Soup_ 

 soup = BeautifulSoup(r.content, 'html.parser')
        target = [item.get_text(strip=True, separator=" ") for item in soup.find(
            "h3", class_="screen-reader-text").find_next("ul").findAll("li")[:8]]
        head = [soup.find("h1", class_="plugin-title").text]
        new = [x for x in target if x.startswith(
            ("V", "Las", "Ac", "W", "T", "P"))]
        return head + new


with ThreadPoolExecutor(max_workers=50) as executor1:
    futures1 = [executor1.submit(parser, url) for url in allin]

for future in futures1:
    print(future.result())

 

see the formal output

 

Quote

[lorem ipsum dolor sit amet', 'Version: 2.34.1', 'Last updated: 5 months ago', 'Tags: magna aliquyam erat, sed diam voluptua. At vero eos et accusam']
[consetetur sadipscing elitr', 'Version: 6.54.1', 'Last updated: 5 months ago', 'Tags: lorem ipsum dolor sit amet']
[sed diam nonumy eirmod tempor invidunt ut labore', 'Version: 7.16.1', 'Last updated: 5 months ago', 'Tags: tarifa, sevilla lisabin invidunt ut labore et dolore magna aliquyam erat']
[tempor invidunt ut taria malaga jerusalem labore', 'Version: 9.58.1', 'Last updated: 5 months ago', 'Tags: ilabore et lissabon dolore magna aliquyam erat']

 

background: https://stackoverflow.com/questions/61106309/fetching-multiple-urls-with-beautifulsoup-gathering-meta-data-in-wp-plugins

Well - i guess that we c an do this with the simple DOM Parser - here the seclector reference. 

https://stackoverflow.com/questions/1390568/how-can-i-match-on-an-attribute-that-contains-a-certain-string

 

look forward to any hint and help.

 

have a great day 

Edited May 3, 2020 by dil_bert

Hi first post here and very new to php. I built a form so that a data entry perosn can "build" new pages for our companies website. I pretty much have the basics of the form and the action page coded correctly, however my issue is that I need this final page where the form inputs are posted too to be an html document.

I read some about the str_replace() function, but I'm not understanding how I could use that to replace the file extension of a page.

Maybe I'm going about this all wrong. I also read that one can edit the htaccess folder to allow an html file to run php script. Not sure which way would be the best. One thing to mention though, is this application would be run locally to our back up files which is then scheduled to be uploaded to the remote server very night.

Any help or suggestions would be greatly appreciated.

I have a website that I did for my uncle and now, the requirements are going beyond my html knowledge and into the realm of server scripting.

My issue is that I need to loop through all the images in a given folder and subfolders and preload them into the html so that the user can view them within a slide show.  There are currently 6 subfolders with over 20 pics in each and there are more pics to come and different folders to add.  I have a media page with the first picture within each category (subfolder) displayed as an eye catcher to the category.  once the user clicks on the image, the slide show will start in a popup.  The problem is that with js, I have to list out every image since it cannot look through a directory listing.  I would like to simply call a server side script that will create an array of the images and preload them so that when the slide show is ran, the appropriate images will show.

I am found numerous scripts for preloading images, written in php, but I am unsure how to implement them and keep a html extension for my pages.  As the site is written entirely in html and javascript, I would like to keep the html extentions if possible. 

As this is my first site, I am sure that server side scripting will become a bigger role, but baby steps are what I need right now.  In addition, if anyone has a good site for learning the server scripting world, I would greatly appreciate the reference.

I am working on a PHP project that will allow a user to upload a handful of images, and then submit text to go along with it.

To simplify the uploading process of a file, I restricted the type of file that can be uploaded (jpg) and the size (2.5mb). The problem is, the form is calling to a PHP script, and I have error logging built in to alert the user of why a file didn't upload, but since the PHP script is just being called, no errors are being displayed on the page. I guess unless I do a refresh of a few seconds to view any errors on the page....

What I would like to do is show a pop-up/error message notifying the user that the file type or size is incorrect and needs to be adjusted. Now part of the problem is, the file needs to be uploaded to the server first, and then everything can be checked, so I am really getting confused.

If someone can point me in the right direction on what I could use to accomplish this, it would be greatly appreciated. Thanks

-beemer

Hey all   I have been building sites for a long time and I am pretty confident in modern frameworks like Bootstrap etc. I have played about with JQuery Mobile and UI. I know how to make mobile sites from a front end side. I understand that I can upload my site to PhoneGap and turn it into a native app.   However, I would love to know how to make useful apps that dont just display static text. E.g. using things like Google Places API.   I know how to make mobile sites easily enough that display info. I just don't really understand how to link that up with dynamic data from APIs etc.  How to feed that content back to the user. For a vasic example, say I wanted to have a simple app that had a search box. You enter 'PIZZA' into it and it comes back with a result of all local pizza spots. How would I do that?   If someone could point me in the right direction for where I could experiment with this I would be really grateful .

There is a picture with dimensions of 10000 by 10000 pixels. Need to display a picture with dimensions of 200 by 100. Only need to use native php.

Hello,

My current server setup :

PHP version: 7.1.4
MySQL 5.5.28 
Database client version: libmysql - mysqlnd 5.0.12-dev

I'm using PDO _MYSQL to run database queries. 

From what I read, in order to have MySQL return native datatypes (as opposed to just strings) : 
 

if using MySQLi: set $mysqli->options(MYSQLI_OPT_INT_AND_FLOAT_NATIVE, 1) if using PDO and have mysqlnd:  "mysqlnd returns native data types when using Server-side Prepared Statements, for example an INT column is returned as an integer variable not as a string."

Now, I do have mysqlnd, but a simple PDO Prepared statement still returns me all strings, even though some columns are set as INT, VARCHAR, DATETIME, etc..

$stmt = $pdo->prepare("SELECT * FROM tbl_users WHERE id=1");
$stmt->execute();
$user = $stmt->fetch();

var_dump($user);

OUTPUT:
array(13) {
  ["id"]=>
  string(1) "1"
  ["is_admin"]=>
  string(1) "1"
  ["is_active"]=>
  string(1) "1"
  ["username"]=>
  string(5) "jdoe"
...
}

 

I'm not sure what to look for to resolve this issue if I want it to return INT as INT and not STRING, etc. 

Does having `extension_loaded('mysqlnd') == true`,  make my system automatically use this native mysql driver? How can I tell ? Could this be a version problem perhaps?

 

Much thanks!

Pat