PHP - Php Simple Html Dom Parser - How To Get Up To Speed With This Approach?

Hi All,

I am using the PHP Simple HTML DOM parser to connect to a financials website, parse out a companies financial information (Income statement in this case) and then insert the scrapped data into a mysql database that I can then later use to run automated calculations.

Here is the code I have so far:

Code: [Select]
<?php
include_once 'simple_html_dom.php';

//Connect to financial Website and Create DOM from URL
$income_statement = file_get_html('http://www.WEBSITE.com/finance?etc..etc...etc...etc...');

//PULL FINANCIAL DATA
foreach($income_statement->find('td[class]' ) as $lines=>$data) {

echo $data->plaintext . "<br/>";

}

// clean up memory
    $html->clear();
    unset($html);
?>

So far I am able to get output that looks like this:

Code: [Select]
Revenue
336.57
331.52
324.32
319.29
320.40
Other Revenue, Total
-
-
-
-
-
Total Revenue
336.57
331.52
324.32
319.29
320.40
etc.............................
But being a newb I do not understand how I can break each $ value and each - into their own variables and then insert them to their corresponding mysql table fields. During the database insert I would like to ignore field headings from insertion (i.e Revenue, Total Revenue, etc....

Any help would be absolutely amazing, as I have been reading, scripting and searching for information like crazy, but just can't seem to figure it out.

I'm using PHP 5.2 Server and Simple HTML DOM 1.5. This script scrape or extract data from a football site, its fully working on PHP 5.9 Server but I need to know how I can fix it for PHP 5.2 server. Can someone give me a hint on how can I fix the error? Thanks in advance.

My PHP 5.2 Server script output shows:
++++++++++++++++
Object id #599 Object id #604 Object id #609 Object id #614 Object id #619
Object id #627 Object id #632 Object id #637 Object id #642 Object id #647
Object id #655 Object id #660 Object id #665 Object id #670 Object id #675
Object id #683 Object id #688 Object id #693 Object id #698 Object id #703
Object id #711 Object id #716 Object id #721 Object id #726 Object id #731
++++++++++++++++

while PHP 5.9 Server says
++++++++++++++++
Rk Player Team POS OPPONENT
1 Aaron Rodgers GB QB at CAR
2 Tom Brady NE QB vs. SD
3 Matt Schaub HOU QB at MIA
4 Michael Vick PHI QB at ATL
++++++++++++++++

I did applied the bug solution listed on https://sourceforge.net/tracker/index.php?func=detail&aid=3107230&group_id=218559&atid=1044037 but it is still not working. It says:
++++++++++++++++
Details:

I get compiler errors in PHP 5.2 when using this as an object.

The offending lines are 609 and 940, which both contain this construct:

if ($this->size>0) $this->char = $this->doc[0];

This tries to get the first character of $this->doc, but PHP 5.2 sees it as trying to access it as an array. It's easily fixed by this:

if ($this->size>0) $this->char = substr($this->doc, 0, 1);

Or you could probably use chr(ord($this->doc)) as well. Either way solves the compile error without changing functionality.
++++++++++++++++

Here are my codes:

Code: [Select]
<?php
# don't forget the library
include('simple_html_dom.php');

# this is the global array we fill with article information
$articles = array();
$source = 'http://www.athlonsports.com/columns/winning-game-plan/fantasy-football-qb-rankings';
# passing in the first page to parse, it will crawl to the end
# on its own
getArticles($source);      


function getArticles($page) {
global $articles, $descriptions;

$html = new simple_html_dom();
$html->load_file($page);

//$items = $html->find('div[class=preview]');
$items = $html->find('tbody tr');

foreach($items as $post) {
    # remember comments count as nodes
   /*$articles[] = array($post->children(3)->outertext,
                        $post->children(6)->first_child()->outertext);*/
    $articles[] = array($post->children(0), $post->children(1), $post->children(2), $post->children(3), $post->children(4));
}

# lets see if there's a next page
if($next = $html->find('a[class=nextpostslink]', 0)) {
    $URL = $next->href;
    echo "going on to $URL <<<\n";
    # memory leak clean up
   $html->clear();
    unset($html);

    getArticles($URL);
}
}

?>


<html>
<head>
</head>
<body>
<?
echo "Source: " . $source;
?>
<table cellpadding="5" cellspacing="0" border="0">
<?php
    foreach($articles as $item) {
        echo "<tr>";
        echo "<td>" . $item[0] . "</td><td>" . $item[1] . "</td><td>" . $item[2] . "</td>";
        echo "<td>" . $item[3] . "</td><td>" . $item[4] . "</td>";
        echo "<tr>";
    }
?>
</table>


</body>
</html>

Hello dear friends,

first of all : merry merry Xmas!!!


i want to parse with the simple Simple HTML DOM Parser,


well i am pretty new to php and to the Simple HTML DOM Parser.

My example: http://schulen.bildung-rp.de/gehezu/startseite/einzelanzeige.html?tx_wfqbe_pi1[uid]=60119

I want to collect the data in the block:

I have investigated the sourcecode - and found out that the attribute of interest should be this one: class="content"div class="content"><!-- TYPO3SEARCH_begin -->


here the code is: - my trails.

 

// inculde the Simple HTML DOM Parser 
include_once('simple_html_dom.php');

// get the file we want to parse right now,create a DOM 
$html = file_get_html('');

// simple_html_dom::find() creates a new 
// simple_html_dom-Objekt, that consists out of
// corresponding childelements 

foreach($html->find('class: content ') as $h3) {

  // simple_html_dom::get the text in a tag
  // den Text innerhalb eines Tags
  if($h3->innertext == 'Text of a H3 Tag') {
    break;
  }
}

// simple_html_dom::next_sibling() gives the
// next   Element
$table = $h3->next_sibling();



but believe me - it  gives me not back what is aimed.

what  have id done wrong...? 

dbone

good day dear community,


this is a big issue. I have to decide: between native PHP DOM Extension or of simple DOM html parser

well i want to parse the site he http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

http://buergerstiftungen.de/cps/rde/xchg/SID-A7DCD0D1-702CE0FA/buergerstiftungen/hs.xsl/db.htm

I will suggest to use the native PHP "DOM" Extension instead of "simple html parser", since it will be much faster and easier

What do you think about this one here...:

Code: [Select]
$doc = new DOMDocument
@$doc->loadHTMLFile('...URL....'); // Using the @ operator to hide parse errors
$contents = $doc->getElementById('content')->nodeValue; // Text contents of #content


look  forward to hear from you

best regards
db1

I currently use HTML strings within my PHP code to display output. And while it might not be best practice, I find it non-restrictive and I can easily add loops, manipulate variables, etc. within my display.

However in the interest of having a cleaner code I'm thinking of separating the HTML, without having to use a Template engine like Smarty.

I don't care much about replacement patterns to be honest, I don't mind some PHP code withing the HTML, however my biggest issue is loops and having to modify a variable within each loop. Say I have the following example code (Similar to what I'm using right now):

Code: [Select]

function student_output() {

    //Retrieve students from array
   $students_arr = students_info();

    $selected_student = $_GET['selected_student'];

    $output = '<div id="students_container">';
    $output .= '<div class="items_list">';

    $i = 1;

    //Loop through students
    foreach($students_arr as $key=>$value) {

if ($selected_student == $key) {
    $output .= '<div id="student_name_'.$i.'" class="selected">'.$value['student_name'].'</div>';
            $output .= '<div id="student_img_'.$i.'" class="selected"><img src="'.$value['student_image'].'" /></div>';
}
        else {
            $output .= '<div id="student_name_'.$i.'">'.$value['student_name'].'</div>';
        }

        $i++;

    }

    return $output;
   
}   


What's the best way to represent this in HTML cleanly without having too much PHP code within?

Im using some software called php html dom parser i wont to be able to keep the souce tidy

i.e before dom parser

<?php

//////////////////////SEO TOOL///////////////////////////

$title = 'Green Deal Nationwide - PB Energy Solutions Ltd';
$description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';

///////////////////////////////////////////////////////

?>
<?php include('includes/settings.php'); ?>
<?php include('includes/header.php'); ?>

    <div class="container">

        <div id="large-page-img">
        <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />
        <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />
        </div>
        <div id="content-home">
     
            <div class="iedit">

after dom parser saved to file

<?php    //////////////////////SEO TOOL///////////////////////////    $title = 'Green Deal Nationwide - PB Energy Solutions Ltd';  $description = 'Delivering all your environmental needs to \'green\' up your business, improve reputation, increase profitability and give a competitive advantage.';    ///////////////////////////////////////////////////////    ?>  <?php include('includes/settings.php'); ?>  <?php include('includes/header.php'); ?>        <div class="container">            <div id="large-page-img">          <img src="<?php echo URL(); ?>images/home-page-slide.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-1.jpg" width="911" height="230" />          <img src="<?php echo URL(); ?>images/home-page-slide-2.jpg" width="911" height="230" />          </div>          <div id="content-home">                      <div class="iedit"><div class="iedit">

is there anyway i can keep it like the original fil after dom?

how should i approach the following:
a page with a products list+link to product page

i want to build a crawler that loops through all the products in the list and goes to the product page and
and parses the product page.

need help with the loop

hello dear Freaks 

 

i am currently  musing bout the portover of a python bs4 parser to php -  working with the simplehtmldom-parser / pr the DOM-selectors... (see below).

The project: for a list of meta-data of wordpress-plugins: - approx 50 plugins are of interest! but the challenge is: i want to fetch meta-data of all the existing plugins. What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp - that are updated (most) recently. It is all aobut acutality...

https://wordpress.org/plugins/participants-database ....and so on and so forth.  

https://wordpress.org/plugins/wp-job-manager
https://wordpress.org/plugins/ninja-forms
https://wordpress.org/plugins/participants-database ....and so on and so forth.
 

we have the following set of meta-data for each wordpress-plugin:

Version: 1.9.5.12 
installations: 10,000+    
WordPress Version: 5.0 or higher 
Tested up to: 5.4 PHP  
Version: 5.6 or higher    
Tags 3 Tags:databasemembersign-up formvolunteer
Last updated: 19 hours ago

 

the project consits of two parts:   the looping-part: (which seems to be pretty straightforward). the parser-part: where i have some issues - see below. I'm trying to loop through an array of URLs and scrape the data below from a list of wordpress-plugins. See my loop below-

as a base i think it is good starting point to work from the following target-url: 

 plugins wordpress.org/plugins/browse/popular with 99 pages of content: cf ...
 wordpress.org/plugins/browse/popular/page/1 
wordpress.org/plugins/browse/popular/page/2
wordpress.org/plugins/browse/popular/page/99

 

the Output of text_nodes:

['Version: 1.9.5.12', 'Active installations: 10,000+', 'Tested up to: 5.6 ']  

but if we want to fetch the data of all the wordpress-plugins and subesquently sort them to show the -let us say - latest 50 updated plugins. This would be a interesting task:

 

first of all we need to fetch the urls

then we fetch the information and have to sort out the newest- the newest timestamp. Ie the plugin that updated most recently

List the 50 newest items - that are the 50 plugins that are updated recently ..

 

we have the following set
 

see here the Soup_ 

 soup = BeautifulSoup(r.content, 'html.parser')
        target = [item.get_text(strip=True, separator=" ") for item in soup.find(
            "h3", class_="screen-reader-text").find_next("ul").findAll("li")[:8]]
        head = [soup.find("h1", class_="plugin-title").text]
        new = [x for x in target if x.startswith(
            ("V", "Las", "Ac", "W", "T", "P"))]
        return head + new


with ThreadPoolExecutor(max_workers=50) as executor1:
    futures1 = [executor1.submit(parser, url) for url in allin]

for future in futures1:
    print(future.result())

 

see the formal output

 

Quote

[lorem ipsum dolor sit amet', 'Version: 2.34.1', 'Last updated: 5 months ago', 'Tags: magna aliquyam erat, sed diam voluptua. At vero eos et accusam']
[consetetur sadipscing elitr', 'Version: 6.54.1', 'Last updated: 5 months ago', 'Tags: lorem ipsum dolor sit amet']
[sed diam nonumy eirmod tempor invidunt ut labore', 'Version: 7.16.1', 'Last updated: 5 months ago', 'Tags: tarifa, sevilla lisabin invidunt ut labore et dolore magna aliquyam erat']
[tempor invidunt ut taria malaga jerusalem labore', 'Version: 9.58.1', 'Last updated: 5 months ago', 'Tags: ilabore et lissabon dolore magna aliquyam erat']

 

background: https://stackoverflow.com/questions/61106309/fetching-multiple-urls-with-beautifulsoup-gathering-meta-data-in-wp-plugins

Well - i guess that we c an do this with the simple DOM Parser - here the seclector reference. 

https://stackoverflow.com/questions/1390568/how-can-i-match-on-an-attribute-that-contains-a-certain-string

 

look forward to any hint and help.

 

have a great day 

Edited May 3, 2020 by dil_bert

I am wanting to make a product list page with PHP.

At the moment I using this code:
Code: [Select]
$query = "select * from products where category = 'Belts'";
$result = mysql_query($query);
while($row=mysql_fetch_assoc($result)) {
echo $row['name'];
}
The result is something like this:
http://i51.tinypic.com/2w56ljq.png

What I want is for them to be aligned like this:
http://i53.tinypic.com/w811tz.png

Eventually, I am also going to include an image with each product.
How would I go about alignment? With CSS or PHP?

I have this php page that based on what the user chooses shows them the appropriate photo galleria and i was wondering if their was any way i could speed this up this is the php code for selecting the galleria and I'm not showing the javascript that does the slide show because it s the galleriffic jquery plugin
Code: [Select]
<div id="Info">
<div id="page">
<div id="container">
            <?php
$g = mysql_real_escape_string($_GET['g']);
$query = "SELECT * FROM pinkpanther_games WHERE Gallery_no = '$g'";
$results = mysql_query($query) or die("Query failed ($_query) - " . mysql_error());
$row = mysql_fetch_assoc($results);
echo "<h1>" . $row['Day_Played'] . " vs " . $row['Opponent'] . "</h1>";
?>
<!-- Start Advanced Gallery Html Containers -->
<div id="gallery" class="content">
<div id="controls" class="controls"></div>
<div class="slideshow-container">
<div id="loading" class="loader"></div>
<div id="slideshow" class="slideshow"></div>
</div>
<div id="caption" class="caption-container"></div>
</div>
<div id="thumbs" class="navigation">
<ul class="thumbs noscript">
<?php
$x = $row['no_Pics'];
$y = 1;
$year = $row['Year_Played'];
$day = $row['Day_Played'];
$scheck = $row['Sessions'];
if ($scheck == 1){
$sessions = "Session1";
}else{
$sessions = "Session2";
}
if ($x == 0){
echo "<li> <a  class='thumb' href='../images/nopics.jpg' title=''><img src='../images/nopicsthumb.jpg ' /></a></li>";
}else if ($x == 10000){
echo "<li> <a  class='thumb' href='../images/coming.jpg' title=''><img src='../images/comingthumb.jpg ' /></a></li>";
}else{
while ($y <= $x){
echo "<li> <a  class='thumb' href='../images/Sections/pinkpanthers/" . $year . "/" . $sessions . "/" . $day . "/" . $y . ".jpg' title=''><img src='../images/Sections/pinkpanthers/" . $year . "/" . $sessions . "/" . $day . "/thumbs/" . $y . ".jpg ' /></a><div class='caption'><div class='download'><a href='../images/Sections/pinkpanthers/" . $year . "/" . $sessions . "/" . $day . "/big/" . $y . ".jpg ' target='_blank' />Download</a></div></div></li>";
$y ++  ;
}
}
?>
</ul>
</div>
<div style="clear: both;"></div>
</div>
</div>
<div id="footer"></div>

I'm looking for some direction as I approach a new challenge.

I have a table named friends_and_family and it contains name, email, and age fields.

I'm planning a party and want to invite 10 of the people that are between 20 and 35 years old.

The format I am considering would query the table and provide a list of those members that fit the age requirement.

I would like to generate that result (which I should be capable of handling) so that each person listed has a checkbox next to their name.

I imagine I will be creating a resultant form, so that I can then evaluate the result and select the checkboxes for those specific people that I want to invite.

Upon submission, an email will be sent to the people with selected checkboxes.

How should I approach this best?

A for each loop? Implode an array? 

I'm not very good with AJAX, so I'm thinking along these lines.

 

 

Hello everyone,

I am new to php and I need a confirmation if this is the correct approach for a code I wrote. Basically, I have a contacts.php page where I have a bootstrap table and a modal with some fields to add a new contact. I managed to write all the code, it works perfect but I want to know if this approach is ok.  The application will be much more complex and I don;t want to start on the wrong foot here. I have 2 files: contacts.php and add_new_contact.php.

First file, contacts.php:

<!DOCTYPE html>
<html lang="en" dir="ltr">
  <head>
    <!-- ............ -->
   </head>
  <body>
    <!-- ............ here is the page layout-->
    <!-- Then I have my modal from bootstrap -->
    
	<div class="modal fade" id="addContact" tabindex="-1" role="dialog" aria-labelledby="myModalLabel">
       <form action = "add_new_contact.php" method="post">
         <!-- .....content with all the inputs -->         
       	<button type="submit" class="btn btn-primary">Save to database</button>
    	</form>
    </div>
    
   </body>
</html>
    

Now, in form action I am telling html to go to add_new_contact.php where I wrote the code for inserting all the values to the database. When doing this, it opens the page and it stays blank because there is no html there. So, in that php file I added a redirect code to the initial contacts.php. Here is the code:

Second file: add_new_contact.php:

<?php
	//all the code needed to insert the contact in the database

	header("Location: http//..../contacts.php");
  	exit();
?>

So, this works fine. But is this the best way to do it?

Thank you.

Hi everyone,   So I'm relatively new to OO PHP and moreover, OO PHP with MVC design pattern.   This may be a largeish post so please bear with me on this one!   So here is a scenario that I'd like to understand. There are likely multiple ways to go about this, but it'd be nice to see what is said. I'll include what I think should be the solution here and hopefully i'll get some feedback about it.   Scenario: A page needs to display a list of "parts" for a car. A database table already exists with these parts. The list of parts on the page need to be ordered by name on first load, but then can be re-ordered by users using a drop down list. They can also be filtered, and searched.   A page also exists to display a single car part.    What I think should be, and what i'm struggling with:   Model: I will have a "part" object which represents an individual car part. The part object will use a database abstraction layer. On "new Part()" will generate an empty object. ->Load( id ) will load an individual part.   Controller:   I do not know how I would implement this. I know it would contain methods to Filter(), Search() and Order() and that would directly access the Model.   View:   I am lost here too, I need to display a list of car parts, and on another page, a single car part. I understand I should use the same Model for both. However I do not see how I would list the parts.      Some Questions:   Should I have another "model" that is a list of the "Part" model called PaetListModel, or should "Part" be able to generate that list? I clearly should have 2 views, one for singular part, one for list of parts. Should the view access the model to generate the list before using the data for output? Should the controller be used in the view instead of the model to generate the initial list (or singular) on page load? Should the filter functions in the controller reload the "PartsList" from wherever our list is stored?    I think the most important question for me though is:    How would YOU implement the above green scenario?    I would like to learn from peoples examples so I get an idea of what road to follow

Im going to use a large array of arrays, each of one having a lot of values and some sub arrays. My question is...  is faster to use arrays or is better to have a object to acces using methods and all? i suppose objects are slower...

Also i was planing in use arrays with string keys in nearly all places, normally these are slower, but in php hashes and arrays are the same tipe  so i dont know...