|
PHP - Scraping Lottery Results Automation
Hi, I have lottery blog and i want to automate lottery results to post on my website instead of manually enter into website. Is there anyway to do it. I am not technical expert. Please guide me step by step so i can do it easily. Own website: https://uk49sresult.co.uk/ Scraping website: https://49s.co.uk//49s/ Edited May 26 by Johnwilliam Similar TutorialsHello Everyone.. I have a different sort of a question here.. I was looking if its possible to check the no. of files in a folder (wheather they be in any format) and if thts possible can i manupulate those files like renaming them.. like for instance i have 1000 files in a folder nd i wanna rename them all according to sum criteria.. like if there are 300 mp3 songs in a folder.. 1st i wanna do i check how many are in the folder exactly.. then i wanna rename each song cuz they have sum extra information of artist on the file name.. like hello darling (alize).mp3 i wanna rename them to hello darling.mp3.. nd i wanna do the same to all the files in that folder.. is der a way i can make this possible..? 1st check the total no. of files in a folder.. renaming each file according using a special symbol.. like if "(" symbol cums up it can explode the name.. nd we can have the correct name in title[0] nd the (alize).mp3 part in title[1]. we can just join title[0].mp3 to get the correct name out of each file..!! i hope u understand wat i need to doo!! wud be thankfull if anyone can help ok i have a lottery system very nice one choose 36 balls/etc 3 numbers must be exaxt, odds of winning is like 1:45920 lol prob is for the lottery to countdown and select the lottery, somone needs to be at the site to refresh the "DELETE FROM" query/etc/etc how would I go about making it so after the time is run down, what if no1 is viewing the lottery page? then the new lottery wont run, i need something refresh that page every other 1hour, cron job? is there any available for me to enhanced on /etc? How would I go about starting 1 i want 1 with 32 numbers and i need to have to run a query every 1hour how would i go about this? I plan on learning PHP for web scraping, automating repetitive tasks and other related stuff. I have no programming background or knowledge at all aside from HTML and CSS (which are not really programming languages). I would like to know if I need to learn everything there is about PHP? Or I can just learn the basics and then just learn specific parts or topics of the language that are related to such tasks (scraping and automation)? Any suggestions are welcomed. Thanks in advance. Struggling with creating the following: I am creating a program to simulate the Powerball Lottery game, I have everything working but i forgot to add the Quick Pick Option, having alittle bit of an issue with my "if" statement I believe. My confusion is whether it should be a nested "if", or should it be it's own entity? Here's what i have so far: Code: [Select] <?php $intGrandPize = 10000000; $intRegCount = 5; $intMaxReg = 59; $intMaxPB = 39; //$quick = rand(1, 59); //$quickPB = rand(1, 49); if (isset($_POST['Quick Pick'])){ if (count($_POST)>0) { $aryPick = array(); $intPower = 0; for($t=1;$t<=$intRegCount;$t++) { if (isset($_POST['num'.$t])) { $intPick = (int)$_POST['num'.$t]; if ($intPick > 0 && $intPick <= $intMaxReg && !in_array($intPick,$aryPick)) { $aryPick[] = $intPick; } } } if (isset($_POST['Power']) && (int)$_POST['Power'] > 0 && $_POST['Power'] <= $intMaxPB) { $intPower = (int)$_POST['Power']; } if (count($aryPick) < 5 || $intPower == 0) { echo "<font color='red'>'*Five unique numbers and a PowerBall selection are Required*'</font>";; } else { // Have valid numbers... sort($aryPick); // For if you are going to display them, they will be in order... // Pick your winners $aryAllBalls = range(1,$intMaxReg); // array of numbers 1 - 59 shuffle($aryAllBalls); // Randomize their order $aryAllBalls = array_chunk($aryAllBalls,$intRegCount); // The above breaks up into an array with 5 numbers each // [0] contains the first 5 balls of all balls randomized, these are what are "picked" $aryPickedBalls = $aryAllBalls[0]; sort($aryPickedBalls); // Sort them for display $intPowerBall = rand(1,$intMaxPB); echo "YOUR WINNING NUMBERS A <br>",implode(' ',$aryPickedBalls),' PB: ',$intPowerBall,"<br>\n<br>\n"; echo "You Picked: "; foreach($aryPick as $key=>$val) { if (in_array($val,$aryPickedBalls)) { echo '<strong>',$val,'</strong> '; } else { echo $val,' '; unset($aryPick[$key]); // Remove it since it didn't match } } $bMatchPB = ($intPower == $intPowerBall); // Set here since checked in 3 places... if ($bMatchPB) { echo 'PB: <strong>',$intPower,"</strong><br>\n<br>\n"; } else { echo 'PB: '.$intPower,"<br>\n<br>\n"; } // At this point, $aryPick will only contain matching numbers... $intMatches = count($aryPick); echo 'You matched '.$intMatches,' numbers and did '; if (!$bMatchPB) { echo 'not '; } echo "match the PowerBall number.<br><br>\n\n"; // HERE YOU WOULD DO SOMETHING TO SAY HOW MUCH THEY WON if ($intMatches>=3 || $bMatchPB) { $intWinnings = 0; switch ($intMatches) { case 0: if ($bMatchPB) { $intWinnings = 3; } break; case 1: if ($bMatchPB) { $intWinnings = 4; } break; case 2: if ($bMatchPB) { $intWinnings = 7; } break; case 3: if ($bMatchPB) { $intWinnings = 100; } else { $intWinnings = 7; } break; case 4: if ($bMatchPB) { $intWinnings = 10000; } else { $intWinnings = 100; } break; case 5: if ($bMatchPB) { $intWinnings = $intGrandPize; } else { $intWinnings = 200000; } break; default: echo "ERROR: Winning Combination not defined in systen!!!<br>\n"; } echo "<strong>YOU ARE A WINNER!!!!</strong><br>\n"; echo 'You Won: $'.number_format($intWinnings,0),"<br>\n"; } else { echo "Sorry, you didn't win. Try again!<br>\n"; } } // END: if (had valid numbers picked) } // END: There was data posted (ie. Form submitted) ?> Hello,
Currently my webscraper signs into the site and pulls all the html -> perfect.
What I need to do is to loop only specific information (horses that ran)
here is my current php code
<?
$url = 'site';
$postdata = array('username' => "username",
'password' => "password");
$ch = curl_init();
if($ch){
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt'); // set cookie file to given file
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt'); // set same file as cookie jar
$content = curl_exec($ch);
$headers = curl_getinfo($ch);
curl_close($ch);
// Debug option
// print_r($headers);
if($headers['http_code'] == 200){
echo $content;
}
}
?>
here is the html im pulling
<table width=100% border=1><tr><td class=instruction6 colspan=4><b>My Race Notes</b></td></tr> <tr><td width=90%><form action='races.php?id=7456132' method=post> <textarea name='comments' rows=2 cols=38>Type notes & press Add</textarea></td> <td width=5%><input type=submit class='weestatbutton' value='Add'></form></td></tr></table></td></tr></table><table width=100%><tr class=databreakdown2253><th><a href='races.php?id=7456132&sortby=1'>Place</a></th><th>Dist Bt</th><th>Stall</th> <th>Horse</th><th>Age</th><th><a href='races.php?id=7456132&sortby=3'>Weight</a></th><th>Headgear</th><th>OR</th><th>Trainer</th> <th><a href='races.php?id=7456132&sortby=2'>Odds</a></th><th>Jockey (Claim)</th></tr><tr><td class=databreakdown2253>1st</td><td class=databreakdown2253></td><td class=databreakdown2253>4</td> <td class=databreakdown2253><a href='horses.php?id=298745'>Telegraph (IRE)</a></td> <td class=databreakdown2253>3</td><td class=databreakdown2253>9-3</td><td class=databreakdown2253></td> <td class=databreakdown2253>57</td> <td class=databreakdown2253><a href='trainers.php?id=2448'>Evans, P D</a></td> <td class=databreakdown2253>28/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=694'>Egan, John</a> </td></tr><tr class=databreakdown18><td colspan=12>soon led, brought field stands side from 3f out, headed 2f out, rallied inside final furlong, bumped and led again towards finish</td></tr><tr><td class=databreakdown2253>2nd</td><td class=databreakdown2253>0.5</td><td class=databreakdown2253>3</td> <td class=databreakdown2253><a href='horses.php?id=305855'>Ecliptic Sunrise</a></td> <td class=databreakdown2253>3</td><td class=databreakdown2253>8-12td><td class=databreakdown2253></td> <td class=databreakdown2253>52</td> <td class=databreakdown2253><a href='trainers.php?id=4516'>Donovan, D</a></td> <td class=databreakdown2253>10/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=3414'>Cosgrave, Pat</a> </td></tr><tr class=databreakdown18><td colspan=12>chased leaders, challenged 2f out, led 2f out, edged right inside final furlong, rider lost whip and headed towards finish</td></tr><tr><td class=databreakdown2253>3rd</td><td class=databreakdown2253>1.5</td><td class=databreakdown2253>1</td> <td class=databreakdown2253><a href='horses.php?id=300316'>Bookmaker</a></td> <td class=databreakdown2253>4</td><td class=databreakdown2253>9-6</td><td class=databreakdown2253><a title='Blinkers worn'>Blnk</a></td> <td class=databreakdown2253>59</td> <td class=databreakdown2253><a href='trainers.php?id=933'>Bridger, J J</a></td> <td class=databreakdown2253>6/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=3848'>Carson, William</a> </td></tr><tr class=databreakdown18><td colspan=12>prominent, took keen hold, led 2f out, headed over 1f out, not much room inside final furlong, stayed on same pace</td></tr><tr><td class=databreakdown2253>4th</td><td class=databreakdown2253>1</td><td class=databreakdown2253>2</td> <td class=databreakdown2253><a href='horses.php?id=261986'>Night Trade (IRE)</a></td> <td class=databreakdown2253>7</td><td class=databreakdown2253>8-8</td><td class=databreakdown2253><a title='Cheekpieces worn'>CkPc</a></td> <td class=databreakdown2253>50</td> <td class=databreakdown2253><a href='trainers.php?id=2653'>Harris, R A</a></td> <td class=databreakdown2253>6/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=7348'>Hardie, Cameron</a> (3)</td></tr><tr class=databreakdown18><td colspan=12>prominent, ridden over 2f out, switched left inside final furlong, no extra close home</td></tr><tr><td class=databreakdown2253>5th</td><td class=databreakdown2253>1.5</td><td class=databreakdown2253>6</td> <td class=databreakdown2253><a href='horses.php?id=299296'>Trigger Park (IRE)</a></td> <td class=databreakdown2253>3</td><td class=databreakdown2253>8-10</td><td class=databreakdown2253></td> <td class=databreakdown2253>50</td> <td class=databreakdown2253><a href='trainers.php?id=2653'>Harris, R A</a></td> <td class=databreakdown2253>20/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=3422'>Dobbs, Pat</a> </td></tr><tr class=databreakdown18><td colspan=12>chased leaders, ridden over 2f out, one pace over 1f out, no impression</td></tr><tr><td class=databreakdown2253>6th</td><td class=databreakdown2253>2.25</td><td class=databreakdown2253>7</td> <td class=databreakdown2253><a href='horses.php?id=300337'>Port Lairge</a></td> <td class=databreakdown2253>4</td><td class=databreakdown2253>8-11</td><td class=databreakdown2253><a title='Blinkers worn'>Blnk</a></td> <td class=databreakdown2253>50</td> <td class=databreakdown2253><a href='trainers.php?id=914'>Gallagher, J</a></td> <td class=databreakdown2253>33/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=193'>Catlin, Chris</a> </td></tr><tr class=databreakdown18><td colspan=12>slowly into stride, in rear, stayed on inside final furlong, never dangerous</td></tr><tr><td class=databreakdown2253>7th</td><td class=databreakdown2253>NK</td><td class=databreakdown2253>11</td> <td class=databreakdown2253><a href='horses.php?id=289934'>Lionheart</a></td> <td class=databreakdown2253>4</td><td class=databreakdown2253>8-13</td><td class=databreakdown2253></td> <td class=databreakdown2253>59</td> <td class=databreakdown2253><a href='trainers.php?id=4910'>Crate, Peter</a></td> <td class=databreakdown2253>10/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=7375'>Crouch, Hector</a> (7)</td></tr><tr class=databreakdown18><td colspan=12>reared start and slowly away, held up in rear, headway over 1f out, weakened inside final furlong</td></tr><tr><td class=databreakdown2253>8th</td><td class=databreakdown2253>2.75</td><td class=databreakdown2253>14</td> <td class=databreakdown2253><a href='horses.php?id=289421'>Koharu</a></td> <td class=databreakdown2253>4</td><td class=databreakdown2253>9-4</td><td class=databreakdown2253><a title='Cheekpieces worn'>CkPc</a></td> <td class=databreakdown2253>60</td> <td class=databreakdown2253><a href='trainers.php?id=2495'>Makin, P J</a></td> <td class=databreakdown2253>9/4 (Fav) </td> <td class=databreakdown2253><a href='jockeys.php?id=5952'>Bates, Mr D J</a> (3)</td></tr><tr class=databreakdown18><td colspan=12>in rear, ridden over 3f out, no impression</td></tr><tr><td class=databreakdown2253>9th</td><td class=databreakdown2253>3</td><td class=databreakdown2253>5</td> <td class=databreakdown2253><a href='horses.php?id=269827'>Saskias Dream</a></td> <td class=databreakdown2253>6</td><td class=databreakdown2253>9-6</td><td class=databreakdown2253><a title='Visor worn'>Vsor</a></td> <td class=databreakdown2253>59</td> <td class=databreakdown2253><a href='trainers.php?id=2002'>Chapple-Hyam, Jane</a></td> <td class=databreakdown2253>4/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=3544'>Hughes, Richard</a> </td></tr><tr class=databreakdown18><td colspan=12>mid-division, headway and switched left over 1f out, edged left entering final furlong, soon eased</td></tr><tr><td class=databreakdown2253>10th</td><td class=databreakdown2253>1.75</td><td class=databreakdown2253>12</td> <td class=databreakdown2253><a href='horses.php?id=304248'>Crafty Business (IRE)</a></td> <td class=databreakdown2253>3</td><td class=databreakdown2253>9-2</td><td class=databreakdown2253><a title='Visor worn'>Vsor</a></td> <td class=databreakdown2253>59</td> <td class=databreakdown2253><a href='trainers.php?id=695'>Moore, G L</a></td> <td class=databreakdown2253>14/1 </td> <td class=databreakdown2253><a href='jockeys.php?id=6669'>Bishop, Mr C</a> (3)</td></tr><tr class=databreakdown18><td colspan=12>towards rear, pushed along over 3f out, well beaten 2f out</td></tr></table><br><hr></td></tr></table>*note I'm using this for personal reasons Okay so I am scraping websites for their descriptions keywords and titles. I noticed that a lot of websites use the same keywords and descriptions on every page.. so my idea is to scrape the index and find all the links in there and scrape them all then after they been scraped check all of the descriptions and if the descriptions match then pull some text unique to each page and use that. I can't seem to wrap my head around it.. how would I accomplish this? I scrape with curl then find keywords description and title then find all links on the site and scrape those. soo I was thinking making an array of the descriptions and then checking and inserting to the db but doesn't seem like it would work. Any ideas? Oh also.. how would I grab just text from each page that is different from every other page? lol very confusing Ok, I know how to screen scrape, but I don't know how to screen scrape when there is a login. I've looked this up for awhile, but no luck. I'd like to also make it so I can execute a url when I am logged in on the script for the script, for an example execute this url: http://site.com/data.php?id=9912&submit=1 Thanks in advanced. Hi, Im trying to work out a way to get the New York Lottery's Take 5 results. Theres a few sites that list the winning numbers, i assume automatically as there is alot of lottery games on these sites. what would be the best way to get this? http://www.myfreepost.com/lottery/index.php/us/newyorklottery/takefive/result/ http://www.elite-lottery-results.com/?action=view_game&gid=NY2 I am a newbie and I am trying to do a site scraping project to obtain all the following fields: Test Year, Test Name, Grade Level, Question #, Question Type, Reporting Category, Standard #, Standard description, Example Question (with image) for this web page that has a page for each question. http://www.doe.mass.edu/mcas/search/question.aspx?mcasyear=2010&QuestionSetID=1&grade=8&subjectcode=MTH&questionnumber=36 I am a newbie at PHP and would love if you could point me in the right direction. The page uses tables and I need to extract the data from the body of the page as well as some of the info from the url and then have it inserted into a MySQL database. Thank you so much for your help. I need to scrape pages - I only need one page at a time I'm only looking for 2/3 bits of data within each page Can someone give me some pointers where to start? I've searched and see names like DOMXpath and Xpath mentioned - do I need these? It's important that I can run the script on a standard Linux hosting with nothing extra installed like packages - I'd like to have something I can just use immediately using standard php and functions I've seen plenty of tutorials + youtube videos - just looking for recommendations and pointers for recommended practices Thanks OM i need some help to scrape a link from specified page. for example if i have a page like this http://br.4ce.info/ i want to scrape all link on that page and i want to show all link in that page on my wordpress widget in another blog ? can you help me with this ? dont use iframe i think better using cURL thanks More information on the job posting. I am looking to fetch information from daily deal website, Such as tuango.ca, socialliving.com, groupon.com...ect I want to retrieve data from different daily deal sites, and I want to retrieve all the deals of the day from each different city in the website. For example www.tuango.ca Has a deal a day in Montreal, Toronto,...ect I want to be apply to retrieve data from all the different location within the site. I want the script to fetch the data of deals. To be more clear I want the script to fetch What site the deal was on What location was it for What's the tittle of the deal What price is the deal What's the value of the deal What's the saving in percentage of the deal How much were sold What's the minimum amount of the deal before it becomes activated What's the company who did the deal Company address Company postal code Company phone number (there might be more categories..will talk more if you pass this stage of the interview process) Ones all this data is fetched I need it to automatically be store in a database. Every morning at 4:am (eastern time) I need it to run the script, because the days deals finish at midnight and it's the only way of getting a number of the total number of coupons sold. you'll usually see the final stats of the deal on their recent deals page of the website. I want to know how a site like http://onespout.com/deals/montreal did it.. I'm not asking somebody to do it for me I'm just asking someone to guide me in takeing the right steps I have a form which lets the user put in the URL to their twitter account. When the enter their URL I am trying to create a screen scraping script that scrapes that page to get basic information like their twitter name and number of tweets. I'm not sure how I am going to do this, I don't think there is a twitter API for this so I may have to use something like cURL. I was just wondering if anyone has done this and could give me any advice about the best method? Thanks for any help I'm trying to pull the stock quotes Beta from yahoo finance since the yahoo query language doesn't support it. My code returns an empty array. Any ideas why? Code: [Select] <?php $content = file_get_contents('http://finance.yahoo.com/q?s=NFLX'); preg_match('#<tr><th width="48%" scope="row">Beta:</th><td class="yfnc_tabledata1">(.*)</td></tr>#', $content, $match); print_array($match); ?> Hello, I have checked out many of the scripts and tried implementing them to help me scrape 1 single image from a url. Example www.123.com/333.png Getting a script to scrape that image isnt the problem. Im not sure on how to implement the simple curl to save the image every 30mins and name it in successive order so it appears as , 1.jpg, 2.jpg, 3.jpg I am working with a debian 6 server and php would be the easiest way to do this that i can work with. I have searched the web endlessly and still cant produce such thing. Any help is appreciated. I'm looking to scrape the schedule details for any particular class at my university as part of a school project. I have been able to log a student into the university site, grab their name and course information. In order to grab the schedule for a particular class I now have to visit a different area of the university site, the registrar. The course schedule section of the registrar is coded in ASP .net and I'm having trouble making HTTP requests to this area of the site. I understand the need to make post requests to mimic the Viewstate but I'm running into an issue before I even get to that part. I am able to load the page via an HTTP request almost every time. But it always takes almost exactly 2 minutes. I have tried simple get requests, post requests with the Viewstate, and other variations to one of a few different pages on the site. Each time it works. But each time it takes 2 minutes. Any ideas why it takes so long? Any suggestions on what I can possibly do differently? Here is the basic site I'm using to test my code on before implementing it fully into my program: University Site Here is my link that takes 2 minutes to load the same page: My Site Here is my latest code I've tried: Code: [Select] <?php $postdata = "__VIEWSTATE=/wEPDwULLTIwNjY2MzUzMDEPZBYCAgUPDxYCHgRUZXh0BRNNYXIgMjMgMjAxMSAgNzoxNVBNZGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3N0QmFja0tleV9fFgEFDmN0bDEwJGltZ0xvZ2luaYy4H4gz+Bjb4GVdsO1ecd9c9EA="; $postdata .= "&__EVENTVALIDATION=/wEWAgKs/IaWBAKpyP2zAXWcNEO0tMqDX53r6m+Hzo/nKHwZ"; $postdata = urlencode($postdata); $host = 'courseschedules.njit.edu'; $path = '/index.aspx'; $fp1 = fsockopen($host,80,$errno,$errstr,30); if(!$fp1) die($_err.$errstr.$errno); else { fputs($fp1, "POST $path HTTP/1.1\r\n"); fputs($fp1, "Host: $host\r\n"); fputs($fp1, "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.15) Gecko/20110303 Firefox/3.6.15 ( .NET CLR 3.5.30729)\r\n"); fputs($fp1, "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"); fputs($fp1, "Accept-Language: en-us,en;q=0.5\r\n"); fputs($fp1, "Accept-Encoding: gzip,deflate\r\n"); fputs($fp1, "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n"); fputs($fp1, "Keep-Alive: 115\r\n"); fputs($fp1, "Connection: keep-alive\r\n"); fputs($fp1, "Content-length: ".strlen($postdata)."\r\n\r\n"); fputs($fp1, $postdata."\r\n\r\n"); $response = ''; while(!feof($fp1)) $response .= fgets($fp1,2000); fclose($fp1); echo $response; } ?> Like I said, I've also tried a standard get request which works as well, just takes 2 minutes. Could anyone point me in the right direction for downloading app store statistics from: -the Apple App Store -The Android Market -the Amazon AppStore Specifically, I'd like to get the -average selling price of apps -the top selling apps -the distribution of tablets vs. phones, etc. (e.g. how many apps are there for Honeycomb? How many for iPad?) -total number of apps in store -free vs. paid apps I've seen some sites like http://148apps.biz/app-store-metrics/ and http://www.appbrain.com/stats/ How do these sites get their data? There must be a way to export the whole app store database as a CSV file, or import it to MySQL and run queries. Thanks much for any direction. |
|||||||