PHP - Http/request Issue

Hello,
If i am not posting this question at right location then i apologize for it.

My problem is: I need to send HTTP get request to the webservice end point which must include the following http headers:

GET / HTTP/1.1
Host: apis.live.net
Accept: application/json
Content-Type: application/json
Authorization: WRAP access_token=AuthToken (here AuthToken is the Access Token String)

Note: I am using OAuth for getting an access Token but for getting contents I need to send an HTTP get request.

Any kind of help will be appreciated.
Thanks

Hey guys, every now and then some of my scripts that consistently access headers seem to fail to open the stream. I'm running about 1-50 of these shell scripts at once, but it doesn't seem to make a difference weather I have 1 or 50 scripts open at once. I was wondering if there was a way to tell PHP: Alright, so it failed to open the http stream, I want you to try again until it works. I just want it to try again until it works, because when it doesn't load certain headers, things go very wrong. Thanks.

Here's an example of the code where the problem occurs:
$Attack = file_get_contents($MobLink."attack?user_id=".$UserID."&target_id=".$TargetID."&auth_key=".$AuthKey);
xml_parse_into_struct($iP=xml_parser_create(), $Attack, $iS, $iX);xml_parser_free($iP);
echo "$i :.:.: ".strip_tags($iS[$iX['MESSAGE'][0]]['value'])." :.:.:\n\n";

First thanks for any help that you can provide. I am somewhat new to PHP and this is the first "Web Service" I have had to create.

THE GOAL:
I need to pull XML data from another server. This company's API is setup so that you must give an IP and so you can only pull data from server to server instead of client to server. The data is pulled from the API using HTTP requests...very similar to YQL. (in essence the structured query is located in the URL).  This API also requires that my server only ping's their server every 10-15 mins, in order to cut down on server requests.

The logical thought in my head was to setup: a cron job to run a PHP script every 10 mins. The PHP scripts would then do the following:
1. Make the HTTP request
2. Open an existing file or create one (on my server)
3. Take the returned XML data from the API and write to the newly opened file.
4. Convert that XML to JSON.
5. Save the JSON
6. Cache the JSON file
7. STOP

My thought was to use curl and fopen for the first 3 steps. I found a basic script that will do this on PHP.net (as shown below). After that I am pretty much lost on how to proceed.


Code: [Select]
<?php

$ch = curl_init("http://www.example.com/");
$fp = fopen("example_homepage.txt", "w");

curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);

curl_exec($ch);
curl_close($ch);
fclose($fp);
?>

I would REALLY appreciate any help on this. Also, if you have the time to please comment and explain what you are doing in any code examples and why....that would HELP out a lot. I truly want to learn not just grab a snippet and run. So, your comments are vital for me.

thank!!!

Hey! I am trying to get a database table of users but am running into the error:

Warning: file_get_contents(http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&amp;user=68583) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 500 in get_user_from_parameter(nabble:utilities.naml:890) - <n.get_user_from_parameter.as_user_page.do/> - public v in C:\xampp\htdocs\website4js\stanford\loadUsernames.php on line 32

I have looked it up and it may be a security thing...The urls I am getting are http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=68583 but the error adds an nodes&amp; part that screws it up. Any way around this?

Code: [Select]
<?PHP
$maxPage = 0;
$mainPage = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136";
$mainContent = file_get_contents($mainPage);
$pattern = "/(?<=\"Page )\d\d+/";
preg_match_all($pattern, $mainContent, $pageNumb);

//find the max page
for($i=0;$i<sizeof($pageNumb[0]);$i++) {
if($pageNumb[0][$i] > $maxPage) {
$maxPage = $pageNumb[0][$i];
}
}
//echo('Max page is: '.$maxPage.'\n');

//Get an array of all the pages
$pages = array();
 for($i=1;$i<$maxPage;$i++) {
$pages[$i] = "http://protege-ontology-editor-knowledge-acquisition-system.136.n4.nabble.com/template/NamlServlet.jtp?macro=app_people&node=136&i=".($i*20);
 }
//print_r($userPages);

//Get personal page urls and add to MySQL
mysql_connect("localhost" , "root") or die("Cant connect");
mysql_select_db("protegeusers") or die("Cant find database");
foreach($pages as $url) {
$urlContents = file_get_contents($url);
$pattern = "/http:\/\/protege-ontology-editor-knowledge-acquisition-system\.136\.n4\.nabble\.com\/template\/NamlServlet\.jtp\?macro=.+;user=\d+/";
preg_match_all($pattern, $urlContents, $personalPage);
foreach($personalPage as $user) {
for($i=0; $i<sizeof($user);$i++) {
[color=green]$userContents = file_get_contents($user[$i]);[/color]
$pattern1 = "/user\/SendEmail\.jtp\?type=user.+;user=\d+/";
$pattern2 = "/(?<=\">Send Email to ).+(?=<)/";
preg_match_all($pattern1, $userContents, $userEmail);
preg_match_all($pattern2, $userContents, $username);
[color=green]print_r($username);
print_r($email);[/color]
//$query = "INSERT INTO users (username, userurl) values ('$userName','$userUrl')";
//mysql_query($query);
}
}
}
?>

I am trying to figure how to code around this. I have a DOM scraper function that pulls urls from my database, opens the page, scrapes the data, and then moves onto the next URL to scrape. my issue is that if the page fails to load the script bombs and I have to restart it again. Trying to figure out how if the Code: [Select]
$html->find('div[class="itemHeader address"]') as $div fails to open the page it just skips the DOM inspection.

Here is my error...

Failed to open stream: HTTP Request failed.

Here is where I am at with my script....

Code: [Select]
mysql_select_db("scraped") or die(mysql_error());
$result = mysql_query("SELECT PKEY, URL, HASSCRAPED, SHOULDSCRAPE FROM CRAWLED WHERE SHOULDSCRAPE ='1' AND HASSCRAPED = '0'") or die(mysql_error()); 

while($row = mysql_fetch_array( $result )) {
mysql_query("UPDATE CRAWLED SET CRAWLED.HASSCRAPED =  '1' WHERE CRAWLED.URL = '" . $row['URL'] . "'");
$html = file_get_html($row['URL']);
foreach($html->find('div[class="itemHeader address"]') as $div) {
foreach($div->find('a') as $element){
$CleanData = CleanData($element->innertext);
if (strlen($CleanData[0]) > 5){
    mysql_query("INSERT INTO SCRAPED (ADDR1, CITY, STATE, ZIP, URL, DATE) VALUES ('" . $CleanData[0] . "','" . $CleanData[1] . "','" . $CleanData[2] . "','" . $CleanData[3] . "','". $row['URL'] . "','". date( 'Y-m-d H:i:s ' ) ."')");
        }
}
}

$html->clear();
unset($html);
unset($CleanData);
}

function CleanData($data) {
  $NewData = trim($data);
  $NewData = str_replace("<em>", "", $NewData);
  $AddrCityStateZip = explode("</em>",$NewData);
  $CityStateZip = explode(",",$AddrCityStateZip[1]);
  $StateZip = explode(' ',$CityStateZip[1]);
  $NewDataArray = array ($AddrCityStateZip[0], $CityStateZip[0], $StateZip[1], $StateZip[2]);
  return $NewDataArray;
  unset($NewData);
  unset($AddrCityStateZip);
  unset($CityStateZip);
  unset($StateZip);
}
mysql_close($link);
echo 'Scraping has compleated without error';

Hi guys.. i am not very sure where to post about this issue but im giving a try and hope to get some ideeas where this might come maybe.

So i have a small website with a members database.. the website itself it has SSL and works properly on a https connection. The problem comes when a member tries to access via http. Unfortunetaly on a page i have to keep it http due to some ads that i display there and dont work on https.

If a member logs in and surfs the website via https everything is working perfectly.. but when he changes and tries to access via http he automatically gets logged out.. if he tries to login via http he also cant and gets an error with an invalid token.

So basically i think my database cant connect when accessed by http..

Any of you ever encountered this issue ? Or maybe have some ideeas where this issue is coming from ?

 

Thanks and any ideea or help very appreciated

Hi:

I am trying to pass/request a Variable/QueryString from one page to another, but it isn't working.

Seems like it would be simple, so maybe I am overlooking the obvious.

I have the 1st page:
Code: [Select]
<a href="NewCity.php?abbr_state=<?php echo $abbr_state ?>">Insert New City</a>

Writes the URL properly - CityList.php?abbr_state=CT

The 2nd page:
Code: [Select]
<?php

$abbr_state = $_REQUEST['abbr_state'];

?>


<form>

State: <input type="text" name="abbr_state" value="<?php echo $abbr_state; ?>" readonly=""/><br />

</form>

But it does not get the variable/QueryString for "CT" and add it to the form field (it's not the "readonly" causing it).

Why?

What did I miss?

Doing something wrong, but don't see it.  How should one retrieve a POST parameter?  My $request->toArray()['html'] works, but I am sure it is not the "right way".

<?php
namespace App\DataPersister;
use ApiPlatform\Core\DataPersister\DataPersisterInterface;
use Symfony\Component\HttpFoundation\RequestStack;

class ArchivePersister implements DataPersisterInterface
{
    public function __construct(RequestStack $requestStack)
    {
        $request = $requestStack->getCurrentRequest();
        syslog(LOG_ERR, '$request->getMethod(): '.$request->getMethod());
        syslog(LOG_ERR, '$request->getContent(): '.$request->getContent());
        syslog(LOG_ERR, '$request->request->get(html): '.$request->request->get('html'));
        syslog(LOG_ERR, '$request->query->get(html): '.$request->query->get('html'));
        syslog(LOG_ERR, '$request->get(html): '.$request->get('html'));
        syslog(LOG_ERR, '$request->toArray(): '.json_encode($request->toArray()));
        syslog(LOG_ERR, '$request->toArray()[html]: '.$request->toArray()['html']);
    }
}

output

$request->getMethod(): POST
$request->getContent(): {"project":"/projects/1","description":"","html":"<p>{{ project_name }}</p>"}
$request->request->get(html):
$request->query->get(html):
$request->get(html):
$request->toArray(): {"project":"\/projects\/1","description":"","html":"<p>{{ project_name }}<\/p>"}
$request->toArray()[html]: <p>{{ project_name }}</p>

Trying to run a simple program that, when submitted, stores the username and password as cookies.  When clicking Submit, I get the error  "HTTP Error 405 - The HTTP verb used to access this page is not allowed". If the username and password fields are left blank when submitting it's suppose to give a message to enter a username and password, but, I still get that error message.

HTML form:

Code: [Select]

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Week 1 Project--Cookies</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />


</head>

<body>
<form action="cookie1.php" method="post">
<h2 align="center">Cookies</h2>

<br />
<div>
<p>Enter your username and password and click "Submit":</p><br />
<p>Username:<input type="text" name="username" size="20"></p>
<p>Password:<input type="text" name="password" size="20"></p>
</div>
<br />
<div><input type="submit" name="submit" value="Submit" /></div>
<br />
<div>
<input type="reset" name="Reset" value="Start Over" />
</div>

</form>



</body>
</html>


PHP file:

Code: [Select]

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Cookie File</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
</head>

<body>
<div>
<?php
if ($_SERVER['REQUEST_METHOD'] == 'POST')
{
 setcookie('username', $_POST['username'], time() + 2592000);
 setcookie('password', $_POST['password'], time() + 2592000);
}
if(($_POST['username'] == "") || ($_POST['password'] == "")) {
print "You must enter both a username and password. Press the Back button on your browser and try again.";
}
else if (isset($_COOKIE['username'])) {
print "Welcome, " .$_COOKIE['username'];
}
?>
</div>
</body>
</html>

How do I only redirect the page when index.php is present?

Not sure if this is possible.  What I am trying to do is remove a section of text a user posts.

ie:  go to this url:  http://url.com  or http://this.domain.org

I already have it to strip they <a href="whatever

but not sure if I can do the other.

Any help would be appreciated.

Thanks

Hi.  I am trying to extract the main url from a string.   For example.   I want to extract:   "http://google.com" from "http://google.com/search/pic.jpg" "https://secure.co.uk" from "https://secure.co.uk/secure/secure.gif"   I am able to do this with this code, where $url is the url:   preg_match("http://(.*?)/", $url, $matches); preg_match("https://(.*?)/", $url, $matches);   Is there away to consolidate this into one regex search?  Or am I being picky?   Thanks!

Just a quick question.  I'm using the Live HTTP Headers add-on for Firefox that display all the HTTP headers etc.  I'm writing an article on it and want to include a screenshot of it in action.  Are there any headers that I should hide from the reader for security reasons?

Thanks for any help. 

I am learning PHP using the book "Beginning PHP and MySQL 5 - From Novice to Professional", and I have come across a lesson in the book which I am stuck on.
The lesson is on using the echo() statement. I've typed in the following code and saved it as a php file....

<?php
$heavyweight = "Lennox Lewis";
$lightweight = "Floyd Mayweather";
echo &heavyweight, " and ", $lightweight, " are great fighters.";
?>

My Apache and PHP is working fine as my other PHP files work properly.

When I try and view the file it says...
"The website encountered an error while retrieving http://localhost:8888/phpdocs/multiplestringsecho.php. It may be down for maintenance or configured incorrectly."

and when I click on more information it says...
"HTTP Error 500 (Internal Server Error): An unexpected condition was encountered while the server was attempting to fulfill the request."

Any ideas? Have I done something wrong? Any help would be much appreciated.

I am trying to build an app which will scan a site multple times, the only problem is the 403 error, how do I get around this.
Searching seems to imply curl or user_agent, but can't get it working.

Any suggestions?

Thanks