RuneTrack Forums
It is currently Wed Oct 17, 2018 12:40 pm

All times are UTC [ DST ]




Reply to topic  [ 4 posts ] 
Author Message
 Post subject: A little help...
PostPosted: Thu Nov 25, 2010 5:22 am 
Offline

Joined: Mon Nov 22, 2010 4:55 am
Posts: 7
I've been trying, and very unsuccessfully, at creating a new Grand Exchange search.

The search would generate results based on the name, and then the current market price.

The get file contents takes >15 seconds to get data from the Grand Exchange site... and I don't particularly know how to get the contents between the market <span>'s...

I was wondering if you could help me out a bit here, Sword Kill11


Top
 Profile  
 
 Post subject: Re: A little help...
PostPosted: Thu Nov 25, 2010 9:06 am 
Offline
Site Admin

Joined: Sat Jun 06, 2009 12:38 am
Posts: 546
Hi Kamron3,

For getting page data, look into cURL. file_get_contents() is far too slow for a page of that size. The best thing about cURL is that it allows you to make parallel data fetches, so you should be able to get the contents of 50 grand exchanges in under a minute - http://php.net/manual/en/function.curl-multi-exec.php . Getting a single page with cURL using curl_exec() (http://www.php.net/manual/en/function.curl-exec.php) is also probably faster than file_get_contents().

cURL however does have some odd tendencies you might run into - it's not always successful, so you sometimes have to validate the data it fetches to make sure it worked (and trigger it to loop back through if it didn't), and if you use curl_multi_exec(), there's a limit to how many parallel fetches you can make at once - usually 50. However, it's really the best option out there at the moment for fetching pages. RuneTrack's system update actually uses cURL, and captures roughly 7,000 (one for each user) stat pages each night in a matter of minutes.

Unfortunately though, unless Jagex has released something I'm not aware of, they don't have a raw page for Grand Exchange data like they do for highscores data (http://hiscore.runescape.com/index_lite ... ord_kill11), which will drastically complicate the process.

The idea way to get the exact numbers you want out of the data would be preg_match (http://php.net/manual/en/function.preg-match.php), though if that doesn't work out, just use strpos (http://us3.php.net/manual/en/function.strpos.php) to get the position of some unique tag/identifier directly before and after the number you want to extract. Then, using those two position values, substr() the specific number you want.

You may also want to take a look at http://www.grandexchangecentral.com/ - it's essentially a tracker site for the Grand Exchange.

Hope that helps. :)

_________________
ImageImage


Top
 Profile  
 
 Post subject: Re: A little help...
PostPosted: Fri Nov 26, 2010 1:18 am 
Offline

Joined: Mon Nov 22, 2010 4:55 am
Posts: 7
Thanks, that helped out a bit.

The only problem with this is, I'm trying to make it where I can get this into a nice little Chrome Addon.

It would be *EXTREMELY* helpful (and rather profitable) for me to have a tool that I don't even have to leave the page to check the grand exchange prices.


Top
 Profile  
 
 Post subject: Re: A little help...
PostPosted: Fri Nov 26, 2010 3:16 am 
Offline
Site Admin

Joined: Sat Jun 06, 2009 12:38 am
Posts: 546
I'm not really familiar with Chrome Addons, so I'm not really sure what you're trying to accomplish. Would you be making a PHP page, then porting it directly into a Chrome Addon (kind of like an iframe), or making the Chrome Addon in a different programming language which just calls some other PHP pages to handle the GE data fetching functions?

If the former and you are looking to dynamically send/receive data in PHP without reloading the page, then AJAX would be your answer. With AJAX, you could make an entire program from that dynamically fetches the contents/runs other scripts all from a single PHP file which wouldn't need to be reloaded.

Basically, you'd be doing what I stated in my above post, just using AJAX to do it behind the scenes so you wouldn't have to use multiple pages.

_________________
ImageImage


Top
 Profile  
 
Display posts from previous:  Sort by  
Reply to topic  [ 4 posts ] 

All times are UTC [ DST ]


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group