Tools

[Delivos]

I wanna write some tools. Way back when, I wrote the slightly quirky and definitely faulty http://tools.summitmedia.co.uk/spider/ and that's certainly up for a revamp. Any other ideas? Domain auction scraper and evaluator? Anything else?

[matt1982]

How about a tool that takes over the world at the click of a button?

Failing that something that helps automatically track your serp rankings could be good? I was considering doing something like this a while back. Though i suspect it may not be possible now that google has stopped certain ways you can use its service. Let me know your thoughts I would gladly help with this one.

[netmunky]

google doesn't give out soap keys anymore, so the only way to get this data from them is to break TOS and scrape it. msn took link: data out of it's api and search a long time ago. yahoo is the only one you can get this full data on anymore.

google will also block you as 'virus infected' or similar if you make too many automated requests. i don't recall what the actual number was, but if you try to scrape data on too many sites from 1 host, you'll be blocked for 24hrs.

[DaveJ|UK]

Originally Posted by netmunky

google doesn't give out soap keys anymore, so the only way to get this data from them is to break TOS and scrape it. msn took link: data out of it's api and search a long time ago. yahoo is the only one you can get this full data on anymore.

google will also block you as 'virus infected' or similar if you make too many automated requests. i don't recall what the actual number was, but if you try to scrape data on too many sites from 1 host, you'll be blocked for 24hrs.

Probably not worth the effort but I'm sure you could use an ActiveX control to do the scraping from a users IP and then upload it to the server.