|
|
发表于 2013-5-27 12:23:02
|
显示全部楼层
OodleBot (aka. oodlebot/1.0) is the name of our crawler, and we've let it loose on the Internet to search for classified listings.3 N3 i2 \4 V3 t, P3 a
5 ?' d5 m& z- W- a9 G+ A7 B
When OodleBot finds a listing, it tries to understand what the listing is all about. Once the information is gathered, it strips all the contact info from the listing and puts the listing into our search engine index, where we then show the listing to all our users and partners with links back to the page it was found on.
, I# c _7 }$ D+ H0 _- u& W- j) R# n2 K( ?3 h1 O Z+ J+ V! i
OodleBot tries very hard to make sure it has the freshest and most accurate information available without hurting the site it's crawling from downloading too much data too frequently.% {0 W* o/ h. F) D
" {# I: M) F# I" X5 {/ m* \( ^Instead of letting OodleBot crawl your site, we'll happily accept a feed to download all your listings in just one simple file. Please see our feed guidelines and FAQ.$ b. T# i3 Z( z
5 M# @! m% F3 x* N3 S9 GWe work hard to house train our little bot, but if you find that OodleBot has been behaving badly on your site, please contact us at [email protected] and we will make sure to teach it a lesson ASAP.6 M6 O% {( a3 V" H x7 |8 z% U
1 x( l1 C9 i! f
If you want to make sure OodleBot doesn't stray into sensitive areas on your site, you can guide it with your robots.txt file using the User-Agent "oodlebot".
6 O# Y- d# D% v% w# L j; t
m( D; j6 S3 O s& vRead more: http://www.oodle.com/info/oodlebot.html#ixzz2USoC4fYw |
评分
-
查看全部评分
|
发表于 2013-5-27 12:02:26
发表于 2013-5-27 12:06:20
