|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了,ebay、amazon等一些站,采集先看robots.txt,从robots.txt中找到sitemap的索引,剩下的就想办法搞吧# h6 L v- t/ U6 ]8 M- M1 R, a
http://www.ebay.com/robots.txt # sitemaps - SRPs
; }& T! R2 D' y& L( T2 f) y2 {' ?Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
. w0 U+ n; e# K( `/ p; QSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
% C" E9 L) o$ m( G k2 T* T
( P, _0 g6 B6 P" M! Q. C# Guides sitemaps
) z5 h% ^! P6 F/ _3 XSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml1 T! h7 R- r" @2 E* \5 o) b
( F( P! `6 N- ]+ @ |# SSRP sitemaps
; ~$ h# P0 @% X5 r& j1 eSitemap: http://www.ebay.com/lst/SSRP-0-index.xml
- t6 E* u" d# W" | \
" ?( S0 h- N; Y* U, v#Stores Sitemaps
. T# s7 z' T3 E. w2 dSitemap: http://www.ebay.com/lst/STORES-0-index.xml
d, b5 E* T" a7 s+ }% j. Q$ h' U0 u8 j8 N
#BHP Sitemaps
7 U+ |8 x+ ~2 |Sitemap: http://www.ebay.com/lst/BHP-0-index.xml# V% \2 `" e) K: B, K7 r6 `2 H
, ^0 N( U/ d8 K/ P3 k% w
#Collections! L; J" ^* M7 i( d! d
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
6 Y- j, t M6 z7 \! t% q. z! F i( x, j
#VI
- [/ r) H2 Q, i# ?" |& R' K6 GSitemap: http://www.ebay.com/lst/VI-0-index.xml
?; m& b9 Z l y( m; b7 l& [1 ~5 V0 E
#PRP9 f: G! [6 b5 O+ X: O
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml - K! Y! k. C! k
( M* l' o8 h& Y& V. h
|
|