|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
8 d0 I+ ~& e0 S/ _http://www.ebay.com/robots.txt # sitemaps - SRPs' a/ G, c2 ]9 w7 }+ j7 |6 c
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml7 \/ g& o5 M8 `& ?) j$ z" i
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
0 }; F& S. ~8 b8 I
# D+ h6 r* j6 O# Guides sitemaps! N' q2 ]5 Y$ W( ~6 A) n0 A! o/ z
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
# U3 F0 {; ]3 D0 q" R6 J. z& Y3 ]0 Y* S4 Y
# SSRP sitemaps q. x Q$ E9 v' a( a' a, E! B
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml2 j5 S5 ]. }" k4 d# J: i6 T8 N7 F
+ |7 \9 T, |: M' L4 _/ e
#Stores Sitemaps* [. @/ u1 a0 Z' e; X, ~! g' D
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
3 x! i9 e* K& P& R; M
2 N$ h; r# i( S- c4 Y#BHP Sitemaps
: c7 Z Y1 g3 v# r7 O4 k8 j8 QSitemap: http://www.ebay.com/lst/BHP-0-index.xml; \7 H6 v' G @$ C) q0 ^
) J0 I+ _& v0 N. O- |: Z& e#Collections2 h6 k: D6 `. h- t
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml1 P( r4 V% N+ j; P! k4 u( N, N
! {+ n4 D7 B7 i7 j7 ~#VI
9 {# o, n# T8 X5 U7 P) P" E0 V, b+ xSitemap: http://www.ebay.com/lst/VI-0-index.xml4 \1 h5 r4 d7 @3 d
( I( f8 o8 S9 z6 i3 v- T. @5 A H
#PRP
& Z5 o4 c& S7 Y6 MSitemap: http://www.ebay.com/lst/PRP-0-index.xml
4 F& X' g9 a3 s6 G% r# O
2 w% d+ g4 E( } o0 j8 f |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主