|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧/ c) E0 \5 g4 V7 c
http://www.ebay.com/robots.txt # sitemaps - SRPs
) _ J/ @! ?$ l3 YSitemap: http://www.ebay.com/lst/SRP_US_index.xml
B8 n2 T/ [+ x, y/ qSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
6 r1 F4 E, H: }* O! m4 z1 w: [ z6 ~* I0 l7 y* c
# Guides sitemaps( I' e! K/ y/ v4 K8 |3 F( a
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml+ Q; H* W7 K" j0 ~) s- D
; S4 j0 I E2 d) i! |; L
# SSRP sitemaps
. Q; ^; n5 [5 n+ Z7 G, kSitemap: http://www.ebay.com/lst/SSRP-0-index.xml* F" e' i8 F+ V" p" I v( h: k
; S- |4 y& R; j$ q& v: d- H/ u
#Stores Sitemaps
* L- g6 B0 @) }/ y7 @/ L6 kSitemap: http://www.ebay.com/lst/STORES-0-index.xml
; M) E( c: ]8 ~' Q
% a; W2 r/ x4 [9 Z" k) Z#BHP Sitemaps8 C- l m$ q8 c4 S( W( {+ w
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml. L9 d. a6 G% g+ Z& S
0 ~2 l) Q. Y4 a) }" G# L2 A2 ]
#Collections
) y* r$ F% e) GSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml5 s* U3 y. e; R/ B$ C9 \% L" d2 J0 Y+ ^
( r, C% L) D+ J+ |9 U
#VI
3 _0 v% g. p& T7 _) r2 ?9 V! [% iSitemap: http://www.ebay.com/lst/VI-0-index.xml
E' \1 R$ d4 l+ j% g* m; u
4 d2 [' i8 |7 G. P& I+ d#PRP8 s" F6 P: O: ]2 H e) @" a
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
% n: ]. V O% Y
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主