|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧" V8 y* s" l0 K" |' B
http://www.ebay.com/robots.txt # sitemaps - SRPs
/ g1 o: m" B& NSitemap: http://www.ebay.com/lst/SRP_US_index.xml
: Y. g9 I8 R* K8 q @" d' w2 sSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml1 R! n8 G( _; S5 e; [8 ]
( F3 g+ s) O( @/ t7 ~, D% b
# Guides sitemaps3 C) D# `7 p3 X
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
: o0 c% t1 z7 ]
6 u* g$ v2 Z) Y+ h0 z# SSRP sitemaps
% c- K0 O8 k# w! ]) b* o+ ISitemap: http://www.ebay.com/lst/SSRP-0-index.xml4 w) i2 [7 x6 ^4 W, |7 `
1 G/ W* M7 p9 v! i& P: [- T, R/ S
#Stores Sitemaps" j% g7 y* x6 D, m
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
( T1 T5 j8 X% K7 k4 U' t! p, N" V+ [6 V; k/ d) c
#BHP Sitemaps
1 y( T' x- O3 s5 a9 V% X: P6 `Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
' f4 b& v# |/ j
7 m! d8 o* F( P' t/ T' b#Collections3 a* N- |" L! T1 S4 F2 j1 A
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml# k, ~) i! v" k) ]5 K
2 z3 b3 ?8 y! `+ l; \#VI2 ^3 V9 J/ P- a# R# h: E3 e
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
5 `' v, m B% i5 |$ T6 [% a5 A% z2 ?, K2 ]' N5 o
#PRP
8 o3 h& F( @4 G8 O: iSitemap: http://www.ebay.com/lst/PRP-0-index.xml
7 P' n, r7 e* O/ @; B' e; C7 a
6 j7 a! B) g/ G% l7 ? |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主