|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧$ ~% q& V8 T4 [" z3 f4 V: u# D
http://www.ebay.com/robots.txt # sitemaps - SRPs6 L7 A) {% d D8 F3 u* k0 e" s
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml, ?) O" g0 ]' G( e! p) Y6 n- Y$ X
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml! W( b$ e5 c7 _/ V1 b( N
0 _3 C8 P2 S# v3 W# T3 q
# Guides sitemaps
0 {1 x+ A$ ~4 c W R, L. x( F# zSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml/ U( j5 H" V b* @
0 @' A2 t$ A f8 B( r/ w# SSRP sitemaps2 G8 t7 X9 N# k% r, R
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml1 V7 n# r Z6 O, A. K4 N0 w
6 w) M5 Y: `; d( O/ N" J) t8 H#Stores Sitemaps
2 {: `4 k* D! O$ d% p7 @Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
" Q5 {) G- J% q1 S: p
8 Z, i9 E( q" C0 @#BHP Sitemaps
- F5 \8 S) i) k" Q$ }5 USitemap: http://www.ebay.com/lst/BHP-0-index.xml# V2 w+ @' ~6 W9 r
t2 @! W1 I( M5 Q( l6 X
#Collections# Q1 }; y: ]0 U
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml* q7 n: A ?1 }* Y6 ~6 l5 e+ s
) i* t, o: G+ K! R
#VI
$ ]* k& ?& m/ u9 f, g: F! ]Sitemap: http://www.ebay.com/lst/VI-0-index.xml
$ M5 w8 ~5 q+ E
/ \3 { y# O9 c2 f#PRP
& j) f3 {& y1 x! P. jSitemap: http://www.ebay.com/lst/PRP-0-index.xml
! u$ l# ]0 n$ ~6 |9 a6 H2 Y: {3 W, x" z1 S; J4 d4 t
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主