|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧# k! D6 _3 p7 `5 R
http://www.ebay.com/robots.txt # sitemaps - SRPs7 Q4 q1 C+ _1 E
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
0 d+ o* e* Y% C; B8 f3 hSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml% k& F# W7 z+ R- d0 R0 a
& o( q% C4 c% t* s z7 l# Guides sitemaps _4 ~% ^; m# Z( T8 N( ?
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
$ U9 \8 {# U# T- b* E2 l6 x2 y- b2 V* s2 s5 }" K
# SSRP sitemaps, M3 g P" w* x0 K- R
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
. d6 f7 _8 h: B9 T0 G
) p9 d: q" {5 S: G0 W" }# B#Stores Sitemaps% X! N1 [) H3 B
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
: A, E& k; U+ q$ ^( u+ D6 D1 f$ l3 J- Q6 r9 X) o" k
#BHP Sitemaps; ~) W1 T- `) V4 Y' t- \$ r
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml8 R8 y4 m3 A' H# [+ W5 s$ ?) B, k
6 r& r* x- i0 B& [4 T
#Collections$ d+ ^( W- J f$ T
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
% ~. H, X- k& y3 @: U% a2 S' S7 ~+ E4 o. e. n' n. }, L6 s
#VI
. s4 C, R! R) y1 j$ n5 aSitemap: http://www.ebay.com/lst/VI-0-index.xml8 g. C1 [ ?+ S
$ u( L: ^% l( k. p1 t
#PRP9 W: Z# f2 v- _/ q8 W
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
# Y2 A5 J- n& j* t$ L" Q Y V- _% z" n) Z+ G' M
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主