|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧+ E$ _! v& W& H _2 h7 H' P3 r t9 R
http://www.ebay.com/robots.txt # sitemaps - SRPs4 |1 [ G \2 T5 v+ @6 w
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml0 Z2 [- x6 A; [9 y0 _" ]
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml0 h; ~ m) b* h, o/ r
- i' o' g! q$ y! t% z0 x) k# Guides sitemaps3 _( h" o- _4 |9 i& F* L& P/ b
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml0 [ V3 v9 |6 T+ I
# `- p2 Q8 K( O2 |+ z1 \, X
# SSRP sitemaps% s/ F4 m2 x* z- z* r% F1 R
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
( H9 p0 g) {# H8 W$ O- e A* G% t; B3 H
#Stores Sitemaps! L' k/ n& [8 D& U: h1 }
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
, e3 l, P1 Z, S( j
7 |8 k4 z/ W1 I/ t#BHP Sitemaps
3 w+ D- X( o& B$ D! Y3 bSitemap: http://www.ebay.com/lst/BHP-0-index.xml7 _9 Y; l p( Y0 `7 \( n4 N
! p% j4 k" |* }. \
#Collections8 ]0 I4 R: ]/ M3 P
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml2 `9 i7 h/ N4 o# j& V+ L5 |
: q2 `" T( S$ n/ M. _
#VI
/ `4 Z! @' @1 a4 [+ V; R3 ~) _Sitemap: http://www.ebay.com/lst/VI-0-index.xml ]7 F- t) q/ m3 f) ]: f
u$ l9 ]" N4 l6 V7 b4 h8 C2 l#PRP
2 O( H; z+ H4 ]& x. g" C1 i: @Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
0 }! ~' K; ? T) e9 W# S- C2 r
4 @4 w, t+ Q/ {" c# m0 y |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主