|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧) X0 `- ^ x- u) ?5 z. X8 }$ R% c2 V
http://www.ebay.com/robots.txt # sitemaps - SRPs# [1 }- P9 y( t' ]+ w; _8 _
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
3 b# G0 v; ^0 _Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
4 B4 n7 X) `0 U2 o P; `. ~, B9 z- k3 d9 y% M: J0 B# d
# Guides sitemaps, A; k# B" j- R* F
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
# E7 H7 f0 L3 j: n2 s4 m+ i$ r
# L9 T0 q6 y/ H3 N Z* ~5 i# SSRP sitemaps* v5 M7 r, b" W) q, k3 J5 T
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
8 b( p+ i4 I( Z. R0 ]3 @7 c% b5 F- T, g/ M
#Stores Sitemaps
2 f- e: ^9 h, \% R3 D& j mSitemap: http://www.ebay.com/lst/STORES-0-index.xml
' L1 W* p) l" G$ T
0 D: O0 O( p$ w$ U; y1 @( M7 @#BHP Sitemaps
+ ^" j5 i) h1 l& z: NSitemap: http://www.ebay.com/lst/BHP-0-index.xml
7 n1 N E$ ~ w. [. b5 h( ?" J/ s) [. j# a* V6 F
#Collections
9 U9 I& |# t6 E0 F4 rSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
% Z8 P5 n; r; m% j2 t; A5 I, Y( u" w; `
#VI3 R* R7 M. Z' u) y5 q3 j
Sitemap: http://www.ebay.com/lst/VI-0-index.xml9 H7 A2 P+ Y+ N: F* S0 z
2 J5 |* ?$ `$ w; F$ e8 ?! @
#PRP* i1 H4 T) m7 I) c3 H$ h( l& P
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
9 p! }& F8 [* X# ^
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主