|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
5 O9 j, E. p, k" i: nhttp://www.ebay.com/robots.txt # sitemaps - SRPs1 e# n4 h2 c* m
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
7 o1 }# V" [1 ]7 MSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml5 w) ]; g! n8 J0 y$ }8 Q6 C
5 z* F0 Y z5 a8 N! H" T$ U" F
# Guides sitemaps5 j, g0 L5 a |
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml \) m( e! v* S$ V7 Y* s
3 \! M/ A" c- c9 |( H4 @- q
# SSRP sitemaps
. ?' j) T3 q( d& K q) t0 X' u E: OSitemap: http://www.ebay.com/lst/SSRP-0-index.xml5 N9 F, X) L" {2 ]+ ^
' z2 d% Z! M: E m
#Stores Sitemaps0 X5 j* P' `# ^- {8 J& {, l
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml, Y# ]" c! b( A0 J, d! i
6 X- g8 L$ w+ |0 ?#BHP Sitemaps! y- {4 [; @2 j& x! B
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
4 ~, R& Y7 P9 d5 C0 S$ V' a
9 Y# C5 z. y* @: E, H9 Z#Collections+ \. _- t+ ^, k/ G- }5 p* Y- O" y& c
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml) b) h9 q3 W2 {7 h: J
4 S( Q4 @( W+ W M
#VI% P; F+ C! [% M1 t* S- V
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
" ]- H( k3 z$ u6 V7 N
/ w, Y) k+ C3 h5 O, m" ~#PRP1 l) p5 O* u4 z8 I+ r: o
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
+ S3 a0 a8 c' [$ s5 K |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主