|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧6 h9 A5 ~6 f( }2 Y2 ~. M
http://www.ebay.com/robots.txt # sitemaps - SRPs
5 M" u ?7 \: I% }Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
% U, K& i4 \% I7 z# z! XSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
# c5 N" ~# k# U. c/ I" [
- @$ r ^4 E, I4 F# Guides sitemaps9 }2 `; i, k1 U, C+ [
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml( J ?4 \& g6 M) d+ n t
7 w8 t2 H% J, x
# SSRP sitemaps
; [1 b7 @0 c) R1 kSitemap: http://www.ebay.com/lst/SSRP-0-index.xml, I% H0 d h* z. ~
6 Y& b& z2 H! W$ J2 R#Stores Sitemaps
8 [, D7 {* E$ z1 u0 _7 y5 |' uSitemap: http://www.ebay.com/lst/STORES-0-index.xml/ i+ l' P3 o7 y9 d! E I
s; m4 }: ?% u& A; d1 ?* ~
#BHP Sitemaps4 b2 ^! v! l1 d K+ K7 H
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
6 w5 v6 G* N2 M6 B% @% F5 I9 m5 n1 F9 W1 g2 S% O$ ~1 m3 Z
#Collections
. f- L8 e! Q. p8 z& BSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
/ `3 H, S/ c/ N" S [+ T
% r! A) V7 Z1 p#VI% J% w. L. n* J
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
: o1 L7 a3 m, b7 I9 o) L; [
: G, H; k. @, z#PRP/ m1 V( U; u+ g# Z* j% Y6 T
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
) y+ c9 L7 N# v* n |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主