|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
5 O+ S3 y2 q! U& L% T, Yhttp://www.ebay.com/robots.txt # sitemaps - SRPs; A( g6 f$ ~) \- `4 T, Q
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml: U% E6 p% w: g
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml' {8 m0 s7 k- n4 b* [
1 H' E' T! S$ W4 d4 t# Guides sitemaps
0 w$ G3 M( D3 o0 kSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
5 p7 U. c, W' v. |3 M- F1 t
( u1 q. @; c/ i1 R! q# SSRP sitemaps
6 f* G, \! c% N* S* wSitemap: http://www.ebay.com/lst/SSRP-0-index.xml7 g" V: T5 t1 H" z- Q
" N4 h% T, @7 N& z2 {2 ^$ a#Stores Sitemaps
8 f. ] P% }: f9 |Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
. S, I% A/ {# O* x7 l( G7 ]2 `! f. C1 C6 v- X1 ^" Q3 _
#BHP Sitemaps
* ^2 v% X. Z8 \) d! QSitemap: http://www.ebay.com/lst/BHP-0-index.xml
J7 x0 y( r, M- \6 Z8 D* z) M6 q0 j/ M9 `
#Collections F+ y( d! G( q$ y2 ~
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
8 {! B# |( q0 }; b+ T! u" F( [
1 V8 B. g. f) k- r#VI5 P3 x: ]& w$ O9 m( W2 a/ K5 C
Sitemap: http://www.ebay.com/lst/VI-0-index.xml& p" o5 Z( }( v/ Y
& C; z+ `0 B3 A- }# Q. i; I" ?#PRP4 N+ O- x7 ]# o5 S, _" h3 V
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
# ^4 n# X# }- Z% B, `- T
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主