|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
7 A4 V+ z" b+ M ]; Fhttp://www.ebay.com/robots.txt # sitemaps - SRPs
7 A* T. u6 ^ G* `6 O8 H+ `Sitemap: http://www.ebay.com/lst/SRP_US_index.xml9 q2 s8 \: |' Q& i
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml# ^, ~" Q+ D* s1 D" M5 {
! G! ]; m% I$ t# Guides sitemaps
; a6 h7 \, n& e. V, M0 o! A. r9 M" iSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml) o7 }' }* M v0 `; X6 N8 c* f0 a& a
! |& T' I3 F( ]0 i) Y% p
# SSRP sitemaps( e7 [, B$ F0 p- u) T
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
! Z0 R- w6 G4 s9 q# u; P; h6 p* j6 g. [1 L" _
#Stores Sitemaps
* @* S3 h" o# h0 Z/ {Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
1 Q; \# W9 K/ `: L' t. e
7 i" z2 ^8 r# V& T8 r#BHP Sitemaps
! D* u, K c5 U( K* B! vSitemap: http://www.ebay.com/lst/BHP-0-index.xml
- U% t( b! u: n$ C9 I2 `2 M% d; }. [# _! z, G# c3 D
#Collections) C, v% i; M; P1 A5 d! m
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml& B) Y. [7 J$ z" n4 O
3 ]8 y( u/ }4 T+ g6 q/ H
#VI
/ Q; A' F4 q5 f4 b5 U( I# ^Sitemap: http://www.ebay.com/lst/VI-0-index.xml$ {+ {5 o0 m( J+ } e4 h8 B
3 e9 ?9 p5 A0 U
#PRP+ ~4 ^; D x8 J! ]
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
8 a8 _3 @5 @# Y7 _4 @ |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主