|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
( b5 j& o \( A# b$ Ehttp://www.ebay.com/robots.txt # sitemaps - SRPs
# L* k& v' o0 J3 s4 qSitemap: http://www.ebay.com/lst/SRP_US_index.xml) s% F5 m/ u3 r
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml) S- @5 \6 Z( S5 `
" E4 V( o$ D1 r/ Q# Guides sitemaps
- y1 _- s: `+ ESitemap: http://www.ebay.com/lst/GUIDES-0-index.xml% U( l: ~3 e7 [0 L7 [' d$ z
+ Q, _$ ^4 {3 z& R, T
# SSRP sitemaps5 R: N) w$ U: `, u
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
) P: N' Y0 T' r8 B* h9 \7 e
* T) v, e. a/ w* i9 |6 ^1 v- N#Stores Sitemaps) ^) [$ o; T- G% k3 O7 y4 }- N
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
8 a o" z# @# J" i* _
. b- i* U& u, K, }+ J) e3 N#BHP Sitemaps
# @; R, b( I& h& ~2 F8 O* \Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
& q% L; ^8 x; K5 w+ v3 r- S2 X) _% @6 j# A
#Collections N) w! J0 H7 m: W
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml) S+ ?$ j' m- T/ I& a4 B
- q5 r$ E" _% K3 c$ W#VI& ?. G# Z) e/ c" u
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
5 Z8 h5 m {: | `& E4 d
3 L! j1 H3 y# H0 O#PRP }8 {) c' o! t- q! p. d4 y! I
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
2 W: T/ H2 [5 c2 g. V X/ i; k' M+ B% y% Z: q" n; e
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主