|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧* [5 Z# v) l E3 M. Q: ^9 d3 u
http://www.ebay.com/robots.txt # sitemaps - SRPs
% {$ D5 d6 @+ U2 ~Sitemap: http://www.ebay.com/lst/SRP_US_index.xml. b4 z" h8 H# U6 f2 \ J- b$ J1 B
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml* ?% a4 m N5 V& F# r
: |7 u% ^" Q/ e
# Guides sitemaps9 A8 g g0 Z1 O4 \$ q. @1 D1 D
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
' R: M. [7 j5 x# C+ N. B$ D9 K
# SSRP sitemaps/ t* j+ U/ Y) C: p c
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml0 _& ]3 i( J; l) i& G8 e" K1 g
. ^5 A/ h# c% z6 w& E
#Stores Sitemaps
# c& Q9 X2 N" H Z# w6 n& y0 eSitemap: http://www.ebay.com/lst/STORES-0-index.xml
6 j) S$ a% y2 D* A6 g/ r% ]5 Z' l2 G
/ j, I+ H/ K* J# h# a2 q6 ?#BHP Sitemaps3 q2 `3 R, [3 o. X* |/ I3 y: ?9 m
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml( W; r& G) J. e9 I3 t4 c1 G
7 Q8 q! ^8 D% W#Collections0 x0 D: b6 [: r, A( C! s
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml2 C1 f d/ Y6 _- g+ ^8 _8 d4 P
+ ^" E2 f4 s4 z, A$ p1 [' t. t#VI5 c' V8 g9 H9 }
Sitemap: http://www.ebay.com/lst/VI-0-index.xml( o2 w' _" ]/ l4 Z# n X
& Y. ]* q) a4 q0 W+ v2 k
#PRP
" x& R3 O' Z7 t: @; e- L0 V. zSitemap: http://www.ebay.com/lst/PRP-0-index.xml
& t% L/ _% `' i d, P1 F/ d
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33