|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧+ f* z* j9 v0 c/ C
http://www.ebay.com/robots.txt # sitemaps - SRPs, }/ C* t) q% D1 K) Z$ l6 E
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
- \# A8 S) w& _; h5 iSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml
. x* w8 ?5 {; m. W6 F4 k* O/ q. R' U. N
# Guides sitemaps
4 V/ r1 E+ i0 a" I$ x7 l9 ~Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml; x# `$ @; x: y d+ }+ ]0 N
% m4 k+ U( q! L2 F; T8 R! I6 P
# SSRP sitemaps, H! ^* c# {. b% i( g9 k
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml4 H& M, x$ u n& Y
) @$ C- ^3 ], S3 w, c#Stores Sitemaps
3 @& j/ i3 I3 i! F- [Sitemap: http://www.ebay.com/lst/STORES-0-index.xml; r8 w" K7 M) U( U1 W9 ^
& F- G$ t" O; V2 r#BHP Sitemaps" [* w6 C4 H& b% R; w F# R$ [7 d; z
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml$ k4 |9 |3 R/ @, W0 \$ m/ N
) F1 Y" S$ F5 A- j4 }/ V( [
#Collections# x0 t( l3 R8 m
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml& w# n; b' Z4 T/ `* u2 M$ P
. g& S. g) Q& Y1 [5 y/ X+ e
#VI2 O6 [3 `' d: b( T
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
: i7 H( _' F* k D$ r. N+ a/ ^5 d' b8 M0 e
#PRP
- m+ a1 T3 S' d% MSitemap: http://www.ebay.com/lst/PRP-0-index.xml
6 H& p5 E% H. P# j5 g* U4 \& d9 z( M |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主