|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧8 G$ w, B5 s" e- X1 [
http://www.ebay.com/robots.txt # sitemaps - SRPs( V" x( L1 k$ S0 ?$ ^1 n1 ^
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml+ r9 Q( d' S8 s' t
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml/ t9 Y7 r! K/ x0 \3 ^9 R5 L2 p x% e
! m* f m: i' O. K, j
# Guides sitemaps: D9 B; d- T: R
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
9 h0 c4 m6 [( h; L2 v) p3 L& ^2 e/ L s. U( Q+ ?
# SSRP sitemaps
; S1 Q; e( B9 x- k1 aSitemap: http://www.ebay.com/lst/SSRP-0-index.xml
9 I2 x) T O! }9 h' N
3 X0 w" H3 [$ _# N* z#Stores Sitemaps7 f( U; c6 A) k g: w% T
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml
5 l4 P. Q, ^! o b K. c, g/ n
: P+ K) L; R! G+ D0 E" y' Q#BHP Sitemaps
5 R! o4 U- y/ K1 W3 b* q3 J0 gSitemap: http://www.ebay.com/lst/BHP-0-index.xml
, z2 u" V& R t: h) P' y9 \6 K+ M+ ]: Q" c) q% x
#Collections" n0 t5 m8 N. l' H5 g! h7 U/ e
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
+ Q+ O' d6 D3 [ `! d: k! ]1 G& {3 Y8 m
#VI
' B1 E- G4 Y' t! zSitemap: http://www.ebay.com/lst/VI-0-index.xml& s1 b/ \& K2 D8 j; w" @5 E
5 H8 |" s3 s1 L9 l( |#PRP
( F1 M+ i# t0 ~1 [& `3 oSitemap: http://www.ebay.com/lst/PRP-0-index.xml
0 T, \8 ~8 y6 a. t; s" `( h1 w6 [ |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主