|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了,ebay、amazon等一些站,采集先看robots.txt,从robots.txt中找到sitemap的索引,剩下的就想办法搞吧1 F3 A3 E% D z8 h( p0 t' }; k
http://www.ebay.com/robots.txt # sitemaps - SRPs& P2 m3 a' I h0 ?6 ]: Z! i2 D
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml O7 ^& x1 |9 ~/ ?, o% e( x
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml- [+ f( w1 q7 ]& a! |8 \
0 r$ B. n, N" i% j5 C) V
# Guides sitemaps; H' M$ z5 ?6 K
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
2 s z5 l! y% H8 ^' h. N ^7 I
, d( K" ^( U, I# SSRP sitemaps9 [/ g8 ]! C) m" ?! m0 \
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml; b/ w }6 ^/ v1 Z: _8 ?* n
, B2 Q+ \% u: D* F- P#Stores Sitemaps
5 `. I6 k8 @& d. YSitemap: http://www.ebay.com/lst/STORES-0-index.xml
: S0 \, R* c2 Y2 a" c* Z
9 `4 e! q9 Y: A. ]; Z3 C6 ?# h1 C% \#BHP Sitemaps. A, ?: i7 r& p! g0 g0 W' a
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
' J- i! R7 E+ b! I- {, o9 c5 K3 t |( ~1 J" }& R
#Collections
; z: m# X$ G B% X4 y6 OSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
& M& i y, E; @" b6 p
1 i& F9 K; ]& \3 ^- ]% y/ f. Y#VI
/ ], R# m& ?; ]4 ^Sitemap: http://www.ebay.com/lst/VI-0-index.xml
8 y% w0 G* C0 L/ n' g, e5 H3 g0 y ~! c
#PRP
/ K9 Q3 I U, m1 A1 Y5 QSitemap: http://www.ebay.com/lst/PRP-0-index.xml ' W9 H' O5 X. b6 g3 |$ Q7 ^$ g) ]" Z
& h! \* U: L; {6 K' J, P
|
|