|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
& c, H: u' r1 e! Thttp://www.ebay.com/robots.txt # sitemaps - SRPs
# v/ ~! ^' H% a$ _- g5 ySitemap: http://www.ebay.com/lst/SRP_US_index.xml
q9 w& m! e: J9 GSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml" P/ c2 x! K2 L+ H6 S f. `
3 J" q: v! {& ?6 O# }2 \% g; ^ C: a
# Guides sitemaps
1 |5 v2 V) ]6 V5 v% F6 z5 X6 N! @Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
2 I u. B3 v5 ^8 w( R- L" j" M5 c) ?
# SSRP sitemaps$ D8 k/ Y3 W: U- }" `
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml' ~3 \6 L7 Z _
9 |$ J2 L" ]0 M5 Z, e! N#Stores Sitemaps$ P: x( l3 f$ x' @' w6 k P. k0 O+ W
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml0 S j- Q6 V$ E
; `9 K9 {3 k) C$ d; X w8 s, P" c/ v
#BHP Sitemaps
/ t) q: ^7 ]3 ~! k" X" MSitemap: http://www.ebay.com/lst/BHP-0-index.xml, Z* {! J' A5 u
8 N) t: r% `9 ?. V# \- o9 K; e6 j
#Collections! E: U8 g2 B$ ~! _4 j8 T/ i
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml2 V: L$ J' y* S1 G0 s3 j& A- y- }
/ h; Q+ f2 z0 y: F
#VI/ R0 J' K0 s/ J. J$ r1 n
Sitemap: http://www.ebay.com/lst/VI-0-index.xml) [/ A2 o% \; G) s- p2 N
& v( q* u- H* P! O: `#PRP
5 y4 R" d7 C6 w" B7 @0 K" ^Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
/ N6 }5 O+ N% W6 I. l' a4 G
/ g8 \2 J2 G% [. [7 J/ K |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主