|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
8 p) o( S! `/ c3 s, J j- Ohttp://www.ebay.com/robots.txt # sitemaps - SRPs
; N+ q k6 ^" d6 G- hSitemap: http://www.ebay.com/lst/SRP_US_index.xml
# l6 m7 x2 b; g( x0 [4 Q- bSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml' L4 y8 D9 ~4 K4 o, B+ t
1 l1 W3 R% _5 g8 b# Guides sitemaps
& S2 q% I# f( u8 e* w5 BSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml. {2 M6 q+ z: M, \$ S
) }! q% ?' V& {! l; H) r
# SSRP sitemaps. n9 B4 ?. {4 }! M. Q6 K; }1 d
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml5 d5 t9 X$ p8 v) ~ n) [0 Q
) o- S3 A9 l8 x/ Y% p5 V! O$ a& P#Stores Sitemaps& }/ i8 a! i- s2 Y, ]! l
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml r" ~5 R2 L1 B6 ^% J4 T8 ]
& L; E, c! r6 D
#BHP Sitemaps
9 c! D' i4 v& p- Q. WSitemap: http://www.ebay.com/lst/BHP-0-index.xml
8 T- T( `- d# x n+ j5 |+ N C. v! [6 N, f e0 l. t8 k
#Collections! h$ z7 ]- V& M4 g. `* r
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
! C! Q" t0 N5 K6 V* l0 u
W- N& q) C# K8 A1 g" Y#VI3 F. h- {2 }5 e2 Z- @
Sitemap: http://www.ebay.com/lst/VI-0-index.xml9 r, j$ F# B# j5 n
5 c3 \# I% G1 R" ^! ]% V# z#PRP9 f8 Z; y5 z" C" f/ F
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
" _$ q/ a5 ?$ a6 t% f# M |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主