|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧 H3 }$ d+ o) d: a0 t- y
http://www.ebay.com/robots.txt # sitemaps - SRPs$ h/ \+ f) F) _' B( m4 n& j
Sitemap: http://www.ebay.com/lst/SRP_US_index.xml
6 K5 [ h8 p: v+ a% SSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml- T2 Z; M+ Q( t' d
' i# ~2 f C3 H6 ?* k; n4 I
# Guides sitemaps
" N9 k3 I* e- P8 ?! CSitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
4 S& B4 t- p/ B" R) M! z
" Z: R c* H/ ~. {) q# C# SSRP sitemaps
" b; ~+ c; Q& I8 @& b: L4 `% DSitemap: http://www.ebay.com/lst/SSRP-0-index.xml% t2 g/ m: J# }
6 \: W2 h! D: I# P8 k- n# U#Stores Sitemaps
/ j" z6 U( }7 RSitemap: http://www.ebay.com/lst/STORES-0-index.xml
, J9 f: h M9 |; t5 L. l G8 ]. |4 H' H
#BHP Sitemaps
3 D0 O9 F/ i1 o6 ^! w0 F& ySitemap: http://www.ebay.com/lst/BHP-0-index.xml
& ~7 a/ c @: |5 m& B! G# ]* I8 v1 ]0 O2 h$ M
#Collections2 z; g1 u Q& S$ W
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
( i) Q+ k" Z, [7 `$ N$ ^9 c) M$ O0 s( r8 t$ ]$ G# O" q9 x# a
#VI+ {, o6 f$ h& m7 ^9 E
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
* u" }* A- h: e0 |- o7 G( Z( z1 ?
! o" D5 c( b6 }4 ~8 x* f#PRP
' T7 A" g) K! E. |6 sSitemap: http://www.ebay.com/lst/PRP-0-index.xml
8 w$ c& S0 T3 u& |
( w" r" i- I2 H* K7 X |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33