|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧
( V5 V; _1 Z. i8 Uhttp://www.ebay.com/robots.txt # sitemaps - SRPs
& R: O! j+ {- r1 W) g MSitemap: http://www.ebay.com/lst/SRP_US_index.xml
; M ?' P; g7 `4 X) \& G+ \$ _" GSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml1 {- @* p4 c! F. f9 L5 u* `
" f& ]8 M5 _' |% O5 R
# Guides sitemaps* e* ^0 |4 z) k. w! K1 F: S
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
, M+ r" ~9 q, x6 _- k: o2 {" Y$ a5 U6 z4 t! H3 `
# SSRP sitemaps6 h/ {) M+ i1 g. m5 Z4 x) x
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
: q7 u& F' G" g# \! p/ [0 [
, A/ ^* [4 h2 b4 J- L8 T4 F, o#Stores Sitemaps
0 Z" Y* D" {# D+ U! YSitemap: http://www.ebay.com/lst/STORES-0-index.xml q) l% A1 r- w, f7 a' M
4 `2 ?& p0 E4 O! O l3 s+ M
#BHP Sitemaps0 Y0 x* A; P. D+ }( G( s
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
9 m# }. y8 }/ N1 Q
: s6 p1 k- w: f! u. n Y) P: n) _/ ], ] a#Collections @6 a% |9 _+ D' @4 ~
Sitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml* H. G. B1 ?# p/ W1 ?; Q6 b% ]3 Q
$ W/ u5 Q1 L T0 Q#VI \! L& b A0 \* c4 f8 s
Sitemap: http://www.ebay.com/lst/VI-0-index.xml% H9 D9 ~/ n$ t1 }
5 v$ _1 T8 h4 [- }, o& G
#PRP' j* L) s J& G# `3 G/ B' C
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
+ p$ ^4 p* M, Z- F( D
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33