|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧1 c1 l( J1 T! M4 h3 v
http://www.ebay.com/robots.txt # sitemaps - SRPs
, s8 E1 B5 Z5 q' r* aSitemap: http://www.ebay.com/lst/SRP_US_index.xml
/ Z" b. M- S' V! o- y6 M3 fSitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml. A& w3 N6 k4 p
~' y# \. r$ }( S1 L$ N
# Guides sitemaps, v4 J3 f+ A0 z: m8 p4 B/ e/ f" p
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
" n9 ?6 [) \2 g6 z
% V( K4 t- L, U! J# SSRP sitemaps2 f$ ]4 p( P/ E1 V: O0 l! m& W' l
Sitemap: http://www.ebay.com/lst/SSRP-0-index.xml
2 w9 K# D4 p& A: m6 {0 |& d! L- n# u% o: A; f
#Stores Sitemaps, ^2 T- ^, Y# [9 I/ y% Y
Sitemap: http://www.ebay.com/lst/STORES-0-index.xml: t8 D' B: R2 q( T; C
$ N+ K* h4 p5 ?! n. h6 \) v, H
#BHP Sitemaps2 l- u! C) d1 P: G- b
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml
8 J% P Q6 J X
4 d8 ~: ]! V2 _6 S. R* j |0 w#Collections
2 a' z1 l* A$ r. gSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml" Q) q: {; C' f+ ^( L4 F
+ i5 E% g4 C! W4 S5 Q+ o
#VI D, w* t0 O7 C5 q; A
Sitemap: http://www.ebay.com/lst/VI-0-index.xml
3 F% {# b0 M8 c) G+ b8 {/ c" u }# S) ~
#PRP
% Q0 Q/ m7 W+ m; `Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
/ k# ] Q/ c6 k3 o! P2 ]" o# V" H+ C. ?* u |
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主