|
|
发表于 2015-6-17 11:29:57
|
显示全部楼层
我也只能帮你到这里了，ebay、amazon等一些站，采集先看robots.txt,从robots.txt中找到sitemap的索引，剩下的就想办法搞吧$ c) B2 G5 \; {; y2 B% I
http://www.ebay.com/robots.txt # sitemaps - SRPs
' ?8 d7 {6 V# H# M8 OSitemap: http://www.ebay.com/lst/SRP_US_index.xml" X7 n9 w7 _3 l' l8 W
Sitemap: http://www.ebay.com/lst/ng/SRP_US_index.xml& B' X+ D' f9 F* R( f- X5 k
6 ^$ y0 F, j: l v
# Guides sitemaps. X" b7 p- K- y7 c0 J0 E K( Y9 D& G
Sitemap: http://www.ebay.com/lst/GUIDES-0-index.xml
: ]( x; p+ F3 w8 V5 J& @/ Y# | G' Q) B' H
# SSRP sitemaps
" k5 H2 [4 r- {, M- i* tSitemap: http://www.ebay.com/lst/SSRP-0-index.xml6 u; a; x+ L: `. g. P
7 s5 A0 {! O- X4 Y! x5 Q
#Stores Sitemaps
* H! h. L: x- {4 S3 x! Y6 c3 a' ASitemap: http://www.ebay.com/lst/STORES-0-index.xml6 i* w' `/ N! t+ Q$ J, ^1 H, }5 H
4 N# t, [7 G6 K3 f0 |: e% Z#BHP Sitemaps! Z* j; t( }: Y6 J
Sitemap: http://www.ebay.com/lst/BHP-0-index.xml" ~* E$ q8 T5 P3 g; M) ]
2 B, \* d: O* C! R1 L/ R
#Collections
/ Q; |( i; f! i9 sSitemap: http://www.ebay.com/lst/COLLECTIONS-0-index.xml
& o; s8 F: l! O1 I8 `7 Z( ~. P7 \2 T" k7 T! t! q F9 S
#VI
9 Z# e# B2 S' A7 f4 B1 ?Sitemap: http://www.ebay.com/lst/VI-0-index.xml
9 I" W5 N H" O" r. }; f# S1 P- Y; W0 K: h' C7 v" L, k
#PRP% y l4 p. X! x4 U5 y- i
Sitemap: http://www.ebay.com/lst/PRP-0-index.xml
1 I+ L+ C8 r, S a( w" W$ Z" W- G6 I. n( b- a
|
|
发表于 2015-6-17 08:44:18
发表于 2015-6-17 09:17:34
发表于 2015-6-17 12:29:33
楼主