看日志很多垃圾请求该怎么处理

幻中雪

! U* l/ l9 z0 S9 N* {
如图，最近很多这种请求，怎么禁都禁不掉，这种情况该怎么处理？请各位大神指教

2 T1 p9 N) N  b' a

* v7 W2 c) \6 \- K+ q. N( K  w
下面是请求的完整信息：
) h# m3 C, l$ w- N
5 o6 [0 r# Y$ I% f4 p& \172.70.39.35 - - [06/Feb/2024:06:28:35 +0000] ＂GET /40074brgq169.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
, h% q0 m6 |) I7 M; ?172.70.39.153 - - [06/Feb/2024:06:28:38 +0000] ＂GET /17074lamo26773s HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.199 - - [06/Feb/2024:06:28:41 +0000] ＂GET /17892idlt98418eil.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
' G4 A3 ?. t5 t5 j172.70.43.216 - - [06/Feb/2024:06:28:43 +0000] ＂GET /19828bpej97860q.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
172.70.38.169 - - [06/Feb/2024:06:28:44 +0000] ＂GET /143fgvj326.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
% |$ H* I# o& S* Y. C172.70.39.22 - - [06/Feb/2024:06:28:47 +0000] ＂GET /2226vvcj51483.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.134.15 - - [06/Feb/2024:06:28:47 +0000] ＂GET /18981qecl11688be.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
* e) Q! A& ]5 @2 o* D7 t172.70.39.186 - - [06/Feb/2024:06:28:50 +0000] ＂GET /5161rofm2119.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
' u+ h8 t( C) B! v$ l; B/ Z* z172.70.39.166 - - [06/Feb/2024:06:28:53 +0000] ＂GET /43672evjv6551 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.58 - - [06/Feb/2024:06:28:56 +0000] ＂GET /11782weir32886eil17 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
/ E, e7 s; N& I- i. _5 \172.70.38.176 - - [06/Feb/2024:06:28:59 +0000] ＂GET /34410weiritem/5426.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.85 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.134.124 - - [06/Feb/2024:06:29:00 +0000] ＂GET /33848acyg59963wav57 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
/ ]$ \; X9 W6 }; C! A& z2 y. a8 d# ~172.70.39.12 - - [06/Feb/2024:06:29:01 +0000] ＂GET /28186esxw52301c HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.169 - - [06/Feb/2024:06:29:03 +0000] ＂GET /7100iojk90709cir HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
8 y  Z  G  _9 [" c4 n& _4 O172.70.38.158 - - [06/Feb/2024:06:29:05 +0000] ＂GET /29526nkrc59400qrx34.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
. }6 f$ f' @" K: x7 p172.70.38.169 - - [06/Feb/2024:06:29:08 +0000] ＂GET /41423dwjw23648fwl53 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.93 - - [06/Feb/2024:06:29:11 +0000] ＂GET /10352dlom50245ed.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
- R9 J1 X# ]9 V) q172.71.223.163 - - [06/Feb/2024:06:29:11 +0000] ＂GET /44134rofm51169mfl.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
% V( R/ s& m3 o172.70.39.68 - - [06/Feb/2024:06:29:14 +0000] ＂GET /49372lqxg58776ylf HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.124 - - [06/Feb/2024:06:29:14 +0000] ＂GET /19989enpr81863vn.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
172.70.39.148 - - [06/Feb/2024:06:29:17 +0000] ＂GET /18586acyg28927w HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.215 - - [06/Feb/2024:06:29:18 +0000] ＂GET /16264rrlk58606orn20.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
172.70.38.114 - - [06/Feb/2024:06:29:20 +0000] ＂GET /30401roff52229.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.87 - - [06/Feb/2024:06:29:23 +0000] ＂GET /57205vhos47309f.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.39.103 - - [06/Feb/2024:06:29:26 +0000] ＂GET /41748zrlb61996szz.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.10 - - [06/Feb/2024:06:29:29 +0000] ＂GET /50310giik14829aig6.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
; B: U3 \! w& e172.70.39.20 - - [06/Feb/2024:06:29:32 +0000] ＂GET /51565hzqr88622h.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.39.127 - - [06/Feb/2024:06:29:35 +0000] ＂GET /49330enpr92303i.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.185 - - [06/Feb/2024:06:29:38 +0000] ＂GET /29345jfsy84455.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
- m: x- P; {' j7 Q& Q1 j172.70.38.146 - - [06/Feb/2024:06:29:41 +0000] ＂GET /43612luoh84417klr HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
3 J3 i3 o7 V9 I: r: R+ Y172.70.39.156 - - [06/Feb/2024:06:29:44 +0000] ＂GET /49332tzba7871 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
) n4 n: G6 T( i1 k172.70.39.198 - - [06/Feb/2024:06:29:47 +0000] ＂GET /48169roff83852roc6.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.63 - - [06/Feb/2024:06:29:49 +0000] ＂GET /7194weiritem/32869sib.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.85 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
7 z% w5 a0 h/ |3 e: V4 X) p172.70.39.136 - - [06/Feb/2024:06:29:51 +0000] ＂GET /9376befu73386.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
& h* V; {  k; F* [7 i172.70.38.128 - - [06/Feb/2024:06:29:53 +0000] ＂GET /28279nkrc19052r HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
; t+ h4 F$ x* ~, k9 I; B( `172.70.35.5 - - [06/Feb/2024:06:29:54 +0000] ＂GET /9976bpej81716sbr30.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
2 u6 d8 I& p. D) y/ A172.70.39.39 - - [06/Feb/2024:06:29:56 +0000] ＂GET /57899whqq87970x HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
7 ~' _' I6 b5 {, ^9 o9 V; G172.70.39.167 - - [06/Feb/2024:06:29:58 +0000] ＂GET /7692lqxg52259wld.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.85 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
4 ^7 G! y1 _  O6 E  ^172.70.38.9 - - [06/Feb/2024:06:30:00 +0000] ＂GET /58803giik15626rk.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.4 - - [06/Feb/2024:06:30:02 +0000] ＂GET /8521lwjy48666twe8.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.39.25 - - [06/Feb/2024:06:30:05 +0000] ＂GET /8736befu16686abm6.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.68 - - [06/Feb/2024:06:30:08 +0000] ＂GET /23333wqlq21234l HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
. F& E' E8 W* T172.70.38.120 - - [06/Feb/2024:06:30:11 +0000] ＂GET /10973mlkj72204blh HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.78 - - [06/Feb/2024:06:30:14 +0000] ＂GET /14028idlt56568oiv.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
5 Y/ a9 m% A$ Z& y172.70.38.181 - - [06/Feb/2024:06:30:17 +0000] ＂GET /31rofm28756fm.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.39.20 - - [06/Feb/2024:06:30:20 +0000] ＂GET /56163esxw42795dwm1.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
" Z! v* M) g# |+ m4 m3 l4 j172.70.39.55 - - [06/Feb/2024:06:30:23 +0000] ＂GET /11363yszz21765bz.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.39.154 - - [06/Feb/2024:06:30:26 +0000] ＂GET /43016esxw19951mex.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
2 y4 `$ k! R) L% T1 d172.70.39.103 - - [06/Feb/2024:06:30:29 +0000] ＂GET /15970tzba68859g.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.71.223.133 - - [06/Feb/2024:06:30:30 +0000] ＂GET /37318vecj60944ich.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
" H; p2 H4 v2 f6 J: y) C172.70.38.141 - - [06/Feb/2024:06:30:32 +0000] ＂GET /52357mlcy35961.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
! m' d# G/ }2 U172.70.39.131 - - [06/Feb/2024:06:30:35 +0000] ＂GET /23756lwjy21111slb05 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
4 K8 N, `) G5 F% k172.70.39.3 - - [06/Feb/2024:06:30:37 +0000] ＂GET /21912acygitem/990 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.85 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
7 W* L4 @' J3 b; V1 A172.70.38.103 - - [06/Feb/2024:06:30:39 +0000] ＂GET /39125brgq97507vrc6.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
! F: `8 @- N% |& o3 b3 `) X172.70.39.144 - - [06/Feb/2024:06:30:42 +0000] ＂GET /40540yszz56035gyo9.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
. {0 ~' k& A& {; Y, p: T: ]' M172.70.38.101 - - [06/Feb/2024:06:30:44 +0000] ＂GET /52746yszz81535szd69 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
) S( @- T1 x' b172.70.175.20 - - [06/Feb/2024:06:30:47 +0000] ＂GET /35883vvcj92969.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
' u: g1 K! J& G- y* ]172.70.38.36 - - [06/Feb/2024:06:30:47 +0000] ＂GET /1529mgnjitem/121 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
3 e" e/ y; N5 q3 i# @1 r172.70.39.48 - - [06/Feb/2024:06:30:50 +0000] ＂GET /54230gsel84276uej76 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
172.70.38.122 - - [06/Feb/2024:06:30:51 +0000] ＂GET /17888zymq90582azz31 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
* o1 n$ w2 \# x0 S172.70.39.150 - - [06/Feb/2024:06:30:53 +0000] ＂GET /58667mlcy6372 HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
0 H3 `+ B" Z9 `: Q0 o! H$ r172.70.134.152 - - [06/Feb/2024:06:30:54 +0000] ＂GET /41127mlkj15657vju2.html HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)＂
172.70.39.112 - - [06/Feb/2024:06:30:56 +0000] ＂GET /33918kjda45560odv02.htm HTTP/2.0＂ 404 12099 ＂-＂ ＂Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)＂
# t) J/ }+ b8 n% o' e. @

sead

检查看看是否中招，可能是webshell生成的链接，但是链接出了问题，爬虫过来抓就404.
3 n' m  N- |- m* ~7 ^如果站点页面不多，可以先site看下有没有多出陌生的收录。

朝阳风雨

看请求头中有没有猫腻，用cf禁掉这些请求

幻中雪

" o% a* E- P2 t  Y% j0 y' z

sead 发表于 2024-2-6 12:35
& o7 u: I. K8 U检查看看是否中招，可能是webshell生成的链接，但是链接出了问题，爬虫过来抓就404.
如果站点页面不多，可 ...

目前这个网站没有这些连接，但是很多这样的请求

您说的这种情况我另一个网站是有生成的很多类似的连接收录都四五十万了网页也被篡改了，排名都没了，把感染文件删除后排名恢复一些，但是这四五十万的垃圾收录该怎么办呢？这些页面也没有什么共同的前缀什么的......

幻中雪

8 N( [" p4 p, V$ \9 K

朝阳风雨 发表于 2024-2-6 13:40
" Z4 W4 E7 o; a3 q# m0 v看请求头中有没有猫腻，用cf禁掉这些请求

后面都是这样的
# K0 ?$ g! H! }0 G% q/ a7 |9 x6 n

幻中雪

sead 发表于 2024-2-6 12:35
# t2 N) E4 W4 U+ [6 }; q检查看看是否中招，可能是webshell生成的链接，但是链接出了问题，爬虫过来抓就404.
3 W2 U  O* r6 {8 u如果站点页面不多，可 ...

9 j0 L# {# A) V) X; n看了那些请求的完整信息，我感觉可能也是您说的这种情况了........疯了

朝阳风雨

幻中雪 发表于 2024-2-6 14:27
. A9 z" ^/ i- h. F2 z! @后面都是这样的

感觉这些都是爬虫了

fall2019

被黑了 做的寄生虫