Apache log

巴林的狗尾草 发布于 2013/07/16 13:45
阅读 658
收藏 0

@红薯 你好,想跟你请教个问题:

我在自己的服务器上面的apache的log里面发现大量的下面这种东西的log,请问是不是在被爬虫搞还是什么东西?这个东西巅峰的时候给我搞出来好几百个连接,导致我都没法登陆自己的服务器。 

199.188.110.3 - - [16/Jul/2013:01:42:16 -0400] "GET http://www.progarchives.com//www.progarchives.com/forum_topics.asp?FID=19 HTTP/1.0" 404 525 "http://www.progarchives.com//www.progarchives.com/forum_topics.asp?FID=19" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
199.188.110.3 - - [16/Jul/2013:01:42:16 -0400] "GET http://favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers) HTTP/1.0" 404 498 "http://favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers)" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
218.31.6.180 - - [16/Jul/2013:01:42:17 -0400] "GET http://www.baidu.com/ HTTP/1.1" 301 364 "http://www.baidu.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)"
199.188.110.3 - - [16/Jul/2013:01:42:17 -0400] "GET http://favorites.live.com//favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers) HTTP/1.0" 404 518 "http://favorites.live.com//favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers)" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
218.31.6.180 - - [16/Jul/2013:01:42:17 -0400] "GET http://www.baidu.comhttp/www.baidu.com/ HTTP/1.1" 404 445 "http://www.baidu.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)"
199.188.110.3 - - [16/Jul/2013:01:42:17 -0400] "GET http://favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers) HTTP/1.0" 404 498 "http://favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers)" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
199.188.110.3 - - [16/Jul/2013:01:42:18 -0400] "GET http://favorites.live.com//favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers) HTTP/1.0" 404 518 "http://favorites.live.com//favorites.live.com/quickadd.aspx?url=http%3A%2F%2Fwww%2Eprogarchives%2Ecom%2Fforum%2Fforum%5Fposts%2Easp%3FTID%3D89611&title=Nektar release new album "Time Machine" (+ covers)" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
199.188.110.3 - - [16/Jul/2013:01:42:18 -0400] "GET http://www.progarchives.com/forum_topics.asp?FID=19 HTTP/1.0" 404 503 "http://www.progarchives.com/forum_topics.asp?FID=19" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
199.188.110.3 - - [16/Jul/2013:01:42:18 -0400] "GET http://www.progarchives.com//www.progarchives.com/forum_topics.asp?FID=19 HTTP/1.0" 404 525 "http://www.progarchives.com//www.progarchives.com/forum_topics.asp?FID=19" "Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11"
109.60.115.203 - - [16/Jul/2013:01:42:19 -0400] "GET http://www.thelisaann.com/members/ HTTP/1.0" 404 493 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/522.11.3 (KHTML, like Gecko) Version/3.0 Safari/522.11.3"
174.132.223.199 - - [16/Jul/2013:01:42:22 -0400] "GET http://login.yahoo.com/  HTTP/1.0" 301 327 "-" "-"
109.60.115.203 - - [16/Jul/2013:01:42:25 -0400] "GET http://members.tonightsgirlfriend.com/?&m=login HTTP/1.0" 301 380 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/522.11.3 (KHTML, like Gecko) Version/3.0 Safari/522.11.3"
113.238.0.53 - - [16/Jul/2013:01:42:25 -0400] "GET http://www.baidu.com/ HTTP/1.1" 301 364 "http://www.baidu.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)"
113.238.0.53 - - [16/Jul/2013:01:42:26 -0400] "GET http://www.baidu.comhttp/www.baidu.com/ HTTP/1.1" 404 445 "http://www.baidu.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)"
109.60.115.203 - - [16/Jul/2013:01:42:26 -0400] "GET http://members.tonightsgirlfriend.comhttp/members.tonightsgirlfriend.com/?m=login HTTP/1.0" 404 532 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/522.11.3 (KHTML, like Gecko) Version/3.0 Safari/522.11.3"

加载中
0
geminiblue
geminiblue
把rebots.txt写好......好几百的连接你服务器就打不开了?神码情况?
0
巴林的狗尾草
巴林的狗尾草

引用来自“geminiblue”的答案

把rebots.txt写好......好几百的连接你服务器就打不开了?神码情况?
可是我不确定这个是蜘蛛啊,你看他们请求的URL都不是我的网站的内容,我也不懂为什么会出现这种情况,求指教,还有就是好几百是并发连接,就是检测到netstat有三百左右的连接同时发生的时候,apache还是正常工作的,但是其余的操作基本都block了。这个可能跟我我的服务器配置有关。
0
巴林的狗尾草
巴林的狗尾草

引用来自“geminiblue”的答案

把rebots.txt写好......好几百的连接你服务器就打不开了?神码情况?

还有个问题,如果是蜘蛛的话应该是下面的格式的吧, 请求的URL应该是不会再有域名了吧?上面的请求看起来更加的像是一个请求转发,表示真心不知道什么情况了。

access.log.1:220.181.108.90 - - [14/Jul/2013:03:44:38 -0400] "GET / HTTP/1.1" 200 16800 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:123.125.71.52 - - [14/Jul/2013:03:45:11 -0400] "GET / HTTP/1.1" 200 16800 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:123.125.71.31 - - [14/Jul/2013:04:32:19 -0400] "GET /wp-content/themes/iShopping_V1.0/images/random/small/tb10.jpg HTTP/1.1" 200 48607 "-" "Baiduspider-image+(+http://www.baidu.com/search/spider.htm)"
access.log.1:123.125.71.25 - - [14/Jul/2013:04:44:05 -0400] "GET / HTTP/1.1" 200 17029 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:220.181.108.91 - - [14/Jul/2013:04:47:09 -0400] "GET / HTTP/1.1" 200 16800 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:220.181.108.90 - - [14/Jul/2013:05:44:34 -0400] "GET /wp-content/themes/iShopping_V1.0/images/random/big/big7.jpg HTTP/1.1" 200 74666 "-" "Baiduspider-image+(+http://www.baidu.com/search/spider.htm)"
access.log.1:123.125.71.36 - - [14/Jul/2013:05:55:25 -0400] "GET / HTTP/1.1" 200 17039 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:220.181.108.117 - - [14/Jul/2013:05:55:36 -0400] "GET / HTTP/1.1" 200 16800 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
access.log.1:220.181.108.112 - - [14/Jul/2013:06:47:32 -0400] "GET / HTTP/1.1" 200 16800 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

0
巴林的狗尾草
巴林的狗尾草
谁能帮忙解释一下这个log是什么情况??
0
巴林的狗尾草
巴林的狗尾草

引用来自“geminiblue”的答案

把rebots.txt写好......好几百的连接你服务器就打不开了?神码情况?
求指教,我写了robots然后就好了,但是为什么呢,求指教
返回顶部
顶部