8
回答
Jedis异常处理方式
利用AWS快速构建适用于生产的无服务器应用程序,免费试用12个月>>>   
我用Jedis的SharedJedisPool,配置为 MaxActive=500,MaxIdle=100,MaxWait=2000,testOnBorrow=true。redis服务器是Linux CentOS 6.0 内存98G,redis本身所占内存1.9G,Maxclients=655328,Linux的fs.file-max=9876450。服务器机构是 一主两从,主实例只负责写和持久化,两个从实例负责提供客户端读操作。主从在不同的机器,两个从实例在同一台机器上,主从实例在同一局域网。客户端其实也 是服务端程序,为40个Tomcat的实例,每20个访问一个从实例,客户端和redis服务不在同一个机器但是在同一个局域网。当有大量并发请求 redis客户端服务时,Jedis经常报错:
    redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
    at redis.clients.util.Pool.getResource(Pool.java:22)
    …………………………
    Caused by: java.util.NoSuchElementException: Could not create a validated object, cause: ValidateObject failed
           at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1233)
           at redis.clients.util.Pool.getResource(Pool.java:20)
           ... 21 more

或者
    redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
    at redis.clients.util.Pool.getResource(Pool.java:22)
    …………………………
    Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
           at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1233)
           at redis.clients.util.Pool.getResource(Pool.java:20)
           ... 21 more
其实发生这个错误的时候redis实例的connected_clients不足10000,大概就4000左右。使用redis-cli也连不上redis服务,报错:
    Could not connect to Redis at 127.0.0.1:6381: Connection timed out
使用netstat -anlpt|grep 127.0.0.1:6381查看连接到redis服务的tcp连接,发现如下问题:
     tcp        0      0 192.168.0.87:6381           192.168.0.144:48063         ESTABLISHED 15328/redis-server
tcp       14      0 192.168.0.87:6381           192.168.0.16:60948          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:48811         ESTABLISHED 15328/redis-server
tcp       55      0 192.168.0.87:6381           192.168.0.144:55022         ESTABLISHED 15328/redis-server
tcp       47      0 192.168.0.87:6381           192.168.0.144:23939         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:12101         ESTABLISHED 15328/redis-server
tcp       28      0 192.168.0.87:6381           192.168.0.144:45477         ESTABLISHED -
tcp       14      0 192.168.0.87:6381           192.168.0.144:47559         ESTABLISHED -
tcp        0      0 192.168.0.87:6381           192.168.0.16:2219           ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:9343          ESTABLISHED 15328/redis-server
tcp       14      0 192.168.0.87:6381           192.168.0.144:41640         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:17647         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:55616         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.16:61066          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:12193         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:60982          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:11833         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:60907         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:59617         ESTABLISHED 15328/redis-server
tcp       14      0 192.168.0.87:6381           192.168.0.144:9831          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:58487         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:44767         ESTABLISHED 15328/redis-server
tcp       55      0 192.168.0.87:6381           192.168.0.144:10469         ESTABLISHED 15328/redis-server
tcp       14      0 192.168.0.87:6381           192.168.0.144:54119         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:10207         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:56750          ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.16:61138          ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.16:56897          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:56862          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:16855         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:60538         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:8619          ESTABLISHED 15328/redis-server
tcp       55      0 192.168.0.87:6381           192.168.0.16:61052          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:2391           ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:2248           ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:17951         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:61067          ESTABLISHED 15328/redis-server
tcp       28      0 192.168.0.87:6381           192.168.0.16:30130          ESTABLISHED -
tcp        0      0 192.168.0.87:6381           192.168.0.144:10339         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:9826          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:56731          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:55006         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:11421         ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.16:30215          ESTABLISHED -
tcp        0      0 192.168.0.87:6381           192.168.0.144:30921         ESTABLISHED 15328/redis-server
tcp        0   1343 192.168.0.87:6381           192.168.0.144:8715          ESTABLISHED 15328/redis-server
tcp        0      0 192.168.0.87:6381           192.168.0.144:24553         ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:9021          ESTABLISHED 15328/redis-server
tcp       34      0 192.168.0.87:6381           192.168.0.144:16338         ESTABLISHED 15328/redis-server
tcp       14      0 192.168.0.87:6381           192.168.0.144:41356         ESTABLISHED 15328/redis-server
有些连接的Recv_Q和Send_Q不能很快的清空,redis服务的client list列表如下:
addr=192.168.0.7:44002 fd=452 name= age=222 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:44005 fd=459 name= age=222 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:44006 fd=461 name= age=222 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=sismember
addr=192.168.0.7:43593 fd=599 name= age=522 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43596 fd=602 name= age=522 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:44199 fd=737 name= age=206 idle=4 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43609 fd=619 name= age=522 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=1378 oll=0 omem=0 events=rw cmd=hgetall
addr=192.168.0.7:43610 fd=620 name= age=522 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=4 oll=0 omem=0 events=rw cmd=sismember
addr=192.168.0.7:43611 fd=621 name= age=522 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43615 fd=627 name= age=522 idle=14 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43247 fd=231 name= age=662 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43252 fd=238 name= age=662 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:42598 fd=494 name= age=984 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=118 oll=0 omem=0 events=rw cmd=hgetall
addr=192.168.0.7:43828 fd=390 name= age=333 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=hgetall
addr=192.168.0.7:43829 fd=391 name= age=333 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43979 fd=406 name= age=222 idle=8 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sismember
addr=192.168.0.7:43980 fd=407 name= age=222 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43983 fd=417 name= age=222 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43986 fd=422 name= age=222 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43989 fd=429 name= age=222 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43990 fd=430 name= age=222 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43992 fd=435 name= age=222 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=hgetall
addr=192.168.0.7:43708 fd=58 name= age=344 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43570 fd=575 name= age=522 idle=14 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sismember
addr=192.168.0.7:43572 fd=577 name= age=522 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43573 fd=578 name= age=522 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=hgetall
addr=192.168.0.7:43686 fd=29 name= age=377 idle=2 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43230 fd=202 name= age=662 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
addr=192.168.0.7:43236 fd=211 name= age=662 idle=14 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sismember
addr=192.168.0.7:43744 fd=160 name= age=334 idle=14 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
addr=192.168.0.7:43745 fd=161 name= age=334 idle=14 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
  1. 如果谁在使用redis的时候出现过同类的问题,如果解决?
  2. redis服务端和Jedis客户端如何配置才合适?
举报
yangkf
发帖于4年前 8回/25K+阅
顶部