Last modified: 2014-10-22 21:06:54 UTC
This has been happening since we set up logstash initially, but in the prior version it would fail 3 times and then give up. Now it seems to keep trying to reset the plugin indefinitely. We should either fix whatever config is broken (maybe a password needed for redis?) or remove the unused input filter entirely until we figure out that we actually want to use it. {:timestamp=>"2014-10-18T00:23:35.210000+0000", :message=>"Failed to get event from Redis", :name=>"default", :exception=>#<Redis::CommandError: ERR operation not permitted>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis/client.rb:97:in `call'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis/client.rb:179:in `call_with_timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis/client.rb:244:in `with_socket_timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis/client.rb:242:in `with_socket_timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis/client.rb:178:in `call_with_timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis.rb:1038:in `_bpop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'", "file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'", "file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/monitor.rb:210:in `mon_synchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis.rb:1035:in `_bpop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/redis-3.0.7/lib/redis.rb:1064:in `blpop'", "/opt/logstash/lib/logstash/inputs/redis.rb:148:in `list_listener'", "/opt/logstash/lib/logstash/inputs/redis.rb:229:in `listener_loop'", "/opt/logstash/lib/logstash/inputs/redis.rb:245:in `run'", "/opt/logstash/lib/logstash/pipeline.rb:163:in `inputworker'", "/opt/logstash/lib/logstash/pipeline.rb:157:in `start_input'"], :level=>:warn} {:timestamp=>"2014-10-18T00:23:35.211000+0000", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Redis host=>\"127.0.0.1\", data_type=>\"list\", key=>\"logstash\", name=>\"default\">\n Error: ERR operation not permitted", :level=>:error}
Fixed by https://gerrit.wikimedia.org/r/#/c/168182/