Last modified: 2014-03-25 17:41:57 UTC
I'm unable to write to the database. Programs are just hanging. Having troubles reading in that it can take awhile to receive data. Things were ok 12 hours ago. Looks like problem has been happening atleast for 3 hours.
Now unable to connect to database via the website. I get "Could not connect to database: User 'p50380g50450' has exceeded the 'max_user_connections' resource (current value: 80)" I only have on hung program on the queue and nothing else that should be doing so many connctions.
I can access the database again. Coincidentally(?), about 50 mysql processes are no longer running on tools-login. Though there are still a few running by users since last year.
Which database server were you connecting to, i. e. tools-db or the replicas? (We had a full disk on tools-db, so that could have saturated that server (accepting connections, but then not being able to answer queries).)
Sorry, that would have been helpful: tools-db Having a full disk could have caused. Thanks for telling me.
This was, indeed, caused by a disk full on the (now gone) old database instance.