Last modified: 2013-05-20 08:25:40 UTC
The deployment-bastion instance is a udp2log receiver for MediaWiki. On log rotation, the logs files are recreated with root:root ownership which prevents the udp2log daemon from writing to it (it is run by user udp2log). The instance has the class role::beta::logging::mediawiki which is really: role::logging::mediawiki with /data/project/logs as a log directory. The /etc/logrotate.d/udp2log-mw does not specify anything. Maybe that is caused by the NFS mount.
As root, I could not change the ownership of a log file on the NFS share: $ sudo touch /data/project/logs/logrotatetest.log $ ls -l /data/project/logs/logrotatetest.log $ -rw-rw-r-- 1 root root 0 May 2 08:49 /data/project/logs/logrotatetest.log $ sudo chown udp2log:udp2log logrotatetest.log chown: changing ownership of `logrotatetest.log': Invalid argument $ `copytruncate` would solve that since the file is copied and then its content removed. But that would mean losing some logs sent between the two operations, I dont think this is acceptable for production. `nocreate` would not attempt to recreate the log files. That would only work if HUP signal sent to udp2log actually recreate the file. I believe that is the case since when the files are rotated and recreated by log rotated, the inode changed and thus udp2log must be reopening them, possibly recreating it.
Related URL: https://gerrit.wikimedia.org/r/61964 (Gerrit Change I048c6914b6bf7c93cc7d98658391ba90d388ff4b)
This is now fixed. Maybe there was a NFS configuration issue as well.
