Last modified: 2013-05-21 08:49:41 UTC
2013-04-19 19:32:17.144065] W [client3_1-fops.c:2385:client3_1_rename_cbk] 1-deployment-prep-project-client-0: remote operation failed: Cannot allocate memory [2013-04-19 19:32:17.144123] W [fuse-bridge.c:1516:fuse_rename_cbk] 0-glusterfs-fuse: 267286479: /apache/common-local/php-master/extensions/.git/modules/GoogleMaps/refs/remotes/origin/master.lock -> /apache/common-local/php-master/extensions/.git/modules/GoogleMaps/refs/remotes/origin/master => -1 (Cannot allocate memory) That is on deployment-prep project when trying to update a git extension. $ cd /data/project/apache/common-local/php-master/extensions $ git submodule update --init GoogleMaps Submodule 'GoogleMaps' () registered for path 'GoogleMaps' error: Couldn't set refs/remotes/origin/master From https://gerrit.wikimedia.org/r/p/mediawiki/extensions/GoogleMaps ! 91a381a..7f6454f master -> origin/master (unable to update local ref) Unable to fetch in submodule path 'GoogleMaps' $ Got again the same error: [2013-04-19 19:35:42.160256] W [client3_1-fops.c:2385:client3_1_rename_cbk] 1-deployment-prep-project-client-0: remote operation failed: Cannot allocate memory [2013-04-19 19:35:42.160537] W [fuse-bridge.c:1516:fuse_rename_cbk] 0-glusterfs-fuse: 267341371: /apache/common-local/php-master/extensions/.git/modules/GoogleMaps/refs/remotes/origin/master.lock -> /apache/common-local/php-master/extensions/.git/modules/GoogleMaps/refs/remotes/origin/master => -1 (Cannot allocate memory) I have no idea whether that is client side or server side.
And /logs/ is dead too: [fuse-bridge.c:292:fuse_entry_cbk] 0-glusterfs-fuse: 9759: LOOKUP() /logs/web.log => -1 (Input/output error) Reassigning bug to infra. Impact: that at least break the update of MediaWiki extensions.
See also bug 47479 " GlusterFS deployment-prep-project has too many errors -> beta cluster is down"
apparently the "cannot allocate memory" error no more happens on deployment-bastion. Self fixed?
Still happening when update the mw extensions on deployment-prep project: [2013-04-22 21:45:05.548595] W [client3_1-fops.c:2385:client3_1_rename_cbk] 0-deployment-prep-project-client-1: remote operation failed: Cannot allocate memory [2013-04-22 21:46:10.686871] W [client3_1-fops.c:649:client3_1_rmdir_cbk] 0-deployment-prep-project-client-0: remote operation failed: Directory not empty [2013-04-22 21:46:10.687208] W [client3_1-fops.c:649:client3_1_rmdir_cbk] 0-deployment-prep-project-client-1: remote operation failed: Directory not empty
I got rid of Gluster in favor of role::labsnfs::client so there is no point in keeping this bug around I guess.
