Last modified: 2013-10-08 15:54:52 UTC
Hi Folks, i have been running CirrusSearch for about 2 weeks now. Great job. Can't wait for it to go live on all Wikis... Ubuntu 12.04 Server, MW Version: Master from Git, ElasticSearch 0.90.5 Here is a problem that started a couple of days ago: When I start from scratch and I run: php extensions/CirrusSearch/updateSearchIndexConfig.php this is the output: content index... Infering index identifier...first Creating index...ok Validating analyzers...ok Validating mappings... Validating mapping for page type...different...corrected Validating aliases... Validating content alias...alias is free...corrected Validating all alias...Unexpected non-MediaWiki exception encountered, of type "Elastica\Exception\ResponseException"
Wonderful to hear that you are running it. Can you answer some questions? 1. What is the hash of the commit you are using? 2. Is this error for setting up a new wiki? 3. Can you paste more of the error? I've been meaning to wrap the exceptions in the maintenance scripts in MediaWiki exceptions but haven't gotten around to it. The exception you are seeing basically just says Elasticsearch didn't like what you sent it. 4. What version of elasticsearch are you running?
1. master of today (070151a) 2. Yes and no. I erased the elasticsearch data first and then set it up following the instructions again. Same wiki. But the up to date version (from master) of CirrusSearch did not work anymore. 3. That is all that came back from the command line. sorry 4. I am running 0.90.5 as of today. It did not work yesterday as well with 0.90.3. 2 weeks ago everything was working fine also with 0.90.3. So I think it is more a CirrusSearch thing.
Thanks! 3. Bleh. 4. likely. I'm deep into something right now but I should finish it up soon and get to this, probably on Monday. Sorry for the wait.
no worries. I am happy as long as someone is looking into it. I am glad to run tests for you or help in ANY way!!!
Update: php updateSearchIndexConfig.php --reindexAndRemoveOk content index... Infering index identifier...first Index exists so validating... Validating number of shards...ok Validating number of replicas...ok Validating analyzers...ok Validating mappings... Validating mapping for page type...ok Validating aliases... Validating content alias...is taken... Reindexing... About to reindex 0 documents All done Swapping alias...done Validating all alias...Unexpected non-MediaWiki exception encountered, of type "Elastica\Exception\ResponseException"
Thanks for the update. I've put https://gerrit.wikimedia.org/r/#/c/88138/ up for Chad to review which should give us more information to work with. I'll update this when that is merged.
next update: now with the new output: php updateSearchIndexConfig.php content index... Infering index identifier...first Creating index...ok Validating analyzers...ok Validating mappings... Validating mapping for page type...different...corrected Validating aliases... Validating content alias...alias is free...corrected Validating all alias...Unexpected Elasticsearch failure. Elasticsearch failed at an unexpected place. Here is Elasticsearch's error message: alias [wikidb_mw1_test] missing
Would you mind running these: curl -s elasticsearch0:9200/_status?pretty | grep '"index" : "' | sort | uniq curl -s elasticsearch0:9200/_aliases?pretty echo 'global $wgDBname; echo($wgDBname);' | php maintenance/eval.php and posting the results? It looks like _something_ has gone haywire with the names of indexes and aliases. With those I should be able to post a work around and harden updateSearchIndexConfig.php to whatever it is that has happened.
echo 'global $wgDBname; echo($wgDBname);' | php maintenance/eval.php wikidb_mw1_test the 2 other commands: curl -s elasticsearch0:9200/_status?pretty | grep '"index" : "' | sort | uniq and curl -s elasticsearch0:9200/_aliases?pretty output nothing :-( but I tried this with localhost instead: curl -s http://localhost:9200/_status?pretty { "ok" : true, "_shards" : { "total" : 0, "successful" : 0, "failed" : 0 }, "indices" : { } } the other one: curl -s http://localhost:9200/_status?pretty | grep '"index" : "' | sort | uniq empty...
I can reproduce this locally by running: curl -XDELETE $ES_HOST:9200/$WIKI_DB_NAME I'll have a fix in a few moments.
Change 88479 had a related patch set uploaded by Manybubbles: Fix updateSearchIndexConfig for non-existant alias https://gerrit.wikimedia.org/r/88479
Got it. A fancy work around I added to stop the script from being slow when there were lots and lots of indexes on the cluster (50+) blew up when the index didn't exist. Now it won't. I don't have a good manual work around for this, unfortunately.
looking gooooood: php updateSearchIndexConfig.php content index... Infering index identifier...first Creating index...ok Validating analyzers...ok Validating mappings... Validating mapping for page type...different...corrected Validating aliases... Validating content alias...alias is free...corrected Validating all alias...alias not already assigned to this index...corrected general index... Infering index identifier...first Creating index...ok Validating analyzers...ok Validating mappings... Validating mapping for page type...different...corrected Validating aliases... Validating general alias...alias is free...corrected Validating all alias...alias not already assigned to this index...corrected
Thank you Nik, you have been a great help! So this is fixed!!!!
Change 88479 merged by jenkins-bot: Fix updateSearchIndexConfig for non-existant alias https://gerrit.wikimedia.org/r/88479
Merged and verified by keyler@gmail.com
