Last modified: 2014-11-17 09:21:10 UTC
On some of our wikis, when #asking (or #showing, or browsing with the Semantic Drilldown extension) the properties of some articles, we get all the values twice. The same values, which should be impossible. E.g. imagine we had a Property:Version and {{#set:Version=1.0}}, if we {{#show:Page|?Version}} we'd get "1.0, 1.0". When this starts happening for a given article, it'll happen all the time for that article. So it doesn't happen at run time randomly for any article: at some point it "activates" for some articles, and once it starts happening for a particular article it's there to stay. OTOH the values only appear once on the factbox at the bottom of the page, they're only duplicated on semantic queries and drilldown. After digging on the logs and using the debug bar I found out that the SQL queries run on a wiki without the duplicated values issue and on one with it are the same, the problem comes in the results of this particular query: SELECT p.smw_title as prop,o_blob AS v0,o_hash AS v1 FROM `smw_di_blob` INNER JOIN `smw_object_ids` AS p ON p_id=p.smw_id WHERE s_id='337192' On the wiki with the problem I get something like this: +--------------+------+-----------+ | prop | v0 | v1 | +--------------+------+-----------+ | Property 1 | NULL | Value 1 | | Property 1 | NULL | Value 1 | | Property 2 | NULL | Value 2 | | Property 2 | NULL | Value 2 | | Property 3 | NULL | Value 3 | | Property 3 | NULL | Value 3 | +--------------+------+-----------+ I checked further and it comes down to the smw_di_blob table, all the dupped values are there. I managed to "hide" this issue by forcing "$usedistinct = true" on SMW_SQLStore3_Readers.php line 285, but of course this is merely hiding the symptom instead of fixing the issue. That's when asking and showing properties. Besides when asking for some conditions even if we're not asking for a particular property (just articles matching the condition), sometimes we're getting the same articles in the result twice. I think it's because of dupes on smw_object_ids but I'm not sure. When this happens it doesn't happen for all the queries run, just for some of them. A full refresh (SMW_refreshData -ftpv, then another -v) usually fixes the issue, but not always. And even sometimes, a full refresh may fix SOME of the issues, but not all of them.
Not sure if it can be related to our other bug documented there https://bugzilla.wikimedia.org/show_bug.cgi?id=48706. We can have both on the same wiki, or one but not the other.
I'm using SMW 1.9a (master) and I've seen this repeatedly as well. I haven't dug into the queries as deep as Vicente though. I will comment that as I've upgraded from master this seems to happen less and less. In fact, the repeating properties with 0 instances seem to be gone for me in the current codebase. I used to see these all over in Special:WantedProperties. However, I do still see regular duplicate values in properties. I cannot identify any rhyme or reason for when hey appear. I will comment that they seem to happen less frequently in the current master than they did before. :-\
I guess it is safe to confirm this bug and setting it to NEW. I believe that this is quite a critical bug which needs love.
The frequency of this issue has seemed to decline with recent versions of the master branch. I used to see it all the time, and while I can still find it it is less. I've tried SMW_rebuild in a bunch of different ways and that doesn't make it go away. If I could help with debugging on this let me know.
Please let me know if I can help trouble shooting somehow. I see this a lot atm.
In order to analysis the issue, one would need a scenario that can be replicated that would allow to identify the circumstance of the issue. Doing [1] (as mentioned in the first thread) does not create the issue, which means without a reliable, replicable use case there isn't much one can analysis and since the store doesn't provide any unit tests (to have a yard stick as to how and what is expected of some of the methods used to store and retrieve data) it is difficult to make predication on what is causing this behaviour as it pre-exists SMW 1.9. For example: - Does it only occur for a specific property type - Does it only happen during a specific storage scenario - Are any other extensions involved that would cause the issue etc. [1] {{#set:Version=1.0}}, {{#show:Page|?Version}}
For me, this ONLY happens for preperties of type Text. (However, once it happens, changing type of the property won't help. You're stuck with the double values). How the property is set does not seem to have any impact (standard []-notion, #set, #set_internal, it's all the same).
*** Bug 44204 has been marked as a duplicate of this bug. ***
http://dota2.gamepedia.com/index.php?title=Special:Ask&q=0 http://dota2.gamepedia.com/api.php?action=ask&query=1 In the former case the result "Anti-Mage" is duplicated. In the latter case the result "Anti-Mage" is not duplicated. Is that relevant or helpful? MW 1.19.9 SMW 1.8.0.5 PHP 5.4.19-1+debphp.org~precise+3 (fpm-fcgi) MySQL 5.5.31-55
Hm. Apparently I failed to escape brackets properly. Each query should be for Category:Escape in double square brackets, not "0" and "1".
What makes this bug a bit nasty for 1.9 is that the new release does not afaik offer Store2 as an alternative. I know about the e-mails about this bug [1] so it does not need to be repeated here. This comment is just a "comment". [1] http://wikimedia.7.x6.nabble.com/SMW-1-9-entering-beta-stage-tt5016184.html#none
Just noting that I recently had this bug show up when querying values that are set by Semantic Extra Special Properties. Specifically the page has multiple Creation date values. https://wikiapiary.com/wiki/Special:Browse/Liquipedia_Hearthstone_Wiki I only share this as I figure these properties are created via a different code path. Although, it still points the finger at SQL Store 3.
