Last modified: 2014-09-04 10:52:16 UTC
We unintentionally changed the output of wbgetentities. Previously it said "name" and since the last deployment it says "title". We need to revert this change. [23:25:49] <Lydia_WMDE> can we do one thing at a time please guys? [23:26:00] <Lydia_WMDE> let's figure out the wbgetentities issue now [23:26:04] <Lydia_WMDE> because that looks serious [23:26:10] <Lydia_WMDE> then we can move on to the next one [23:26:29] <Amir1> thanks Lydia_WMDE [23:26:34] <sjoerddebruin> multichill: Oh ja, zag je trouwens dat https://www.wikidata.org/wiki/Property:P1456 is aangemaakt? Ik vroeg me af of we die botmatig kunnen toevoegen. [23:26:49] <Amir1> the old system was like entity['claims'] was a list [23:27:37] <Lydia_WMDE> Tobi_WMDE_SWE: still around by any chance? [23:27:40] <sjoerddebruin> Is there a change regarding size in bytes? Some edits are bigger then ␜before. [23:27:50] <Amir1> and each entity was a dictionary with ['refs':[], 'q':[], 'm':[blah,20,something, "string"] [23:27:51] <Lydia_WMDE> sjoerddebruin: new serialization [23:27:57] <sjoerddebruin> Lydia_WMDE: ah. :P [23:28:08] <sjoerddebruin> 1500 bytes for a sitelink is pretty big. [23:28:09] <aude> that's odd [23:28:29] <Amir1> and now the entity['claims'] is a dictionary instead of list [23:28:49] <multichill> sjoerddebruin: Volkomen gemist [23:28:55] <Amir1> which is like entity['claims']['P12'] [23:29:04] <Amir1> and it's a dictionary [23:29:05] <sjoerddebruin> multichill: Er is ook nog geen overlegpagina. :/ [23:29:20] <Lydia_WMDE> aude: any idea? [23:29:39] <Amir1> Lydia_WMDE: and even removing is making it bigger [23:29:54] <Amir1> see last edits: https://www.wikidata.org/wiki/Special:Contributions/Dexbot [23:30:05] <Lydia_WMDE> let's figure out editentity first [23:30:09] <grrrit-wm1> (CR) Siebrand: Handle redirects gracefully on special pages. (1 comment) [extensions/Wikibase] - https://gerrit.wikimedia.org/r/156298 (https://bugzilla.wikimedia.org/70034) (owner: Daniel Kinzler) [23:30:47] <Amir1> 21:06, 26 August 2014 (diff | hist) . . (+2,628) . . David Lea (Q431792) (Removed claim: date of death (P570): 2 November 1937) !!!! [23:30:55] <Amir1> k [23:31:01] <aude> http://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q17&format=json ? [23:31:05] <aude> looks normal to me [23:31:13] <sjoerddebruin> There is clearly something wrong. [23:32:13] <Lydia_WMDE> Amir1: ^ [23:32:36] <Amir1> checking [23:32:42] <aude> Amir1: id? [23:33:00] <Amir1> id of what? [23:34:02] <Lydia_WMDE> an item that gives you an error [23:34:20] <Amir1> I give this on any item [23:34:39] <Amir1> let me find a comparing system [23:35:14] <Amir1> aude Lydia_WMDE: http://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q1700&format=json [23:35:25] <Amir1> in this entity['claims'] is different [23:35:33] <Amir1> than it was before [23:36:00] <magioladitis> Amir1: I am off to bed. Please clear the badges thing and write a comment at https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/Dexbot_2#Discussion [23:36:15] <magioladitis> Lydia_WMDE: Please check this link too. Thanks [23:36:21] <Amir1> magioladitis: sure :) [23:36:27] <magioladitis> bye! [23:36:33] <Amir1> bye [23:36:44] <Amir1> it was a list [23:36:45] <Lydia_WMDE> Amir1: indeed. something is different [23:37:06] -*- Lydia_WMDE looks when q17 was last edited [23:37:34] <Lydia_WMDE> before the serialisation switch [23:37:40] <Amir1> I use Q1700 because is smaller [23:37:46] <Lydia_WMDE> 1700 as well though [23:37:48] <Lydia_WMDE> wth [23:38:07] <Amir1> and sitelinks are different [23:38:24] <Amir1> before it was items['links']['dewiki']['name'] [23:38:37] <Amir1> now it is items['links'][ [23:38:50] <Amir1> 'dewiki']['title'] [23:39:04] <Amir1> and sitelinks was a list now it's a dictionary [23:39:18] <Amir1> (I'm not sure about the last one) [23:39:32] <Lydia_WMDE> ok [23:41:33] <aude> hmmm [23:41:45] <Amir1> it was a dictionary but it was like entity['items']['dewiki'] = {'name':"foo","badges":[]} [23:43:33] <JohnLewis> legoktm: on a small note - legobot seems CVN blacklisted [23:46:36] <Amir1> aude, Lydia_WMDE: What do you think? [23:47:17] <Lydia_WMDE> Amir1: i am letting katie investigate. there's not much i can do atm. worst case we'll all have to look at this first thing tomorrow morning with the rest of the team [23:48:09] <Amir1> okay, thank you. I'm off to bed [23:48:27] <aude> i don't know that it's something quick to fix [23:48:28] <Amir1> please e-mail whenever it got fixed or any update
to clarify: the accidental switch to "title" is in the representation of sitelinks.
Seems like there is some confusion here: the API output didn't change, we didn't touch the serializers for the API output at all (this is scheduled for the next deployment, or the one after, but should not change anything). We suspect that the pywikipediabot framework is using MediaWiki's export interface for retrieving entities - and the output of this *did* change from the old internal format to the canonical external format, as announced and expected.
per Daniel's comment.
pywikibot does not use export, it uses wbgetentities I wrote that parts.
Wbgetentities still is not ok, it is in progress?
There is no progress because the problem is unclear. As far as I can tell from the code, the behavior of the API did not change at all. Please provide a link to an API call that now returns "changed" data, incompatible with pywikibot, and provide an example of how that data would have looked like before. Ideally, let me know what version of the software would have produced that output.
In the IRC log, I see a reference to ['refs':[], 'q':[], 'm':[blah,20,something, "string"]. This is the internal representation formerly used in the database, in XML dumps and by Special:Export. This was however never used by wbgetentities, as far as I know.
Note that things like action=query&query=revisions would have returned the internal serialization in the past, but uses the external serialization now, just like Special:Export.
Katie just pointed me to https://gerrit.wikimedia.org/r/#/c/156732/ - so it seems to be fixed in pywikibot now, though in a rather horrible way.
(In reply to Daniel Kinzler from comment #6) > There is no progress because the problem is unclear. As far as I can tell > from the code, the behavior of the API did not change at all. > > Please provide a link to an API call that now returns "changed" data, > incompatible with pywikibot, and provide an example of how that data would > have looked like before. Ideally, let me know what version of the software > would have produced that output. Hi Daniel, there is in pywikibot functino, which in returned json finds key "entity[0]" (key entity and its first occurence), in former version it is ok and this key exists, now it is not. there was "entity" at position of "type" etc. {"type":"item","id":"Q45930", "labels": {"eo":{"language":"eo","value":"Lom"},...}, "descriptions": {"it":{"language":"it","value":"comune ceco"},...}, "aliases": {"cs":[{"language":"cs","value":"Lom u Blatn\u00e9"}]}, "claims": {"P373":[{"mainsnak": {"snaktype":"value", "property":"P373", "hash":"ef4e6e979c4ddabc6aa3f6c4a2914714ddb86d7b", "datavalue":{"value":"Lom (Strakonice District)", "type":"string"} },...,... "snaks-order":["P402"]}]}]}, "sitelinks": {"eowiki":{"site":"eowiki","title":"Lom (distrikto Strakonice)","badges":[]},..., "badges":[]}} }
And finally, i am pulled new version of pywikibot and bot is nearly ok (about ten changes in my code and bot is nearly ok). So, for information about "wbsetreferences" - query.py from pywikibot needs to updates line 108 with adding "wbsetferences" to "post". If it was a mistake in pywikibot ... ehm, i think, it was mistake in API code. Maybe.
(In reply to Jiří Sedláček from comment #11) > And finally, i am pulled new version of pywikibot and bot is nearly ok > (about ten changes in my code and bot is nearly ok). > > So, for information about "wbsetreferences" - query.py from pywikibot needs > to updates line 108 with adding "wbsetferences" to "post". > > If it was a mistake in pywikibot ... ehm, i think, it was mistake in API > code. Maybe. Yes because I wrote a wrapper to return old system instead of new one and it got merged yesterday: https://gerrit.wikimedia.org/r/156732
Amir, i know. Thanks, but it's not completely clear, but its near ok, thanks! J.
