Last modified: 2014-08-28 10:29:25 UTC
We unintentionally broke the API with yesterday's deployment. wbgetentities output is grouped by property now. We need to return to the ungrouped version. [23:25:49] <Lydia_WMDE> can we do one thing at a time please guys? [23:26:00] <Lydia_WMDE> let's figure out the wbgetentities issue now [23:26:04] <Lydia_WMDE> because that looks serious [23:26:10] <Lydia_WMDE> then we can move on to the next one [23:26:29] <Amir1> thanks Lydia_WMDE [23:26:34] <sjoerddebruin> multichill: Oh ja, zag je trouwens dat https://www.wikidata.org/wiki/Property:P1456 is aangemaakt? Ik vroeg me af of we die botmatig kunnen toevoegen. [23:26:49] <Amir1> the old system was like entity['claims'] was a list [23:27:37] <Lydia_WMDE> Tobi_WMDE_SWE: still around by any chance? [23:27:40] <sjoerddebruin> Is there a change regarding size in bytes? Some edits are bigger then ␜before. [23:27:50] <Amir1> and each entity was a dictionary with ['refs':[], 'q':[], 'm':[blah,20,something, "string"] [23:27:51] <Lydia_WMDE> sjoerddebruin: new serialization [23:27:57] <sjoerddebruin> Lydia_WMDE: ah. :P [23:28:08] <sjoerddebruin> 1500 bytes for a sitelink is pretty big. [23:28:09] <aude> that's odd [23:28:29] <Amir1> and now the entity['claims'] is a dictionary instead of list [23:28:49] <multichill> sjoerddebruin: Volkomen gemist [23:28:55] <Amir1> which is like entity['claims']['P12'] [23:29:04] <Amir1> and it's a dictionary [23:29:05] <sjoerddebruin> multichill: Er is ook nog geen overlegpagina. :/ [23:29:20] <Lydia_WMDE> aude: any idea? [23:29:39] <Amir1> Lydia_WMDE: and even removing is making it bigger [23:29:54] <Amir1> see last edits: https://www.wikidata.org/wiki/Special:Contributions/Dexbot [23:30:05] <Lydia_WMDE> let's figure out editentity first [23:30:09] <grrrit-wm1> (CR) Siebrand: Handle redirects gracefully on special pages. (1 comment) [extensions/Wikibase] - https://gerrit.wikimedia.org/r/156298 (https://bugzilla.wikimedia.org/70034) (owner: Daniel Kinzler) [23:30:47] <Amir1> 21:06, 26 August 2014 (diff | hist) . . (+2,628) . . David Lea (Q431792) (Removed claim: date of death (P570): 2 November 1937) !!!! [23:30:55] <Amir1> k [23:31:01] <aude> http://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q17&format=json ? [23:31:05] <aude> looks normal to me [23:31:13] <sjoerddebruin> There is clearly something wrong. [23:32:13] <Lydia_WMDE> Amir1: ^ [23:32:36] <Amir1> checking [23:32:42] <aude> Amir1: id? [23:33:00] <Amir1> id of what? [23:34:02] <Lydia_WMDE> an item that gives you an error [23:34:20] <Amir1> I give this on any item [23:34:39] <Amir1> let me find a comparing system [23:35:14] <Amir1> aude Lydia_WMDE: http://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q1700&format=json [23:35:25] <Amir1> in this entity['claims'] is different [23:35:33] <Amir1> than it was before [23:36:00] <magioladitis> Amir1: I am off to bed. Please clear the badges thing and write a comment at https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/Dexbot_2#Discussion [23:36:15] <magioladitis> Lydia_WMDE: Please check this link too. Thanks [23:36:21] <Amir1> magioladitis: sure :) [23:36:27] <magioladitis> bye! [23:36:33] <Amir1> bye [23:36:44] <Amir1> it was a list [23:36:45] <Lydia_WMDE> Amir1: indeed. something is different [23:37:06] -*- Lydia_WMDE looks when q17 was last edited [23:37:34] <Lydia_WMDE> before the serialisation switch [23:37:40] <Amir1> I use Q1700 because is smaller [23:37:46] <Lydia_WMDE> 1700 as well though [23:37:48] <Lydia_WMDE> wth [23:38:07] <Amir1> and sitelinks are different [23:38:24] <Amir1> before it was items['links']['dewiki']['name'] [23:38:37] <Amir1> now it is items['links'][ [23:38:50] <Amir1> 'dewiki']['title'] [23:39:04] <Amir1> and sitelinks was a list now it's a dictionary [23:39:18] <Amir1> (I'm not sure about the last one) [23:39:32] <Lydia_WMDE> ok [23:41:33] <aude> hmmm [23:41:45] <Amir1> it was a dictionary but it was like entity['items']['dewiki'] = {'name':"foo","badges":[]} [23:43:33] <JohnLewis> legoktm: on a small note - legobot seems CVN blacklisted [23:46:36] <Amir1> aude, Lydia_WMDE: What do you think? [23:47:17] <Lydia_WMDE> Amir1: i am letting katie investigate. there's not much i can do atm. worst case we'll all have to look at this first thing tomorrow morning with the rest of the team [23:48:09] <Amir1> okay, thank you. I'm off to bed [23:48:27] <aude> i don't know that it's something quick to fix [23:48:28] <Amir1> please e-mail whenever it got fixed or any update
To clarify: the API was *supposed* to return grouped output for a long time, we introduced the option "ungroupedlist" for disabling this in january. But apparently, this never worked, and client code is relying on the ungrouped form to be returned by default. I propose we depercate the ungroupedlist option, make ungrouped the default, and introduce a groupedlist option for enabling grouping of claims (and qualifiers, and snaks in references) by property.
Change 156505 had a related patch set uploaded by Daniel Kinzler: Make ungrouped output the default for getentities https://gerrit.wikimedia.org/r/156505
Change 156505 abandoned by Daniel Kinzler: Make ungrouped output the default for getentities Reason: never mind! https://gerrit.wikimedia.org/r/156505
Seems like there is some confusion here: the API output didn't change, we didn't touch the serializers for the API output at all (this is scheduled for the next deployment, or the one after, but should not change anything). We suspect that the pywikipediabot framework is using MediaWiki's export interface for retrieving entities - and the output of this *did* change from the old internal format to the canonical external format, as announced and expected.
per Daniel's comment.
