Last modified: 2014-10-24 15:00:05 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T33331, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 31331 - JavascriptFFS shouldn't be building JSON manually
JavascriptFFS shouldn't be building JSON manually
Status: PATCH_TO_REVIEW
Product: MediaWiki extensions
Classification: Unclassified
Translate (Other open bugs)
unspecified
All All
: Low normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-10-03 19:15 UTC by Niklas Laxström
Modified: 2014-10-24 15:00 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Niklas Laxström 2011-10-03 19:15:47 UTC
Roan comments:
Line 508-513 (concatenate separated strings): this doesn't account for single quotes
Why don't you just use real JSON here instead of fragile hacks?

I assume one reason would be formatting, but there should be code snippets which do that too.
Comment 1 Krinkle 2012-05-14 08:33:57 UTC
In the backend FormatJson can be used, including an option to prettify whitespace.

On the client side use the 'jquery.json' module to do this (aliases to the built-in browser method JSON.stringify when available, implements a JSON encoder otherwise).
Comment 2 Niklas Laxström 2012-05-14 08:35:44 UTC
(In reply to comment #1)
> In the backend FormatJson can be used, including an option to prettify
> whitespace.

What option? I haven't seen any readily available pretty-printer for JSON yet.
Comment 3 Krinkle 2012-05-14 08:37:48 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > In the backend FormatJson can be used, including an option to prettify
> > whitespace.
> 
> What option? I haven't seen any readily available pretty-printer for JSON yet.

Check out how API outputs format=jsonfm. There is an boolean parameter to FormatJson::encode that triggers pretty-print (the function documentation is vague about this but it has been this way for a while now).
Comment 4 Niklas Laxström 2012-05-14 10:35:07 UTC
I tried that. It generates diffs like:


-var I18n = {
-       on_leave_page: "Вашыя зьмены могуць быць страчаныя",
+var I18n =
+{
+       "on_leave_page": "\u0412\u0430\u0448\u044b\u044f \u0437\u044c\u043c\u0435\u043d\u044b \u043c\u043e\u0433\u0443\u0446\u044c \u0431\u044b\u0446\u044c \u0441\u0442\u0440\u0430\u0447\u0430\u043d\u044b\
 };

Would be nice to avoid the asciization, any ideas?
Comment 5 Krinkle 2012-05-15 09:07:21 UTC
It depends. There are options for that in the latest PHP's native json_encode, namely JSON_UNESCAPED_UNICODE.

But I'm not sure if the Services_JSON fallback has such option.

Either way, is there a problem with the escaped unicode characters? It is perfectly valid JSON. As far as I know it shouldn't cause any trouble.

It is also valid JavaScript, and outputs fine when put into a DOM element or other output. The JSON decoders also support it (they have to, since this encoding is part of the JSON specification).
Comment 6 Krinkle 2012-05-15 09:10:45 UTC
Looking at JavascriptFFS, I take it the primary use is not related to transferring data from the client to the server. i.e. it is not run on a normal request, is that correct?

If so, and when it is only run from the command-line, you could bump the phpversion requirement for this script to 5.4 and use json_encode directly like this:

echo json_encode( $data, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE );

that'll make it look really pretty.

See also: http://php.net/json_encode
Comment 7 Niklas Laxström 2012-05-15 10:42:44 UTC
The primary purpose is storage of translations in version control system. Overt escaping is not needed and causes increase in file size, and makes it hard for occasional readers (if any).

json_encode looks nice, however translatewiki.net server is still running on PHP 5.3.2 for time being.
Comment 8 Kevin Israel (PleaseStand) 2013-03-11 10:10:30 UTC
Krinkle, Niklas: See Gerrit change #50140.
Comment 9 Andre Klapper 2013-03-25 14:11:28 UTC
(In reply to comment #8)
> Krinkle, Niklas: See Gerrit change #50140.

...which is "Combine JavaScript and JSON encoding logic"
Comment 10 Nemo 2014-03-06 12:10:58 UTC
PleaseStand, I've added this bug to <https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#Extensive_and_robust_localisation_file_format_coverage>. Would you be able to help a student by reviewing contributions such as a patch for this bug? If yes please add yourself there as co-mentor, then we'll be able to mark it featured.
Comment 11 Nemo 2014-10-12 08:40:49 UTC
(In reply to Niklas Laxström from comment #0)
> Roan comments:
> Line 508-513 (concatenate separated strings)

That's around line 100 of ffs/JavaScriptFFS.php now.

> this doesn't account for
> single quotes
> Why don't you just use real JSON here instead of fragile hacks?
Comment 12 Nemo 2014-10-13 13:26:54 UTC
From #mediawiki

2014-10-12 19.57 < Nemo_bis> The bug is about using standard functions to reduce mistakes in JSON production. PleaseStand linked a function in core which you'll probably need to use (with the utf8 option). Comment 0 also mention a specific bug the current implementation might have (single quotes).
Comment 13 Alisha Jain 2014-10-13 20:01:27 UTC
(In reply to Nemo from comment #12)
> From #mediawiki
> 
> 2014-10-12 19.57 < Nemo_bis> The bug is about using standard functions to
> reduce mistakes in JSON production. PleaseStand linked a function in core
> which you'll probably need to use (with the utf8 option). Comment 0 also
> mention a specific bug the current implementation might have (single quotes).

I want to know what type of string is stored in $data and what will be the output of $messages[keys].
It will be very helpful if you can give an example of the data being entered and the corresponding output.
Comment 14 Nemo 2014-10-13 20:06:19 UTC
(In reply to Alisha Jain from comment #13)
> It will be very helpful if you can give an example of the data being entered
> and the corresponding output.

I'm not sure that's worth the effort for this bug: you should probably just look at how JSON is produced in the last gerrit link above.
However, if you want real data you can set up a Translate wiki with some JSON-based projects, copying configuration from https://git.wikimedia.org/tree/translatewiki.git
Comment 15 Alisha Jain 2014-10-14 07:13:32 UTC
I want to know how can I use this file, where can I see output?
Comment 16 Nemo 2014-10-14 07:53:19 UTC
(In reply to Alisha Jain from comment #15)
> I want to know how can I use this file, where can I see output?

I guess "set up a Translate wiki" wasn't clear enough, I added more explicit pointers in https://www.mediawiki.org/?diff=1224226 . Getting support for setting up a test environment, however, is best done at [[mw:Extension talk:Translate]] than an individual bugzilla report.
Comment 17 Gerrit Notification Bot 2014-10-16 12:43:15 UTC
Change 166990 had a related patch set uploaded by Prianka:
ffs/JavaScriptFFS.php Replaced unnecessary use of str_replace

https://gerrit.wikimedia.org/r/166990
Comment 18 Gerrit Notification Bot 2014-10-16 16:41:28 UTC
Change 167018 had a related patch set uploaded by Alishajain:
JSON prouction using standard function

https://gerrit.wikimedia.org/r/167018

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links