Last modified: 2014-02-28 22:41:35 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T42678, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 40678 - Languages are bundled unnecessarily in HTML
Languages are bundled unnecessarily in HTML
Status: RESOLVED FIXED
Product: MobileFrontend
Classification: Unclassified
Feature requests (Other open bugs)
unspecified
All All
: Normal enhancement
: ---
Assigned To: Nobody - You can work on this!
: design
Depends on: 59624
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-01 23:21 UTC by Max Semenik
Modified: 2014-02-28 22:41 UTC (History)
16 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Max Semenik 2012-10-01 23:21:01 UTC
Interwikis for [[testwiki:San Francisco]] take almost 13K even though this feature is not heavily used(?). Ideally, we should load them with JS only when user needs them while people without JS will see just a link to a special page with interwikis for a given page.
Comment 1 Jon 2012-12-03 22:39:53 UTC
So if I understand correctly:

* We add the text "This article is available in <a>X languages</a>" where the <a> tag links to a page listing all the different language versions.
** What should this link to? If a Special page is it is an issue if this doesn't cache?
* In javascript  when clicked an ajax request is triggered that requests languages
** what happens whilst the request is in process - do we see a loader?
*** is a delay acceptable?
* If an article has no alternative languages do we show the text or not?

Note that in beta mode, we have another way to look at this - if languages are treated as a section, then we could retrieve them on initial page load and only add to the DOM when the section is toggled open / button is clicked.
Comment 2 Jon 2013-04-09 21:12:01 UTC
So for this to work we can use javascript but need a fallback for browsers not running javascript for whatever reason.

Ideally we'd need a page that lists all the languages for a given article 
e.g.
San Francisco/Languages
or Special:AvailableLanguages/San Francisco

The javascript code already exists in beta.
Comment 3 Jon 2013-04-18 01:01:54 UTC
In beta/alpha the language overlay this is particularly a problem as it currently also requests via the api language versions of the page.

Ideally we should replace the html for the language section in beta with a link 'View available languages'.

When clicked without javascript/slow connection the 'Read in another language button' should open
Special:PageLanguages?title=San Francisco which simply lists all the available languages that the given article in title parameter is in

When javascript kicks in it opens the overlay via data from the api.
Comment 4 Michelle Grover 2013-04-27 01:04:05 UTC
Assigned Mingle Ticket https://mingle.corp.wikimedia.org/projects/mobile/cards/703
Comment 5 Jon 2013-05-09 18:29:29 UTC
The additional javascript call now doesn't apply. Marking as enhancement as although solving this will reduce the payload for all alpha, beta and stable users it is not urgent
Comment 6 Jon 2013-06-24 19:52:59 UTC
Generation of language code is heavily tied to the parser. I fear that it will be difficult to create a special page that does this without some upstream changes.
Comment 7 Jon 2013-06-24 19:59:55 UTC
An alternative way of doing this might be to introduce a new action value and not think of this a special page - action=languages that allows us to share the parser code more easily. I still think a special page would be a better solution though.
Comment 8 Jon 2013-07-24 19:05:22 UTC
This sounds like it will be useful for zero as well as they can more easily overwrite urls..
Comment 9 Theopolisme 2013-12-22 17:20:57 UTC
@Jon, there is already prop=langlinks...

Example: https://en.wikipedia.org/w/api.php?action=query&prop=langlinks&format=json&lllimit=500&titles=Apple

It seems like a solution might be to create Special:Languages/<pagename> that just pretty prints the contents of that API query, unless I misunderstand you.
Comment 10 Bingle 2013-12-22 17:25:53 UTC
Prioritization and scheduling of this bug is tracked on Mingle card https://wikimedia.mingle.thoughtworks.com/projects/mobile/cards/1573
Comment 11 Jon 2013-12-22 19:12:23 UTC
@Theopolisme exactly - the solution would strip out the language links currently printed at the bottom of the page and replace them with
<a href="wiki/Special:Languages/Article Title" class="button">read in another language</a> (JavaScript would hijack this to render the overlay)

The reader experience would be no different to how it is currently, however the page size would dramatically decrease.

Any work towards realising this would be much appreciated (for example the creation of the special page)
Comment 12 Geoffrey Mon 2013-12-22 19:13:58 UTC
Hello, I'm from GCI, I'm going to try this task out.
I know very little web-related stuff, but I figured I want to challenge myself and learn some new things (that's the point of GCI, right?).
Comment 13 Jon 2013-12-23 17:54:41 UTC
If you know very little web related stuff this might be a very tricky one to start with.. although it will certainly challenge you!
Comment 14 Theopolisme 2013-12-24 17:37:10 UTC
@Jon, a few things.

I've created Special:Languages/<pagename> which returns both the languages and the language variants available for any given page.

As far as the JavaScript overlay, what I've written right now just loads Special:Languages for the given page and then parses it to get the language links. This is more efficient and allows us to remove all of the getPageLanguages JavaScript stuff... who isn't for removing code? ;)

But now for a question. Right now I've implemented all of this stuff directly on the stable version...but reflecting on what I've done, would it make more sense to do it as a beta feature initially (given that a lot of code is being touched)? I don't know what the procedures are exactly (and how you make that determination), so please let me know how you'd like me to proceed.

Once you let me know I'll move the code around as necessary and get the initial patch uploaded. 

Thanks!
Comment 15 Jon 2013-12-24 18:07:14 UTC
Great stuff Theo!

Definitely as a beta feature first. If you push your changes I can help guide you with how to slot it all together.

We particularly have to be careful here as it changes the resulting HTML for all users which will have cache implications!
Comment 16 Theopolisme 2013-12-24 22:22:25 UTC
Hi!

Still having some trouble getting my interwiki/langlinks all set up. I've added rows to the interwiki table for each language and added some interwiki links to a test page. When viewing the test page, the interwiki links all display correctly in the sidebar.

However, when I go to /api.php?action=query&prop=langlinks&llurl=true&lllimit=max&titles=Test4 the language links don't appear. Same with /api.php?action=query&prop=iwlinks&titles=Test4

Is there some magical final step to get the langlinks api to work? (I've already run the update.php script, to no avail...)

Thanks :)
Comment 17 Yuri Astrakhan 2013-12-24 22:52:38 UTC
I'm a bit confused - why not implement this purely in javascript to call the api? Why do we need a Special:Languages page?
Comment 18 Jon 2013-12-24 23:18:05 UTC
as a fallback
1) to be good web citizens
2) to support users without JavaScript (yes these do exist)
3) to provide access to this information when bit servers are down
4) to support users who run browsers that we do not support in ResourceLoader
Comment 19 Yuri Astrakhan 2013-12-25 00:23:49 UTC
Sorry, makes perfect sense - we use the same strategy in zero with hijacking and using an api call. Must have not been fully awake :)
Comment 20 Theopolisme 2013-12-25 15:14:51 UTC
@Yuri, any insights on the langlinks API? Right now I've written the code but I can't test it :p
Comment 21 Yuri Astrakhan 2013-12-25 15:25:19 UTC
(In reply to comment #20)
> @Yuri, any insights on the langlinks API? Right now I've written the code
> but I
> can't test it :p

Strange - API gets its info from langlinks table directly, so there shouldn't be any magic going on. Use the $wgDebugToolbar=true to see what SQL query is actually made against that table, and see if that data is actually there.
Comment 22 Theopolisme 2013-12-25 16:35:31 UTC
So it looks like the langlinks table isn't propagated, but the interwiki table is.

The API query for langlinks:

SELECT  ll_from,ll_lang,ll_title  FROM `langlinks`   WHERE ll_from = '27'  ORDER BY ll_lang LIMIT 5001

This smells strongly of some maintenance script that needs to be run.

* goes to sift through /maintenance *

UPDATE: It was! Through trial and error I ran clearInterwikiCache.php and refreshLinks.php and then the api worked properly. Yay :)
Comment 23 Gerrit Notification Bot 2013-12-26 06:03:59 UTC
Change 103739 had a related patch set uploaded by Theopolisme:
Add language (interwiki) links on demand

https://gerrit.wikimedia.org/r/103739
Comment 24 Gerrit Notification Bot 2013-12-26 21:19:10 UTC
Change 103849 had a related patch set uploaded by Theopolisme:
Add Special:Languages

https://gerrit.wikimedia.org/r/103849
Comment 25 Gerrit Notification Bot 2013-12-27 01:26:19 UTC
Change 103849 merged by jenkins-bot:
Add Special:MobileLanguages

https://gerrit.wikimedia.org/r/103849
Comment 26 Gerrit Notification Bot 2013-12-27 06:41:53 UTC
Change 103739 merged by jenkins-bot:
Use Special:MobileLanguages in beta/alpha

https://gerrit.wikimedia.org/r/103739
Comment 27 Jon 2013-12-27 07:11:24 UTC
Thanks to Theo this is now in beta which is super exciting :D

Two problems to solve before pushing to stable
1) variants do not show - see bug 58988
2) The language button always shows now - this is a question for design - what should happen if the language button is clicked and no languages are available? (ccing Jared)
Comment 28 Yuri Astrakhan 2013-12-27 07:26:24 UTC
Zero note: we replace each langlink with a magical "special:zero?redirect=xxx" where the xxx is the langlink. Later, the zero dynamically catches all such redirects and shows an js overlay. If this info is now coming from the mobile special page and a langlinks api, we need to rethink the interception strategy.
Comment 29 Theopolisme 2013-12-27 07:30:01 UTC
Are you sure the language button always shows? In my tests, it only appears when the page has langlinks...
Comment 30 Jon 2013-12-27 07:32:06 UTC
Could you now simply disable the JavaScript in Zero and rewrite the Special:MobileLanguages URL?

In alpha yes: all searches result in changing the page context and loading the page via JavaScript - in which case the language button will be created regardless. Not a big deal but a slight change in behaviour that we need to think through. Worth noting that now it is only when you click on the language icon that you know if a page has languages or not (previously this was determined beforehand)
Comment 31 Jon 2013-12-30 21:56:16 UTC
Now languages are lazily loaded. This is great!

There's only one odd thing with existing behaviour:

If you edit a page with no languages after saving you will  you see a button which opens an empty language list. We don't know how many languages are available until that button is pressed...

In this situation we should either print a line instead saying "No languages available." or show the existing language in that list. Jared thoughts?


Once we wrap that up after testing this in the wild for a bit we should push the changes to stable!
Comment 32 Jon 2014-01-03 23:39:50 UTC
I guess we should keep things simple and not render the language button when languages are not available (as it currently does)
Opened a bug and added it as a dependency.
Comment 33 Jon 2014-01-08 00:21:31 UTC
Now  the dependency is cleared this code can be pushed to stable. We should ensure it has been in production for at least a week on English Wikipedia and there have been new performance issues but if someone wants to work on that patch they can feel free!
Comment 34 dr0ptp4kt 2014-02-12 21:00:42 UTC
https://gerrit.wikimedia.org/r/#/c/112619/ was deployed to cover the Wikipedia Zero case - the user will just be sent to the Special: page and the dynamic loading will be disabled. This still has the overall perf advantage spread across the corpus of articles.
Comment 35 Gerrit Notification Bot 2014-02-28 22:40:23 UTC
Change 115978 had a related patch set uploaded by Jdlrobson:
Card 703: Promote lazy languages to stable

https://gerrit.wikimedia.org/r/115978
Comment 36 Jon 2014-02-28 22:41:02 UTC
Yeyyy! Thanks to everyone specifically Theo for getting this done! Big bug killed :D
Comment 37 Gerrit Notification Bot 2014-02-28 22:41:35 UTC
Change 115978 merged by jenkins-bot:
Card 703: Promote lazy languages to stable

https://gerrit.wikimedia.org/r/115978

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links