Last modified: 2013-05-28 10:44:41 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T45269, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 43269 - Some Extension:ShortUrl memcached keys exceed key length limit
Some Extension:ShortUrl memcached keys exceed key length limit
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
ShortUrl (Other open bugs)
master
All All
: Normal normal (vote)
: ---
Assigned To: Yuvi Panda
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-12-19 18:56 UTC by Aaron Schulz
Modified: 2013-05-28 10:44 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Aaron Schulz 2012-12-19 18:56:43 UTC
2012-12-19 18:54:25 srv195 tawikisource: Memcached error for key "tawikisource:shorturls:title:திருவிவிலியம்/இணைத்_திருமுறை_நூல்கள்/பாரூக்கு_(எரேமியாவின்_மடல்)/அதிகாரங்கள்_5_முதல்_6_வரை" on server ":": A BAD KEY WAS PROVIDED/CHARACTERS OUT OF RANGE
Comment 1 Niklas Laxström 2012-12-19 18:59:56 UTC
What's the maximum length?
Comment 3 Yuvi Panda 2012-12-19 19:06:34 UTC
It hits the database everytime that is queried, so not as performant as it should be. Plus of course, not the 'right' thing to do.
Comment 4 Ori Livneh 2012-12-19 19:10:31 UTC
(In reply to comment #1)
> What's the maximum length?

The maximum length is 250 *bytes*:
https://github.com/memcached/memcached/blob/master/doc/protocol.txt

(In reply to comment #2)
> What's the practical influence of this? Asking as
> http://ta.wikisource.org/wiki/திருவிவிலியம்/இணைத்_திருமுறை_நூல்கள்/
> பாரூக்கு_(எரேமியாவின்_மடல்)/அதிகாரங்கள்_5_முதல்_6_வரை
> still shows and offers http://ta.wikisource.org/s/30q as a ShortURL which
> works...

You could have overlapping keys if the difference between them exists beyond the 250 character mark.

There's a simple solution to this, and it's to hash the title before using it in a key. sha1($title) will give you a 40-byte string.
Comment 5 Niklas Laxström 2012-12-20 13:27:50 UTC
According to quick google search pecl/memcached does hashing automatically if the key is too long, so just a small change to our php/memcached layer then?
Comment 6 Dereckson 2012-12-25 00:52:32 UTC
Removing 'easy'.
Comment 7 Aaron Schulz 2013-04-09 07:09:55 UTC
Is anyone working on this?
Comment 8 Yuvi Panda 2013-04-10 12:55:03 UTC
I suppose this would be a core change?
Comment 9 Aaron Schulz 2013-04-10 16:41:27 UTC
(In reply to comment #8)
> I suppose this would be a core change?

I would just change the extension now and worry about core later.
Comment 10 Yuvi Panda 2013-04-10 16:51:03 UTC
Is it okay to invalidate current keys? If not, I'll just hash keys that are longer than the limit, assuming that is a hard limit. This will make sure that current keys are not invalidated. 

If it is okay to invalidate I'll just hash everything.
Comment 11 Nemo 2013-04-10 16:59:07 UTC
(In reply to comment #10)
> Is it okay to invalidate current keys? 

If this means breaking links, I'd say: absolutely not okay, ever.
Comment 12 Yuvi Panda 2013-04-10 17:01:09 UTC
(In reply to comment #11)
> If this means breaking links, I'd say: absolutely not okay, ever.

No - the links are stored in the database. Invalidating keys will only mean a slight increase in the memcached miss rate for a period of time.
Comment 13 Gerrit Notification Bot 2013-04-15 17:34:59 UTC
Related URL: https://gerrit.wikimedia.org/r/59165 (Gerrit Change I386794c46cbd8e203cbc45115a40d8f399e0939b)
Comment 14 Gerrit Notification Bot 2013-04-15 17:47:23 UTC
https://gerrit.wikimedia.org/r/59165 (Gerrit Change I386794c46cbd8e203cbc45115a40d8f399e0939b) | change APPROVED and MERGED [by jenkins-bot]
Comment 15 Andre Klapper 2013-05-14 09:25:07 UTC
Gerrit patch got merged, can this bug report get closed as RESOLVED FIXED or is there anything else to do?
Comment 16 Niklas Laxström 2013-05-15 06:31:05 UTC
I guess so, though I would have hoped answer for comment #5
Comment 17 Ori Livneh 2013-05-28 10:44:41 UTC
(In reply to comment #5)
> According to quick google search pecl/memcached does hashing automatically if
> the key is too long, so just a small change to our php/memcached layer then?

It doesn't hash automatically. By default it just silently fails to set key. See https://gist.github.com/atdt/fd88e2e4a0c31884907f.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links