Last modified: 2013-03-15 16:57:45 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T38731, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 36731 - Cache dir listings to avoid excessive fstat calls
Cache dir listings to avoid excessive fstat calls
Status: UNCONFIRMED
Product: MediaWiki
Classification: Unclassified
ResourceLoader (Other open bugs)
unspecified
All All
: Low enhancement (vote)
: ---
Assigned To: Nobody - You can work on this!
: performance
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-10 08:41 UTC by Joe Julian
Modified: 2013-03-15 16:57 UTC (History)
3 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Joe Julian 2012-05-10 08:41:20 UTC
I noticed this in config/index.php:getLanguageList()

The current method of opendir/readdir causes an excessive number of fstat calls. With network based filesystems (glusterfs specifically) this causes a lot of unnecessary delay.

I propose caching the results of those lookups in session variables to reduce that impact, for example:

function getLanguageList() {
        global $wgLanguageNames, $IP;
        if( isset( $_SESSION['getLanguageListcodes'] ) )
                return $_SESSION['getLanguageListcodes']
        if( !isset( $wgLanguageNames ) ) {
                require_once( "$IP/languages/Names.php" );
        }

        $codes = array();

        $d = opendir( "../languages/messages" );
        /* In case we are called from the root directory */
        if (!$d)
                $d = opendir( "languages/messages");
        while( false !== ($f = readdir( $d ) ) ) {
                $m = array();
                if( preg_match( '/Messages([A-Z][a-z_]+)\.php$/', $f, $m ) ) {
                        $code = str_replace( '_', '-', strtolower( $m[1] ) );
                        if( isset( $wgLanguageNames[$code] ) ) {
                                $name = $code . ' - ' . $wgLanguageNames[$code];
                        } else {
                                $name = $code;
                        }
                        $codes[$code] = $name;
                }
        }
        closedir( $d );
        ksort( $codes );
        $_SESSION['getLanguageListcodes'] = $codes
        return $codes;
}
Comment 1 Joe Julian 2012-05-10 08:45:30 UTC
Obviously an untested example... been doing a lot of python recently. Missed the semicolons.
Comment 2 Krinkle 2013-03-06 05:57:54 UTC
Is this (if so, how) ResourceLoader related?

I know there are stat calls inside ResourceLoader, but afaik those are fairly well cached. The example shown in comment looks like Installer::findExtensions.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links