Last modified: 2013-10-19 11:19:03 UTC
https://commons.wikimedia.org/wiki/File:Visual_Editor_Presentation_2013.webm 41 min 45 s, 1,920 × 1,080 (936.69 MB) For big files such as this (in size, resolution and quality), it would make sense to have more than 1 thread doing hte transcode In this cases on the wikimedia boxes they have 16 threads (2 x 4 core + HT). Which means a lot of the box is sitting idle, meaning the transcodes take longer, rather than being sped up by using more of the available capacity
problem with using multiple threads is that the memory use per encode multiplies too. whats the amount of ram available in the box, we might be able to use 2 threads per encode now, the first set of encoding boxes did not allow that.
If y'all have to buy more RAM, please buy more RAM. :) Leaving CPUs idle while transcoding a 40m-minute 1080p video makes no sense at all.
reedy@tmh1001:~$ cat /proc/meminfo MemTotal: 16419920 kB so 16GB See also http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&s=by+name&c=Video%2520scalers%2520eqiad&tab=m&vn= Looks like we're not even using half of the memory of the cluster. CPUs are mostly idle. Should be plenty of room (slot wise) to upgrade RAM. We might even have some spare at EQIAD. I'll drop a procurement ticket and then it can go through the process. Any suggestions what we should go for? 64GB in each? I don't see any reason to also upgrade the PMTPA boxes at the moment.
I suspect we could at least double that one to a 2. The jobs are done synchronously, right? ie one at a time (per job runner)? In which case, we don't care so much how much memory is used at a time
Thinking about it, the simplest way to fix this bug might be to just make a global. It can then be set for WMF usage as appropriate - even (number of cores - 1) if the boxes only do one at a time. Code to count cpus or cores in php gets a little scary. This of course still leaves the potential memory upgrade, and setting of the config as both non issues in the WMF domain
right now we run 10 concurrent jobs(puppet manifests/role/applicationserver.pp) with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot of this is mmapped this is working out. Setting threads to 2 we will need to set $wgTranscodeBackgroundMemoryLimit to 4GB, leaving it at 10 concurrent jobs we end up with 40GB ram. Since there are not so many videos to encode in parallel we could reduce the number of concurrent jobs to 5.
Change 84493 had a related patch set uploaded by J: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493
(In reply to comment #6) > right now we run 10 concurrent jobs(puppet > manifests/role/applicationserver.pp) > with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot > of this is mmapped this is working out. Setting threads to 2 we will need to > set $wgTranscodeBackgroundMemoryLimit to 4GB, leaving it at 10 concurrent > jobs > we end up with 40GB ram. Since there are not so many videos to encode in > parallel we could reduce the number of concurrent jobs to 5. That seems a more sensible option to me - fewer jobs running quicker Any suggestion on ram amounts? Should we double it or quadruple it?
Change 84539 had a related patch set uploaded by J: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539
Change 84493 merged by jenkins-bot: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493
Change 84775 had a related patch set uploaded by Reedy: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775
Change 84775 merged by Akosiaris: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775
Change 84539 merged by jenkins-bot: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539
So is more work needed here (if so: what?) or can this be closed as RESOLVED FIXED?
