Last modified: 2014-05-27 15:24:19 UTC
ie tin deploys to X hosts, and then Y hosts can use the "nearest"/"quickest" host to grab from Like scap does..
What if we just continue down the path of making scap (and trebuchet) fast enough to not worry about a sync-file/sync-dir at all? iow: always use scap. It reduces the complexity and code paths needed to be tested with deployment changes and encourages us to make scap as fast as possible (already around 3 minutes for small/no-op changes and no i10n update).
A no-op scap is taking 3 minutes vs <10 seconds for sync-file today. I think it's going to be a while until we have speed parity between scap and sync-*. If we could efficiently determine if l10n updates were needed or not it would probably mostly eliminate the difference. In beta (much smaller host set), a no-op scap is taking just under 1 minute. If we could get that down to 20-30 seconds we could probably convince people that killing sync-* is reasonable. Converting sync-* to python is the next big thing on my TODO list, so after we do the straight port we can think about either doing this or waiting until we get a pre-computed diff transfer mechanism (aka Trebuchet) figured out.
Change 135513 had a related patch set uploaded by BryanDavis: Convert sync-dir and sync-file to python https://gerrit.wikimedia.org/r/135513
Change 135513 merged by jenkins-bot: Convert sync-dir and sync-file to python https://gerrit.wikimedia.org/r/135513
Use of rsync proxies for sync-dir and sync-file implemented in I5ab7b2fba2f68d3d9219947ef8ab8ac6d0160d8e.
