Last modified: 2014-02-12 14:14:13 UTC
Exactly the same behavior: https://jira.toolserver.org/browse/TS-1479
Wow, that was a hard one to track down. I spent literally an hour suspecting Perl's exec. It's essentially a bug (or a *very* odd feature) in qsub's "-b" option that is set by jsub: | $ /usr/bin/qsub $PWD/test.py '1=2 3' '$4' => "['/var/spool/gridengine/execd/tools-exec-01/job_scripts/143663', '1=2 3', '$4']" | $ /usr/bin/qsub -b n $PWD/test.py '1=2 3' '$4' => "['/var/spool/gridengine/execd/tools-exec-02/job_scripts/143662', '1=2 3', '$4']" | $ /usr/bin/qsub -b y $PWD/test.py '1=2 3' '$4' => "['/home/scfc/tmp/test.py', '1=2', '3']" I have to think more about whether "-b y" is a good or bad idea (for example for binary commands), or if we should double-escape arguments to the user script, or what other options we have, and maybe ask on the mailing list. Thanks for the report.
(In reply to comment #1) > Wow, that was a hard one to track down. I spent literally an hour suspecting > Perl's exec. > > It's essentially a bug (or a *very* odd feature) in qsub's "-b" option that > is > set by jsub: > > | $ /usr/bin/qsub $PWD/test.py '1=2 3' '$4' > > => "['/var/spool/gridengine/execd/tools-exec-01/job_scripts/143663', '1=2 3', > '$4']" > > | $ /usr/bin/qsub -b n $PWD/test.py '1=2 3' '$4' > > => "['/var/spool/gridengine/execd/tools-exec-02/job_scripts/143662', '1=2 3', > '$4']" > > | $ /usr/bin/qsub -b y $PWD/test.py '1=2 3' '$4' > > => "['/home/scfc/tmp/test.py', '1=2', '3']" Note that in this command it's more buggy than that JIRA bug: $4 seems missing here. > > I have to think more about whether "-b y" is a good or bad idea (for example > for binary commands), or if we should double-escape arguments to the user > script, or what other options we have, and maybe ask on the mailing list. > > Thanks for the report.
(In reply to comment #2) > [...] > Note that in this command it's more buggy than that JIRA bug: $4 seems > missing > here. > [...] I wouldn't rate bugginess, they both don't work as they should :-). And just in case someone from Toolserver stumbles onto this bug: The Toolserver bug is different to this ("qcronsub" doesn't use "-b y") and my fix there works (or would work if it was applied).
btw My use case may require -b y (running a PHP script with php command).
This particular bug is actually triggered by qsub evaluating arguments through a shell when it really shouldn't; the bug may or may not be apparent depending on the amount of quoting before the command line reaches it -- relying on a specific behaviour is bound to be troublesome. It's not possible to have jsub just blindly add a level of quoting since that may simply end up overquoting the arguments. The maintainers can add extra quoting when appropriate, though that makes invocation more brittle than it should. The "proper" solution is to not rely on the number of shell evaluations arguments will go through before ending up on the exec node; either by providing any arguments through a script (and submitting /that/) or by passing arguments through a file. Marking "WONTFIX" since the bug is in qsub and there is no plausibility that upstream would even consider a change that would break every user of gridengine and introduce incompatibilities with Oracle's version.
*** Bug 61140 has been marked as a duplicate of this bug. ***
