Skip to content

Commit 8e9f479

Browse files
AtlasPilotPuppypwendell
authored andcommitted
SPARK-1990: added compatibility for python 2.6 for ssh_read command
https://issues.apache.org/jira/browse/SPARK-1990 There were some posts on the lists that spark-ec2 does not work with Python 2.6. In addition, we should check the Python version at the top of the script and exit if it's too old Author: Anant <[email protected]> Closes #941 from anantasty/SPARK-1990 and squashes the following commits: 4ca441d [Anant] Implmented check_optput withinthe module to work with python 2.6 c6ed85c [Anant] added compatibility for python 2.6 for ssh_read command Conflicts: ec2/spark_ec2.py
1 parent 706e38f commit 8e9f479

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

ec2/spark_ec2.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -629,9 +629,23 @@ def ssh(host, opts, command):
629629
time.sleep(30)
630630
tries = tries + 1
631631

632+
# Backported from Python 2.7 for compatiblity with 2.6 (See SPARK-1990)
633+
def _check_output(*popenargs, **kwargs):
634+
if 'stdout' in kwargs:
635+
raise ValueError('stdout argument not allowed, it will be overridden.')
636+
process = subprocess.Popen(stdout=PIPE, *popenargs, **kwargs)
637+
output, unused_err = process.communicate()
638+
retcode = process.poll()
639+
if retcode:
640+
cmd = kwargs.get("args")
641+
if cmd is None:
642+
cmd = popenargs[0]
643+
raise subprocess.CalledProcessError(retcode, cmd, output=output)
644+
return output
645+
632646

633647
def ssh_read(host, opts, command):
634-
return subprocess.check_output(
648+
return _check_output(
635649
ssh_command(opts) + ['%s@%s' % (opts.user, host), stringify_command(command)])
636650

637651

0 commit comments

Comments
 (0)