1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
|
From b20bbe5917008a5588f6109b66a6becaf090c3d9 Mon Sep 17 00:00:00 2001
From: Sandro Tosi <morph@debian.org>
Date: Thu, 8 Oct 2015 10:06:03 -0700
Subject: This patch removes the default password from pp code, requiring users
to
explicitly write one; it updates the documentation along the source code
Patch-Name: 10_remove_default_password.patch
---
doc/ppdoc.html | 12 +++++-------
pp.py | 8 +++-----
2 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/doc/ppdoc.html b/doc/ppdoc.html
index c210f70..8ecfb8f 100644
--- a/doc/ppdoc.html
+++ b/doc/ppdoc.html
@@ -158,13 +158,11 @@ to the number of processors in the system
and list of ppservers to connect with :</p><p><strong> job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>) </strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong> f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong> f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong> f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br> </p><p> ...etc...<br></p><p>6) Retrieve the results as needed:</p><p><strong> r1 = f1()</strong></p><p><strong> r2 = f2()</strong></p><p><strong> r3 = f3() </strong> </p><p> ...etc...</p><p> To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p><hr><h1 id="ADVANCEDCLUSTERS"> Advanced guide, clusters </h1> <p><em><strong>On the nodes</strong></em> </p><p>1) Start parallel python execution server on all your remote computational nodes (listen to a given port 35000,<br> and local network interface only, accept only connections which know correct secret):</p><p><strong> node-1> ./ppserver.py -p 35000 -i 192.168.0.101 -s "mysecret"<br></strong></p><p><strong> node-2> ./ppserver.py -p 35000 -i 192.168.0.102</strong><strong> -s "mysecret"</strong></p><p><strong> node-3> ./ppserver.py -p 35000 -i 192.168.0.103</strong><strong> -s "mysecret"</strong></p><p><em><strong>On the client</strong></em> <br></p> <p>2) Import pp module:</p><p><strong> import pp</strong></p><p>3) Create a list of all the nodes in your cluster (computers where you've run ppserver.py) </p><p><strong> ppservers=("node-1:35000", "node-2:</strong><strong>35000</strong><strong>", "node-3:</strong><strong>35000</strong><strong>")</strong><br></p><p>4)
Start pp execution server with the number of workers set
to the number of processors in the system,
- <br>list of ppservers to connect with and secret key to authorize the connection:</p><p><strong> job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>, secret="</strong><strong>mysecret</strong><strong>") </strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong> f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong> f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong> f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br> </p><p> ...etc...<br></p><p>6) Retrieve the results as needed:</p><p><strong> r1 = f1()</strong></p><p><strong> r2 = f2()</strong></p><p><strong> r3 = f3() </strong> </p><p> ...etc...</p><p> 7) Print the execution statistics:<br></p><p><strong> job_server.print_stats()</strong></p><p>To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p><hr><h1 id="COMMANDLINE"> Command line options, ppserver.py </h1> <pre>Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]<br> Options:<br> -h : this help message<br> -d : debug<br> -a : enable auto-discovery service<br> -i interface : interface to listen<br> -b broadcast : broadcast address for auto-discovery service<br> -p port : port to listen<br> -w nworkers : number of workers to start<br> -s secret : secret for authentication<br> -t seconds : timeout to exit if no connections with clients exist<br> -k seconds : socket timeout in seconds <br> -P pid_file : file to write PID to <br></pre><hr><h1 id="COMMANDLINE"> Security and secret key<a name="SECURITY" title="SECURITY"></a></h1><p> Due
- to the security concerns it is highly recommended to run ppserver.py
-with an non-trivial secret key (-s command line argument) which should
-be paired with the matching <em>secret</em> keyword of PP Server class constructor. Since PP 1.5.3 it is possible to set secret key by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong>
- which should be located in the user home directory (please make this
-file readable and writable only by user). The key set in .pythonrc.py
-could be overridden by command line argument (for ppserver.py) and <em>secret</em> keyword (for PP Server class constructor). </p><p> </p><hr><h1 id="COMMANDLINE"> ppserver.py stats and PID file example<br></h1><p>To print job execution statistics for ppserver.py send a SIGUSR1 signal to its main process.<br>For instance on UNIX platform following commands will start a server and print its stats:<br><strong>ppserver.py -P /tmp/ppserver.pid </strong></p><p><strong>kill -s SIGUSR1 `cat /tmp/ppserver.pid`</strong><br></p>
+ <br>list of ppservers to connect with and secret key to authorize the connection:</p><p><strong> job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>, secret="</strong><strong>mysecret</strong><strong>") </strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong> f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong> f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong> f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br> </p><p> ...etc...<br></p><p>6) Retrieve the results as needed:</p><p><strong> r1 = f1()</strong></p><p><strong> r2 = f2()</strong></p><p><strong> r3 = f3() </strong> </p><p> ...etc...</p><p> 7) Print the execution statistics:<br></p><p><strong> job_server.print_stats()</strong></p><p>To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p><hr><h1 id="COMMANDLINE"> Command line options, ppserver.py </h1> <pre>Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]<br> Options:<br> -h : this help message<br> -d : debug<br> -a : enable auto-discovery service<br> -i interface : interface to listen<br> -b broadcast : broadcast address for auto-discovery service<br> -p port : port to listen<br> -w nworkers : number of workers to start<br> -s secret : secret for authentication<br> -t seconds : timeout to exit if no connections with clients exist<br> -k seconds : socket timeout in seconds <br> -P pid_file : file to write PID to <br></pre><hr><h1 id="COMMANDLINE"> Security and secret key
+
+<a name="SECURITY" title="SECURITY"></a></h1><p> Due to the security concerns, in Debian we decided to disable default password authentication, and require to run <strong>ppserver</strong> with a non-trivial secret key (<strong>-s</strong> command line argument) which should be paired with the matching <em>secret</em> keyword of PP Server class constructor. An alternative way to set a secret key is by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong> which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in <strong>.pythonrc.py</strong> could be overridden by command line argument (for <strong>ppserver</strong>) and <em>secret</em> keyword (for PP Server class constructor). Note that passing the password on the command line allows every user to see it (e.g. using <strong>ps(1)</strong>), and that running it on an open/untrusted network can be a security problem as traf fic, including the password, is not encrypted.</p>
+
+<p> </p><hr><h1 id="COMMANDLINE"> ppserver.py stats and PID file example<br></h1><p>To print job execution statistics for ppserver.py send a SIGUSR1 signal to its main process.<br>For instance on UNIX platform following commands will start a server and print its stats:<br><strong>ppserver.py -P /tmp/ppserver.pid </strong></p><p><strong>kill -s SIGUSR1 `cat /tmp/ppserver.pid`</strong><br></p>
</td>
</tr>
diff --git a/pp.py b/pp.py
index 936c68d..e3f4023 100644
--- a/pp.py
+++ b/pp.py
@@ -275,7 +275,6 @@ class Server(object):
"""
default_port = 60000
- default_secret = "epo20pdosl;dksldkmm"
def __init__(self, ncpus="autodetect", ppservers=(), secret=None,
restart=False, proto=2, socket_timeout=3600):
@@ -286,9 +285,8 @@ class Server(object):
the number of processors in the system
ppservers - list of active parallel python execution servers
to connect with
- secret - passphrase for network connections, if omitted a default
- passphrase will be used. It's highly recommended to use a
- custom passphrase for all network connections.
+ secret - passphrase for network connections; it can be set via
+ command-line or configuration file
restart - whether to restart worker process after each task completion
proto - protocol number for pickle module
socket_timeout - socket timeout in seconds which is also the maximum
@@ -371,7 +369,7 @@ class Server(object):
raise TypeError("secret must be of a string type")
self.secret = str(secret)
else:
- self.secret = Server.default_secret
+ raise ValueError("secret must be set using command-line option or configuration file")
self.__connect()
self.__creation_time = time.time()
self.logger.info("pp local server started with %d workers"
|
