File: programming.html

package info (click to toggle)
python-medusa 0.5.4%2Bclean-1
links: PTS
area: main
in suites: etch, etch-m68k
size: 540 kB
ctags: 1,009
sloc: python: 5,489; makefile: 11
file content (646 lines) | stat: -rw-r--r-- 27,854 bytes
parent folder | download | duplicates (9)
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>Programming in Python with Medusa and the Async Sockets Library</title>
  </head>

  <body>
    <h1>Programming in Python with Medusa and the Async Sockets Library</h1>

    <h2>Introduction</h2>
    <h3>Why Asynchronous?</h3>

    <p>
      There are only two ways to have a program on a single processor do
      'more than one thing at a time'.  Multi-threaded programming is
      the simplest and most popular way to do it, but there is another
      very different technique, that lets you have nearly all the
      advantages of multi-threading, without actually using multiple
      threads.  It's really only practical if your program is <i>I/O
      bound</i> (I/O is the principle bottleneck).  If your program is
      CPU bound, then pre-emptive scheduled threads are probably what
      you really need.  Network servers are rarely CPU-bound, however.
    </p>
      
    <p>
      If your operating system supports the <code>select()</code>
      system call in its I/O library (and nearly all do), then you can
      use it to juggle multiple communication channels at once; doing
      other work while your I/O is taking place in the "background".
      Although this strategy can seem strange and complex (especially
      at first), it is in many ways easier to understand and control
      than multi-threaded programming.  The library documented here
      solves many of the difficult problems for you, making the task
      of building sophisticated high-performance network servers and
      clients a snap.
    </p>
    
    <h3>Select-based multiplexing in the real world</h3>

    <p>
      Several well-known Web servers (and other programs) are written using
      exactly this technique:
      the <a href="http://www.acme.com/software/thttpd/">thttpd</a>
      and <a href="http://www.zeus.co.uk/">Zeus</a>,
      and <a href="http://squid.nlanr.net/">Squid Internet Object Cache</a> servers
      are excellent examples..
      <a href="http://www.isc.org/inn.html">The InterNet News server (INN)</a> used
      this technique for several years before the web exploded.
    </p>

    <p>
      An interesting web server comparison chart is available at the
      <a href="http://www.acme.com/software/thttpd/benchmarks.html">thttpd web site</a>
    <p>
      
    <h3>Variations on a Theme: poll() and WaitForMultipleObjects</h3>
    <p>
      Of similar (but better) design is the <code>poll()</code> system
      call.  The main advantage of <code>poll()</code> (for our
      purposes) is that it does not used fixed-size file-descriptor
      tables, and is thus more easily scalable than
      <code>select()</code>.  <code>poll()</code> is only recently becoming
      widely available, so you need to check for availability on your particular
      operating system.
    </p>

    <p>
      In the Windows world, the Win32 API provides a bewildering array
      of features for multiplexing.  Although slightly different in
      semantics, the combination of Event objects and the
      <code>WaitForMultipleObjects()</code> interface gives
      essentially the same power as <code>select()</code> on Unix.  A
      version of this library specific to Win32 has not been written
      yet, mostly because Win32 also provides <code>select()</code>
      (at least for sockets).  If such an interface were written, it
      would have the advantage of allowing us to multiplex on other
      objects types, like named pipes and files.
    </p>

    <h3>select()</h3>

    <p>
      Here's what <code>select()</code> does: you pass in a set of
      file descriptors, in effect asking the operating system, "let me
      know when anything happens to any of these descriptors".  (A
      <i>descriptor</i> is simply a numeric handle used by the
      operating system to keep track of a file, socket, pipe, or other
      I/O object.  It is usually an index into a system table of some
      kind).  You can also use a timeout, so that if <i>nothing</i>
      happens in the allotted period, <code>select()</code> will return
      control to your program.
    </p>

    <p>
      <code>select()</code> takes three <code>fd_set</code> arguments;
      one for each of the following possible states/events:
      readability, writability, and exceptional conditions.  The last set
      is less useful than it sounds; in the context of TCP/IP it refers
      to the presence of out-of-band (OOB) data.  OOB is a relatively unportable
      and poorly used feature that you can (and should) ignore unless you really
      need it.
    </p>

    <p>
      So that leaves only two types of events to build our programs
      around; <i>read</i> events and <i>write</i> events.  As it turns
      out, this is actually enough to get by with, because other types
      of events can be implied by the sequencing of these two.  It
      also keeps the low-level interface as simple as possible -
      always a good thing in my book.
    </p>
    
    <h3>The polling loop</h3>
    <p>
      Now that you know what <code>select()</code> does, you're ready
      for the final piece of the puzzle: the main polling loop.  This
      is nothing more than a simple while loop that continually calls
      <code>select()</code> with a timeout (I usually use a 30-second
      timeout).  Such a program will use virtually no CPU if your
      server is idle; it spends most of its time letting the operating
      system do the waiting for it.  This is much more efficient than a
      <a href="http://wombat.doc.ic.ac.uk/foldoc/foldoc.cgi?query=busy-wait">busy-wait</a>
      loop.
    </p>
    <p> Here is a pseudo-code example of a polling loop:
    <pre>
while (any_descriptors_left):
  events = select (descriptors, timeout)
  for event in events:
    handle_event (event)
</pre>
    <p>
      If you take a look at the code used by the library, it looks
      very similar to this. (see the file <code>asyncore.py</code>,
      the functions poll() and loop()).  Now, on to the magic that must
      take place to handle the events...
    </p>
    
    <h2>The Code</h2>
    <h3>Blocking vs. Non-Blocking</h3>
    <p>
      File descriptors can be in either blocking or non-blocking mode.
      A descriptor in blocking mode will stop (or 'block') your entire
      program until the requested event takes place.  For example, if
      you ask to read 64 bytes from a descriptor attached to a socket
      which is ultimately connected to a modem deep in the backwaters
      of the Internet, you may wait a while for those 64 bytes.
    </p>
    <p>
      If you put the descriptor in non-blocking mode, then one of two
      things might happen: if the data is sitting in a local buffer,
      it will be returned to you immediately; otherwise you will get
      back a code (usually <code>EWOULDBLOCK</code>) telling you that
      the read is in progress, and you should check back later to see
      if it's done.
    <p>

    <h3>sockets vs. other kinds of descriptors</h3>

    <p>
      Although most of our discussion will be about TCP/IP sockets, on
      Unix you can use <code>select()</code> to multiplex other kinds
      of communications objects, like pipes and ttys.  (Unfortunately,
      select() cannot be used to do non-blocking file I/O.  Please
      correct me if you have information to the contrary!)
    </p>

    <h3>The socket_map</h3>
    <p>
      We use a global dictionary (<code>asyncore.socket_map</code>) to
      keep track of all the active socket objects.  The keys for this
      dictionary are the objects themselves.  Nothing is stored in the
      value slot.  Each time through the loop, this dictionary is scanned.
      Each object is asked which <code>fd_sets</code> it wants to be in.
      These sets are then passed on to <code>select()</code>.
    </p>

    <h3>asyncore.dispatcher</h3>
    <p>
      The first class we'll introduce you to is the
      <code>dispatcher</code> class.  This is a thin wrapper around a
      low-level socket object.  We have attached a few methods for
      event-handling to it.  Otherwise, it can be treated as a normal
      non-blocking socket object.
    </p>
    <p>
      The direct interface between the select loop and the socket object
      are the <code>handle_read_event</code> and <code>handle_write_event</code>
      methods.  These are called whenever an object 'fires' that event.
    </p>
    <p>
      The firing of these low-level events can tell us whether certain
      higher-level events have taken place, depending on the timing
      and state of the connection.  For example, if we have asked for
      a socket to connect to another host, we know that the connection
      has been made when the socket fires a write event (at this point
      you know that you may write to it with the expectation of
      success).
      <br>
      The implied events are
    <ul>
      <li>handle_connect.
	<br>implied by a write event.
      <li>handle_close
	<br>implied by a read event with no data available.
      <li>handle_accept
	<br>implied by a read event on a listening socket.
    </ul>
    </p>
    <p>
      Thus, the set of user-level events is a little larger than simply
      <code>readable</code> and <code>writeable</code>.  The full set of
      events your code may handle are:
    <ul>
      <li>handle_read
      <li>handle_write
      <li>handle_expt (OOB data)
      <li>handle_connect
      <li>handle_close
      <li>handle_accept
    </ul>

    <p>
      A quick terminology note: In order to distinguish between
      low-level socket objects and those based on the async library
      classes, I call these higher-level objects <i>channels</i>.

    </p>
    <h3>Enough Gibberish, let's write some code</h3>
    <p>
      Ok, that's enough abstract talk.  Let's do something useful and
      concrete with this stuff.  We'll write a simple HTTP client that
      demonstrates how easy it is to build a powerful tool in only a few
      lines of code.
    </p>
    
<pre>
<font color="800000"># -*- Mode: Python; tab-width: 4 -*-</font>

<font color="808000">import</font> asyncore
<font color="808000">import</font> socket
<font color="808000">import</font> string

<font color="808000">class</font><font color="000080"> http_client</font> (asyncore.dispatcher):

    <font color="808000">def</font><font color="000080"> __init__</font> (self, host, path):
        asyncore.dispatcher.__init__ (self)
        self.path = path
        self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
        self.connect ((host, 80))

    <font color="808000">def</font><font color="000080"> handle_connect</font> (self):
        self.send (<font color="008000">'GET %s HTTP/1.0\r\n\r\n'</font> % self.path)

    <font color="808000">def</font><font color="000080"> handle_read</font> (self):
        data = self.recv (8192)
        <font color="808000">print</font> data

    <font color="808000">def</font><font color="000080"> handle_write</font> (self):
        <font color="808000">pass</font>

<font color="808000">if</font> __name__ == <font color="008000">'__main__'</font>:
    <font color="808000">import</font> sys
    <font color="808000">import</font> urlparse
    <font color="808000">for</font> url <font color="808000">in</font> sys.argv[1:]:
        parts = urlparse.urlparse (url)
        <font color="808000">if</font> parts[0] != <font color="008000">'http'</font>:
            <font color="808000">raise</font> ValueError, <font color="008000">"HTTP URL's only, please"</font>
        <font color="808000">else</font>:
            host = parts[1]
            path = parts[2]
            http_client (host, path)
    asyncore.loop()

</pre>


<!-- Thanks to Just van Rossum for PyFontify.py! -->
</pre>

    <p>
      HTTP is (in theory, at least) a very simple protocol.  You connect to the
      web server, send the string <code>"GET /some/path HTTP/1.0"</code>, and the
      server will send a short header, followed by the file you asked for.  It will
      then close the connection.
    <p>
      We have defined a single new class, <code>http_client</code>, derived
      from the abstract class <code>asyncore.dispatcher</code>.  There are three
      event handlers defined.<ul>
      <li><code>handle_connect</code>
	<br>Once we have made the connection, we send the request string.
      <li><code>handle_read</code>
	<br>As the server sends data back to us, we simply print it out.
      <li><code>handle_write</code>
	<br>Ignore this for the moment, I'm brushing over a technical detail
	we'll clean up in a moment.
    </ul>

    <p>Go ahead and run this demo - giving a single URL as an argument, like this:
    <p><font color="006000"><code>$ python asynhttp.py http://www.nightmare.com/</code></font>
    <p>You should see something like this:
    <p>
      
<pre>
[rushing@gnome demo]$ python asynhttp.py http://www.nightmare.com/
log: adding channel &lt;http_client  at 80ef3e8&gt;
HTTP/1.0 200 OK
Server: Medusa/3.19
Content-Type: text/html
Content-Length: 1649
Last-Modified: Sun, 26 Jul 1998 23:57:51 GMT
Date: Sat, 16 Jan 1999 13:04:30 GMT

[... body of the file ...]

log: unhandled close event
log: closing channel 4:&lt;http_client connected at 80ef3e8&gt;

</pre>

    <p>
      The 'log' messages are there to help, they are useful when
      debugging but you will want to disable them later.  The first log message
      tells you that a new <code>http_client</code> object has been added to the
      socket map.  At the end, you'll notice there's a warning that you haven't
      bothered to handle the <code>close</code> event.  No big deal, for now.

    <p>
      Now at this point we haven't seen anything <i>revolutionary</i>, but that's
      because we've only looked at one URL.  Go ahead and add a few other URL's
      to the argument list; as many as you like - and make sure they're on different
      hosts...

    <p>
      <i>Now</i> you begin to see why <code>select()</code> is so powerful.  Depending
      on your operating system (and its configuration), <code>select()</code> can be
      fed hundreds, or even thousands of descriptors like this.  (I've recently tested
      <code>select()</code> on a FreeBSD box with over 10,000 descriptors).

    <p>
      A really good way to understand <code>select()</code> is to put a print statement
      into the asyncore.poll() function:
<pre>
        [...]
        (r,w,e) = select.select (r,w,e, timeout)
        print '---'
        print 'read', r
        print 'write', w
        [...]
</pre>      

    <p>
      Each time through the loop you will see which channels have fired
      which events.  If you haven't skipped ahead, you'll also notice a pointless
      barrage of events, with all your http_client objects in the 'writable' set.
      This is because we were a bit lazy earlier; sweeping some ugliness under
      the rug.  Let's fix that now.

    <h3>Buffered Output</h3>
    <p>
      In our <code>handle_connect</code>, we cheated a bit by calling
      <code>send</code> without examining its return code.  In truth,
      since we are using a non-blocking socket, it's (theoretically)
      possible that our data didn't get sent.  To do this <i>correctly</i>,
      we actually need to set up a buffer of outgoing data, and then send
      as much of the buffer as we can whenever we see a <code>write</code>
      event:

<pre>

<font color="808000">class</font><font color="000080"> http_client</font> (asyncore.dispatcher):

    <font color="808000">def</font><font color="000080"> __init__</font> (self, host, path):
        asyncore.dispatcher.__init__ (self)
        self.path = path
        self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
        self.connect ((host, 80))
        self.buffer = <font color="008000">'GET %s HTTP/1.0\r\n\r\n'</font> % self.path

    <font color="808000">def</font><font color="000080"> handle_connect</font> (self):
        <font color="808000">pass</font>

    <font color="808000">def</font><font color="000080"> handle_read</font> (self):
        data = self.recv (8192)
        <font color="808000">print</font> data

    <font color="808000">def</font><font color="000080"> writable</font> (self):
        <font color="808000">return</font> (len(self.buffer) &gt; 0)

    <font color="808000">def</font><font color="000080"> handle_write</font> (self):
        sent = self.send (self.buffer)
        self.buffer = self.buffer[sent:]
</pre>   

    <p>
      The <code>handle_connect</code> method no longer assumes it can
      send its request string successfully.  We move its work over to
      <code>handle_write</code>; which trims <code>self.buffer</code>
      as pieces of it are sent succesfully.
      
    <p>
      We also introduce the <code>writable</code> method.  Each time
      through the loop, the set of sockets is scanned, the
      <code>readable</code> and <code>writable</code> methods of each
      object are called to see if are interested in those events.  The
      default methods simply return 1, indicating that by default all
      channels will be in both sets.  In this case, however, we are only
      interested in writing as long as we have something to write.  So
      we override this method, making its behavior dependent on the length
      of <code>self.buffer</code>.

    <p>
      If you try the client now (with the print statements in
      <code>asyncore.poll()</code>), you'll see that
      <code>select</code> is firing more efficiently.
      
    <h3>asynchat.py</h3>
    <p>
      The dispatcher class is useful, but somewhat limited in
      capability.  As you might guess, managing input and output
      buffers manually can get complex, especially if you're working
      with a protocol more complicated than HTTP.

    <p>
      The <code>async_chat</code> class does a lot of the heavy
      lifting for you.  It automatically handles the buffering of both
      input and output, and provides a "line terminator" facility that
      partitions an input stream into logical lines for you.  It is
      also carefully designed to support <i>pipelining</i> - a nice
      feature that we'll explain later.

    <p>
      There are four new methods to introduce:
    <ul>

      <li><code>set_terminator (self, &lt;eol-string&gt;)</code>
	<br>Set the string used to identify <i>end-of-line</i>.  For most
	Internet protocols, this is the string <code>\r\n</code>, that is;
	a carriage return followed by a line feed.  To turn off input scanning,
	use <code>None</code>

      <li><code>collect_incoming_data (self, data)</code>
	<br>Called whenever data is available from
	a socket.  Usually, your implementation will accumulate this
	data into a buffer of some kind.
      
      <li><code>found_terminator (self)</code>
	<br>Called whenever an end-of-line marker has been seen.  Typically
	your code will process and clear the input buffer.
	
      <li><code>push (data)</code>
	<br>This is a buffered version of <code>send</code>.  It will place
	the data in an outgoing buffer.

    </ul>
    <p>
      These methods build on the underlying capabilities of
      <code>dispatcher</code> by providing implementations of
      <code>handle_read</code> <code>handle_write</code>, etc...
      <code>handle_read</code> collects data into an input buffer, which
      is continually scanned for the terminator string.  Data in between
      terminators is feed to your <code>collect_incoming_data</code> method.
      
    <p>
      The implementation of <code>handle_write</code> and <code>writable</code>
      examine an outgoing-data queue, and automatically send data whenever
      possible.

    <h3>A Proxy Server</h3>
      In order to demonstrate the <code>async_chat</code> class, we will
      put together a simple proxy server.  A proxy server combines a server
      and a client together, in effect sitting between the real server and
      client.  You can use this to monitor or debug protocol traffic.

<pre>

<font color="800000"># -*- Mode: Python; tab-width: 4 -*-</font>

<font color="808000">import</font> asynchat
<font color="808000">import</font> asyncore
<font color="808000">import</font> socket
<font color="808000">import</font> string

<font color="808000">class</font><font color="000080"> proxy_server</font> (asyncore.dispatcher):
    
    <font color="808000">def</font><font color="000080"> __init__</font> (self, host, port):
        asyncore.dispatcher.__init__ (self)
        self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
        self.set_reuse_addr()
        self.there = (host, port)
        here = (<font color="008000">''</font>, port + 8000)
        self.bind (here)
        self.listen (5)

    <font color="808000">def</font><font color="000080"> handle_accept</font> (self):
        proxy_receiver (self, self.accept())

<font color="808000">class</font><font color="000080"> proxy_sender</font> (asynchat.async_chat):

    <font color="808000">def</font><font color="000080"> __init__</font> (self, receiver, address):
        asynchat.async_chat.__init__ (self)
        self.receiver = receiver
        self.set_terminator (None)
        self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
        self.buffer = <font color="008000">''</font>
        self.set_terminator (<font color="008000">'\n'</font>)
        self.connect (address)

    <font color="808000">def</font><font color="000080"> handle_connect</font> (self):
        <font color="808000">print</font> <font color="008000">'Connected'</font>

    <font color="808000">def</font><font color="000080"> collect_incoming_data</font> (self, data):
        self.buffer = self.buffer + data

    <font color="808000">def</font><font color="000080"> found_terminator</font> (self):
        data = self.buffer
        self.buffer = <font color="008000">''</font>
        <font color="808000">print</font> <font color="008000">'==&gt; (%d) %s'</font> % (self.id, repr(data))
        self.receiver.push (data + <font color="008000">'\n'</font>)

     <font color="808000">def</font><font color="000080"> handle_close</font> (self):
         self.receiver.close()
         self.close()

<font color="808000">class</font><font color="000080"> proxy_receiver</font> (asynchat.async_chat):

    channel_counter = 0

    <font color="808000">def</font><font color="000080"> __init__</font> (self, server, (conn, addr)):
        asynchat.async_chat.__init__ (self, conn)
        self.set_terminator (<font color="008000">'\n'</font>)
        self.server = server
        self.id = self.channel_counter
        self.channel_counter = self.channel_counter + 1
        self.sender = proxy_sender (self, server.there)
        self.sender.id = self.id
        self.buffer = <font color="008000">''</font>

    <font color="808000">def</font><font color="000080"> collect_incoming_data</font> (self, data):
        self.buffer = self.buffer + data
        
    <font color="808000">def</font><font color="000080"> found_terminator</font> (self):
        data = self.buffer
        self.buffer = <font color="008000">''</font>
        <font color="808000">print</font> <font color="008000">'&lt;== (%d) %s'</font> % (self.id, repr(data))
        self.sender.push (data + <font color="008000">'\n'</font>)

     <font color="808000">def</font><font color="000080"> handle_close</font> (self):
         <font color="808000">print</font> <font color="008000">'Closing'</font>
         self.sender.close()
         self.close()

<font color="808000">if</font> __name__ == <font color="008000">'__main__'</font>:
    <font color="808000">import</font> sys
    <font color="808000">import</font> string
    <font color="808000">if</font> len(sys.argv) &lt; 3:
        <font color="808000">print</font> <font color="008000">'Usage: %s &lt;server-host&gt; &lt;server-port&gt;'</font> % sys.argv[0]
    <font color="808000">else</font>:
        ps = proxy_server (sys.argv[1], string.atoi (sys.argv[2]))
        asyncore.loop()

</pre>

    <p>
      To try out the proxy, find a server (any SMTP, NNTP, or HTTP server should do fine),
      and give its hostname and port as arguments:

<pre>
python proxy.py localhost 25
</pre>

    <p>
      The proxy server will start up its server on port <code>n +
      8000</code>, in this case port 8025.  Now, use a telnet program
      to connect to that port on your server host.  Issue a few
      commands.  See how the whole session is being echoed by your
      proxy server.  Try opening up several simultaneous connections
      through your proxy.  You might also try pointing a real client
      (a news reader [port 119] or web browser [port 80]) at your proxy.

    <h3>Pipelining</h3>
    <p>
      Pipelining refers to a protocol capability.  Normally, a conversation
      with a server has a back-and-forth quality to it.  The client sends a
      command, and waits for the response.  If a client needs to send many commands
      over a high-latency connection, waiting for each response can take a long
      time.
    <p>
      For example, when sending a mail message to many recipients with
      SMTP, the client will send a series of <code>RCPT</code>
      commands, one for each recipient.  For each of these commands,
      the server will send back a reply indicating whether the mailbox
      specified is valid.  If you want to send a message to several
      hundred recipients, this can be rather tedious if the round-trip
      time for each command is long. You'd like to be able to send a
      bunch of <code>RCPT</code> commands in one batch, and then count
      off the responses to them as they come.
      
    <p>
      I have a favorite visual when explaining the advantages of
      pipelining.  Imagine each request to the server is a boxcar on a
      train.  The client is in Los Angeles, and the server is in New
      York.  Pipelining lets you hook all your cars in one long chain;
      send them to New York, where they are filled and sent back to you.
      Without pipelining you have to send one car at a time.

    <p>
      Not all protocols allow pipelining.  Not all servers support it;
      Sendmail, for example, does not support pipelining because it tends
      to fork unpredictably, leaving buffered data in a questionable state.
      A recent extension to the SMTP protocol allows a server to specify
      whether it supports pipelining.  HTTP/1.1 explicitly requires that
      a server support pipelining.

    <p>
      Servers built on top of <code>async_chat</code> automatically
      support pipelining.  It is even possible to change the
      terminator repeatedly when processing data already in the
      input buffer.  See the <code>handle_read</code> method if you're
      interested in the gory details.

    <h3>Producers</h3>
      
    <p>
      <code>async_chat</code> supports a sophisticated output
      buffering model, using a queue of data-producing objects.  For
      most purposes, you will use the <code>push()</code> method to
      send string data - but for more sophisticated usage you can push
      a <code>producer</code>

    <p>
      A <code>producer</code> is a very simple object, requiring only
      a single method in its implementation, <code>more()</code>.  See
      the code for <code>simple_producer</code> in
      <code>asynchat.py</code> for an example.  Many more examples are
      available in the Medusa distribution, in the file
      <code>producers.py</code>

  <hr>
  <address><a href="mailto:rushing@nightmare.com">Samual M. Rushing</a></address>
<!-- Created: Mon Jan 11 03:53:15 PST 1999 -->
<!-- hhmts start -->
Last modified: Fri Apr 30 21:42:52 PDT 1999
<!-- hhmts end -->
  </body>
</html>