Added thread locals and subprocess guides.

sdiehl · sdiehl · commit 67d97a1118f8 · 2012-09-18T16:30:35.000-04:00
diff --git a/index.html b/index.html
@@ -65,12 +65,13 @@ <h3 class="author">
 <li><a href="#groups-and-pools">Groups and Pools</a></li>
 <li><a href="#locks-and-semaphores">Locks and Semaphores</a></li>
 <li><a href="#thread-locals">Thread Locals</a></li>
+<li><a href="#subprocess">Subprocess</a></li>
 <li><a href="#actors">Actors</a></li>
 </ul>
 </li>
 <li><a href="#real-world-applications">Real World Applications</a><ul>
 <li><a href="#gevent-zeromq">Gevent ZeroMQ</a></li>
-<li><a href="#simple-telnet-servers">Simple Telnet Servers</a></li>
+<li><a href="#simple-servers">Simple Servers</a></li>
 <li><a href="#wsgi-servers">WSGI Servers</a></li>
 <li><a href="#streaming-servers">Streaming Servers</a></li>
 <li><a href="#long-polling">Long Polling</a></li>
@@ -249,16 +250,16 @@ <h2 id="synchronous-asynchronous-execution">Synchronous &amp; Asynchronous Execu
 Task 8 done
 Task 9 done
 Asynchronous:
-Task 0 done
-Task 6 done
 Task 1 done
-Task 9 done
-Task 3 done
-Task 4 done
 Task 2 done
-Task 7 done
 Task 5 done
+Task 7 done
+Task 9 done
 Task 8 done
+Task 0 done
+Task 3 done
+Task 4 done
+Task 6 done
 </pre></code></p>
 <p>In the synchronous case all the tasks are run sequentially,
 which results in the main programming <em>blocking</em> (
@@ -895,11 +896,11 @@ <h2 id="groups-and-pools">Groups and Pools</h2>
 <p></code>
 <pre><code class="python">
 Size of group 3
-Hello from Greenlet 34519152
+Hello from Greenlet 41900624
 Size of group 3
-Hello from Greenlet 34518512
+Hello from Greenlet 41900944
 Size of group 3
-Hello from Greenlet 34519472
+Hello from Greenlet 41899024
 Ordered
 ('task', 0)
 ('task', 1)
@@ -1011,6 +1012,148 @@ <h2 id="locks-and-semaphores">Locks and Semaphores</h2>
 ensure that resources are only in use at one time in the context
 of a program.</p>
 <h2 id="thread-locals">Thread Locals</h2>
+<p>Gevnet also allows you to specify data which is local the
+greenlet context. Internally this is implemented as a global
+lookup which addresses a private namespace keyed by the
+greenlet's <code>getcurrent()</code> value.</p>
+<pre><code class="python">
+import gevent
+from gevent.local import local
+
+stash = local()
+
+def f1():
+    stash.x = 1
+    print(stash.x)
+
+def f2():
+    stash.y = 2
+    print(stash.y)
+
+    try:
+        stash.x
+    except AttributeError:
+        print("x is not local to f2")
+
+g1 = gevent.spawn(f1)
+g2 = gevent.spawn(f2)
+
+gevent.joinall([g1, g2])
+</pre>
+
+<p></code>
+<pre><code class="python">
+1
+2
+x is not local to f2
+</pre></code></p>
+<p>Many web framework thats integrate with gevent store HTTP session
+objects inside of gevent thread locals. For example using the
+Werkzeug utility library and its proxy object we can create
+Flask style request objects.</p>
+<pre>
+<code class="python">from gevent.local import local
+from werkzeug.local import LocalProxy
+from werkzeug.wrappers import Request
+from contextlib import contextmanager
+
+from gevent.wsgi import WSGIServer
+
+_requests = local()
+request = LocalProxy(lambda: _requests.request)
+
+@contextmanager
+def sessionmanager(environ):
+    _requests.request = Request(environ)
+    yield
+    _requests.request = None
+
+def logic():
+    return "Hello " + request.remote_addr
+
+def application(environ, start_response):
+    status = '200 OK'
+
+    with sessionmanager(environ):
+        body = logic()
+
+    headers = [
+        ('Content-Type', 'text/html')
+    ]
+
+    start_response(status, headers)
+    return [body]
+
+WSGIServer(('', 8000), application).serve_forever()
+
+<code>
+</pre>
+
+<p>Flask's system is more a bit sophisticated than this example, but the
+idea of using thread locals as local session storage is nontheless the
+same.</p>
+<h2 id="subprocess">Subprocess</h2>
+<p>As of Gevent 1.0, support has been added for cooperative waiting
+on subprocess.</p>
+<pre>
+<code class="python">import gevent
+from gevent.subprocess import Popen, PIPE
+
+# Uses a green pipe which is cooperative
+sub = Popen(['uname'], stdout=PIPE)
+read_output = gevent.spawn(sub.stdout.read)
+
+output = read_output.join()
+print(output.value)
+<code>
+</pre>
+
+<pre>
+<code class="python">Linux
+<code>
+</pre>
+
+<p>Many people also want to use gevent and multiprocessing together. This
+can be done as most multiprocessing objects expose the underlying file
+descriptors.</p>
+<pre><code class="python">
+import gevent
+from multiprocessing import Process, Pipe
+from gevent.socket import wait_read, wait_write
+
+# To Process
+a, b = Pipe()
+
+# From Process
+c, d = Pipe()
+
+def relay():
+    for i in xrange(10):
+        msg = b.recv()
+        c.send(msg + " in " + str(i))
+
+def put_msg():
+    for i in xrange(10):
+        wait_write(a.fileno())
+        a.send('hi')
+
+def get_msg():
+    for i in xrange(10):
+        wait_read(d.fileno())
+        print(d.recv())
+
+if __name__ == '__main__':
+    proc = Process(target=relay)
+    proc.start()
+
+    g1 = gevent.spawn(get_msg)
+    g2 = gevent.spawn(put_msg)
+    gevent.joinall([g1, g2], timeout=1)
+</pre>
+
+<p></code>
+<pre><code class="python">
+</pre></code></p>
 <h2 id="actors">Actors</h2>
 <p>The actor model is a higher level concurrency model popularized
 by the language Erlang. In short the main idea is that you have a
@@ -1146,7 +1289,7 @@ <h2 id="gevent-zeromq">Gevent ZeroMQ</h2>
 Switched to Server for  9
 Switched to Client for  9
 </pre></code></p>
-<h2 id="simple-telnet-servers">Simple Telnet Servers</h2>
+<h2 id="simple-servers">Simple Servers</h2>
 <pre>
 <code class="python">
 # On Unix: Access with ``$ nc 127.0.0.1 5000`` 
diff --git a/tutorial.md b/tutorial.md
@@ -833,6 +833,148 @@ of a program.
 
 ## Thread Locals
 
+Gevnet also allows you to specify data which is local the
+greenlet context. Internally this is implemented as a global
+lookup which addresses a private namespace keyed by the
+greenlet's ``getcurrent()`` value.
+
+[[[cog
+import gevent
+from gevent.local import local
+
+stash = local()
+
+def f1():
+    stash.x = 1
+    print(stash.x)
+
+def f2():
+    stash.y = 2
+    print(stash.y)
+
+    try:
+        stash.x
+    except AttributeError:
+        print("x is not local to f2")
+
+g1 = gevent.spawn(f1)
+g2 = gevent.spawn(f2)
+
+gevent.joinall([g1, g2])
+]]]
+[[[end]]]
+
+Many web framework thats integrate with gevent store HTTP session
+objects inside of gevent thread locals. For example using the
+Werkzeug utility library and its proxy object we can create
+Flask style request objects.
+
+<pre>
+<code class="python">from gevent.local import local
+from werkzeug.local import LocalProxy
+from werkzeug.wrappers import Request
+from contextlib import contextmanager
+
+from gevent.wsgi import WSGIServer
+
+_requests = local()
+request = LocalProxy(lambda: _requests.request)
+
+@contextmanager
+def sessionmanager(environ):
+    _requests.request = Request(environ)
+    yield
+    _requests.request = None
+
+def logic():
+    return "Hello " + request.remote_addr
+
+def application(environ, start_response):
+    status = '200 OK'
+
+    with sessionmanager(environ):
+        body = logic()
+
+    headers = [
+        ('Content-Type', 'text/html')
+    ]
+
+    start_response(status, headers)
+    return [body]
+
+WSGIServer(('', 8000), application).serve_forever()
+
+
+<code>
+</pre>
+
+Flask's system is more a bit sophisticated than this example, but the
+idea of using thread locals as local session storage is nontheless the
+same.
+
+## Subprocess
+
+As of Gevent 1.0, support has been added for cooperative waiting
+on subprocess.
+
+<pre>
+<code class="python">import gevent
+from gevent.subprocess import Popen, PIPE
+
+# Uses a green pipe which is cooperative
+sub = Popen(['uname'], stdout=PIPE)
+read_output = gevent.spawn(sub.stdout.read)
+
+output = read_output.join()
+print(output.value)
+<code>
+</pre>
+
+<pre>
+<code class="python">Linux
+<code>
+</pre>
+
+Many people also want to use gevent and multiprocessing together. This
+can be done as most multiprocessing objects expose the underlying file
+descriptors.
+
+[[[cog
+import gevent
+from multiprocessing import Process, Pipe
+from gevent.socket import wait_read, wait_write
+
+# To Process
+a, b = Pipe()
+
+# From Process
+c, d = Pipe()
+
+def relay():
+    for i in xrange(10):
+        msg = b.recv()
+        c.send(msg + " in " + str(i))
+
+def put_msg():
+    for i in xrange(10):
+        wait_write(a.fileno())
+        a.send('hi')
+
+def get_msg():
+    for i in xrange(10):
+        wait_read(d.fileno())
+        print(d.recv())
+
+if __name__ == '__main__':
+    proc = Process(target=relay)
+    proc.start()
+
+    g1 = gevent.spawn(get_msg)
+    g2 = gevent.spawn(put_msg)
+    gevent.joinall([g1, g2], timeout=1)
+]]]
+[[[end]]]
+
 ## Actors
 
 The actor model is a higher level concurrency model popularized
@@ -957,7 +1099,7 @@ gevent.joinall([publisher, client])
 ]]]
 [[[end]]]
 
-## Simple Telnet Servers
+## Simple Servers
 
 <pre>
 <code class="python">
