summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGuido van Rossum <guido@python.org>2012-10-29 10:41:25 -0700
committerGuido van Rossum <guido@python.org>2012-10-29 10:41:25 -0700
commit54de22d7c4796a364f88601ee4c91c0f359cff77 (patch)
treea34ea58a225b1eedd1d04323ba661c32583372cb
parenta492cf8505aa63ec92a0a20681255bffc785530c (diff)
downloadtrollius-54de22d7c4796a364f88601ee4c91c0f359cff77.tar.gz
Update README, add TODO and xkcd.py.
-rw-r--r--README29
-rw-r--r--TODO85
-rwxr-xr-xxkcd.py18
3 files changed, 130 insertions, 2 deletions
diff --git a/README b/README
index 35541bc..1339d06 100644
--- a/README
+++ b/README
@@ -1,5 +1,7 @@
Tulip is the codename for my attempt at understanding PEP-380 style
-coroutines (i.e. those using generators and 'yield from').
+coroutines (i.e. those using generators and 'yield from').
+
+*** This requires Python 3.3 or later! ***
For reference, see many threads in python-ideas@python.org started in
October 2012, especially those with "The async API of the Future" in
@@ -8,8 +10,31 @@ their subject, and the various spin-off threads.
A particularly influential tutorial by Greg Ewing:
http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Examples/Scheduler/scheduler.txt
-Python version: 3.3.
+A message I posted with some explanation of the design:
+http://mail.python.org/pipermail/python-ideas/2012-October/017501.html
+
+Essential files here:
+
+- main.py: the main program for testing, and a rough HTTP client
+- sockets.py: transports for sockets and SSL, and a buffering layer
+- scheduling.py: a Task class and related stuff; this is where the PEP
+ 380 scheduler is implemented
+- polling.py: an event loop and basic polling implementations for:
+ select(), poll(), epoll(), kqueue()
+
+Secondary files:
+
+- .hgignore: files I don't care about
+- Makefile: various quick shell commands
+- README: this file
+- TODO: longer list of TODO items and general thoughts
+- longlines.py: stupid style checker
+- p3time.py: benchmark yield from vs. plain functions
+- xkcd.py: *synchronous* ssl example
+- yyftime.py: benchmark yield from vs. yield <future>
Copyright/license: Open source, Apache 2.0. Enjoy.
+Master Mercurial repo: http://code.google.com/p/tulip/
+
--Guido van Rossum <guido@python.org>
diff --git a/TODO b/TODO
new file mode 100644
index 0000000..5c194bb
--- /dev/null
+++ b/TODO
@@ -0,0 +1,85 @@
+- Ensure multiple tasks can do atomic writes to the same pipe (since
+ UNIX guarantees that short writes to pipes are atomic).
+
+- Ensure some easy way of distributing accepted connections across tasks.
+
+- Make pollster take an abstract token and return it.
+
+- Make pollster a sub-object instead of a superclass of the eventloop.
+
+- Be wary of thread-local storage. There should be a standard API to
+ get the current Context (which holds current task, event loop, and
+ maybe more) and a standard meta-API to change how that standard API
+ works (i.e. without monkey-patching).
+
+- See how much of asyncore I've already replaced.
+
+- Write up a tutorial for the scheduling API.
+
+- Change block_r/w into COROUTINE style APIs.
+
+- Do we need _async suffixes to all async APIs?
+
+- Do we need synchronous parallel APIs for all async APIs?
+
+- Add a decorator just for documenting a coroutine?
+
+- Fix recv(), send() to catch EAGAIN.
+
+- Fix ssh recv(), send() to catch SSLWantReadError and SSLWantWriteError.
+
+- Could BufferedReader reuse the standard io module's readers???
+
+[From older list]
+
+- Is it better to have separate add_{reader,writer} methods, vs. one
+ add_thingie method taking a fd and a r/w flag?
+
+- Multiple readers/writers per socket? (At which level? pollster,
+ eventloop, or scheduler?)
+
+- Should poll() return a list of tokens or a list of (fd, flag, token)?
+
+- Could poll() usefully be an iterator?
+
+- Do we need to support more epoll and/or kqueue modes/flags/options/etc.?
+
+- Optimize register/unregister calls away if they cancel each other out?
+
+- Should block() use a queue?
+
+- Add explicit wait queue to wait for Task's completion, instead of
+ callbacks?
+
+- Global functions vs. Task methods?
+
+- Is the Task design good?
+
+- Make Task more like Future? (Or less???)
+
+- Implement various lock styles a la threading.py.
+
+- Handle disconnect errors from send() (and from recv()???).
+
+- Add write() calls that don't require yield from.
+
+- Add simple non-async APIs, for simple apps?
+
+
+MISTAKES I MADE
+
+- Forgetting yield from. (E.g.: scheduler.sleep(1).)
+
+- Forgot to add bare yield at end of internal function, after block().
+
+- Forgot to call add_done_callback().
+
+- Forgot to pass an undoer to block(), bug only found when cancelled.
+
+- Subtle accounting mistake in a callback.
+
+- Used context.eventloop from a different thread, forgetting about TLS.
+
+- Nasty race: eventloop.ready may contain both an I/O callback and a
+ cancel callback. How to avoid? Keep the DelayedCall in ready. Is
+ that enough?
diff --git a/xkcd.py b/xkcd.py
new file mode 100755
index 0000000..474009d
--- /dev/null
+++ b/xkcd.py
@@ -0,0 +1,18 @@
+#!/usr/bin/env python3.3
+"""Minimal synchronous SSL demo, connecting to xkcd.com."""
+
+import socket, ssl
+
+s = socket.socket()
+s.connect(('xkcd.com', 443))
+ss = ssl.wrap_socket(s)
+
+ss.send(b'GET / HTTP/1.0\r\n\r\n')
+
+while True:
+ data = ss.recv(1000000)
+ print(data)
+ if not data:
+ break
+
+ss.close()