# -*- Mode: text -*-

TO DO LARGER TASKS

- Need more examples.

- Benchmarkable but more realistic HTTP server?

- Example of using UDP.

- Write up a tutorial for the scheduling API.

- More systematic approach to logging.  Logger objects?  What about
  heavy-duty logging, tracking essentially all task state changes?

- Restructure directory, move demos and benchmarks to subdirectories.


TO DO LATER

- When multiple tasks are accessing the same socket, they should
  either get interleaved I/O or an immediate exception; it should not
  compromise the integrity of the scheduler or the app or leave a task
  hanging.

- For epoll you probably want to check/(log?) EPOLLHUP and EPOLLERR errors.

- Add the simplest API possible to run a generator with a timeout.

- Ensure multiple tasks can do atomic writes to the same pipe (since
  UNIX guarantees that short writes to pipes are atomic).

- Ensure some easy way of distributing accepted connections across tasks.

- Be wary of thread-local storage.  There should be a standard API to
  get the current Context (which holds current task, event loop, and
  maybe more) and a standard meta-API to change how that standard API
  works (i.e. without monkey-patching).

- See how much of asyncore I've already replaced.

- Could BufferedReader reuse the standard io module's readers???

- Support ZeroMQ "sockets" which are user objects.  Though possibly
  this can be supported by getting the underlying fd?  See
  http://mail.python.org/pipermail/python-ideas/2012-October/017532.html
  OTOH see 
  https://github.com/zeromq/pyzmq/blob/master/zmq/eventloop/ioloop.py

- Study goroutines (again).

- Benchmarks: http://nichol.as/benchmark-of-python-web-servers


FROM OLDER LIST

- Multiple readers/writers per socket?  (At which level? pollster,
  eventloop, or scheduler?)

- Could poll() usefully be an iterator?

- Do we need to support more epoll and/or kqueue modes/flags/options/etc.?

- Optimize register/unregister calls away if they cancel each other out?

- Add explicit wait queue to wait for Task's completion, instead of
  callbacks?

- Look at pyfdpdlib's ioloop.py:
  http://code.google.com/p/pyftpdlib/source/browse/trunk/pyftpdlib/lib/ioloop.py


MISTAKES I MADE

- Forgetting yield from.  (E.g.: scheduler.sleep(1); listener.accept().)

- Forgot to add bare yield at end of internal function, after block().

- Forgot to call add_done_callback().

- Forgot to pass an undoer to block(), bug only found when cancelled.

- Subtle accounting mistake in a callback.

- Used context.eventloop from a different thread, forgetting about TLS.

- Nasty race: eventloop.ready may contain both an I/O callback and a
  cancel callback.  How to avoid?  Keep the DelayedCall in ready.  Is
  that enough?

- If a toplevel task raises an error it just stops and nothing is logged
  unless you have debug logging on.  This confused me.  (Then again,
  previously I logged whenever a task raised an error, and that was too
  chatty...)

- Forgot to set the connection socket returned by accept() in
  nonblocking mode.

- Nastiest so far (cost me about a day): A race condition in
  call_in_thread() where the Future's done_callback (which was
  task.unblock()) would run immediately at the time when
  add_done_callback() was called, and this screwed over the task
  state.  Solution: wrap the callback in eventloop.call_later().
  Ironically, I had a comment stating there might be a race condition.

- Another bug where I was calling unblock() for the current thread
  immediately after calling block(), before yielding.

- readexactly() wasn't checking for EOF, so could be looping.
  (Worse, the first fix I attempted was wrong.)

- Spent a day trying to understand why a tentative patch trying to
  move the recv() implementation into the eventloop (or the pollster)
  resulted in problems cancelling a recv() call.  Ultimately the
  problem is that the cancellation mechanism is part of the coroutine
  scheduler, which simply throws an exception into a task when it next
  runs, and there isn't anything to be interrupted in the eventloop;
  but the eventloop still has a reader registered (which will never
  fire because I suspended the server -- that's my test case :-).
  Then, the eventloop keeps running until the last file descriptor is
  unregistered.  What contributed to this disaster?
  * I didn't build the whole infrastructure, just played with recv()
  * I don't have unittests
  * I don't have good logging to see what is going

- In sockets.py, in some SSL error handling code, used the wrong
  variable (sock instead of sslsock).  A linter would have found this.

- In polling.py, in KqueuePollster.register_writer(), a copy/paste
  error where I was testing for "if fd not in self.readers" instead of
  writers.  This only came out when I had both a reader and a writer
  for the same fd.

- Submitted some changes prematurely (forgot to pass the filename on
  hg ci).

- Forgot again that shutdown(SHUT_WR) on an ssl socket does not work
  as I expected.  I ran into this with the origininal sockets.py and
  again in transport.py.

- Having the same callback for both reading and writing has a problem:
  it may be scheduled twice, and if the first call closes the socket,
  the second runs into trouble.


MISTAKES I MADE IN TULIP V2

- Nice one: Future._schedule_callbacks() wasn't scheduling any callbacks.
  Spot the bug in these four lines:

    def _schedule_callbacks(self):
      callbacks = self._callbacks[:]
      self._callbacks[:] = []
      for callback in self._callbacks:
          self._event_loop.call_soon(callback, self)

  The good news is that I found it with a unittest (albeit not the
  unittest intended to exercise this particular method :-( ).

- In _make_self_pipe_or_sock(), called _pollster.register_reader()
  instead of add_reader(), trying to optimize something but breaking
  things instead (since the -- internal -- API of register_reader()
  had changed).