<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/eventlet.git/tests/isolated, branch subprocess</title>
<subtitle>github.com: eventlet/eventlet.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/'/>
<entry>
<title>Provide green http without undesired side effects</title>
<updated>2016-07-08T14:06:23+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-06-14T18:43:40+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=c93279086c21b9f838e1f9c9b232d60317fcfc71'/>
<id>c93279086c21b9f838e1f9c9b232d60317fcfc71</id>
<content type='text'>
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye &lt;xdegaye@users.sourceforge.net&gt;
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the original http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

A new dependency, enum-compat, is added to ensure that the enum module
is present on Python 3.3 (the http package code comes the latest Python
development branch and uses enum).

[1] https://github.com/getsentry/raven-python/issues/703
[2] https://github.com/eventlet/eventlet/issues/316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The Eventlet patcher and the way we were patching multi-level http
package don't work well[1][2]. I spent a lot of time trying to make it
work but in the end every solution I came up with was breaking something
else and made the patching and providing green http even more
complicated - I wouldn't envy anyone having to debug it in the future.

After a lot of thinking I decided having our own copy of http with the
necessary modifications applied seems like the most straightforward and
the most reliable solution, even considering its downsides (we need to
keep it up to date ourselves and the API won't 100 % match the regular
http module API on older Python 3 versions as our bundled version is the
most recent one and has bug fixes and extra features implemented).

The code introduces by this commit comes from the following Python
commit (development branch):

commit 6251d66ba9a692d3adf5d2e6818b29ac44130787
Author: Xavier de Gaye &lt;xdegaye@users.sourceforge.net&gt;
Date:   2016-06-15 11:35:29 +0200

    Issue #26862: SYS_getdents64 does not need to be defined on android
    API 21.

Changes to the original http package code involve:

* Removing unnecessary import(s)
* Replacing some regular imports with eventlet.green imports
* Replacing fullmatch()[3] usage with match() so we stay Python 3.3
  compatible

I left urllib.parse imports intact as nothing there performs IO.

Green httplib module is also modified because it used to import
http.client using patcher which was breaking things the same way.

A new dependency, enum-compat, is added to ensure that the enum module
is present on Python 3.3 (the http package code comes the latest Python
development branch and uses enum).

[1] https://github.com/getsentry/raven-python/issues/703
[2] https://github.com/eventlet/eventlet/issues/316
[3] https://docs.python.org/3/library/re.html#re.fullmatch

This patch is contributed by Smarkets Limited.
</pre>
</div>
</content>
</entry>
<entry>
<title>greenio: only trampoline when we block</title>
<updated>2016-07-01T09:29:16+00:00</updated>
<author>
<name>David Szotten</name>
<email>davidszotten@gmail.com</email>
</author>
<published>2016-07-01T09:29:16+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=863a1b7605b5c90ba470e597666c61a3261b706c'/>
<id>863a1b7605b5c90ba470e597666c61a3261b706c</id>
<content type='text'>
https://github.com/eventlet/eventlet/pull/314</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
https://github.com/eventlet/eventlet/pull/314</pre>
</div>
</content>
</entry>
<entry>
<title>Remove select.poll and improve subprocess</title>
<updated>2016-05-20T08:23:36+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-05-20T08:23:36+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=614a20462aebfe85a54ce35a7daaf1a7dbde44a7'/>
<id>614a20462aebfe85a54ce35a7daaf1a7dbde44a7</id>
<content type='text'>
green select: Delete unpatched poll once again

https://github.com/eventlet/eventlet/pull/317

Previously attempted in f63165c, had to be reverted in 8ea9df6 because
subprocess was failing after monkey patching.

Turns out we haven't been monkey patching the subprocess module at all,
this patch adds that in order for the tests to pass.

This part is changed because otherwise Popen class instantiation would
cause an infinite loop when monkey patching is applied:

    -subprocess_orig = __import__("subprocess")
    +subprocess_orig = patcher.original("subprocess")

This patch is contributed by Smarkets Limited.

* green subprocess: Provide green check_output

This patch is contributed by Smarkets Limited.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
green select: Delete unpatched poll once again

https://github.com/eventlet/eventlet/pull/317

Previously attempted in f63165c, had to be reverted in 8ea9df6 because
subprocess was failing after monkey patching.

Turns out we haven't been monkey patching the subprocess module at all,
this patch adds that in order for the tests to pass.

This part is changed because otherwise Popen class instantiation would
cause an infinite loop when monkey patching is applied:

    -subprocess_orig = __import__("subprocess")
    +subprocess_orig = patcher.original("subprocess")

This patch is contributed by Smarkets Limited.

* green subprocess: Provide green check_output

This patch is contributed by Smarkets Limited.
</pre>
</div>
</content>
</entry>
<entry>
<title>patcher: patch existing threading locks; Thanks to Alexis Lee</title>
<updated>2016-04-18T20:07:00+00:00</updated>
<author>
<name>Alexis Lee</name>
<email>github@lxsli.co.uk</email>
</author>
<published>2016-04-18T20:07:00+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=bbeaa2e3a0b6da26af08f5e6734767e04c187c5b'/>
<id>bbeaa2e3a0b6da26af08f5e6734767e04c187c5b</id>
<content type='text'>
In projects which dynamically determine whether to activate eventlet,
it can be hard not to import a low level module like logging before
eventlet. When logging is imported it initialises a threading.RLock
which it uses to protect the logging configuration. If two
greenthreads attempt to claim this lock, the second one will block the
/native/ thread not just itself. As green systems usually only have
one native thread, this will freeze the whole system.

Search the GC for unsafe RLocks and replace their internal Lock with a
safe one while monkey-patching.

The tests pass, but were they to fail, the test process would never
return. To deal with this, I've added a test dependency on
subprocess32 which is a backport of the stdlib subprocess module from
Python3. This offers a timeout option on Popen#communicate, which I've
arbitrarily set at 30 seconds.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
In projects which dynamically determine whether to activate eventlet,
it can be hard not to import a low level module like logging before
eventlet. When logging is imported it initialises a threading.RLock
which it uses to protect the logging configuration. If two
greenthreads attempt to claim this lock, the second one will block the
/native/ thread not just itself. As green systems usually only have
one native thread, this will freeze the whole system.

Search the GC for unsafe RLocks and replace their internal Lock with a
safe one while monkey-patching.

The tests pass, but were they to fail, the test process would never
return. To deal with this, I've added a test dependency on
subprocess32 which is a backport of the stdlib subprocess module from
Python3. This offers a timeout option on Popen#communicate, which I've
arbitrarily set at 30 seconds.</pre>
</div>
</content>
</entry>
<entry>
<title>little clean up</title>
<updated>2016-02-12T15:42:03+00:00</updated>
<author>
<name>Sergey Shepelev</name>
<email>temotor@gmail.com</email>
</author>
<published>2016-02-11T20:28:17+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=7f08dfd25077d9044eec361ba2d890bbfd7d3815'/>
<id>7f08dfd25077d9044eec361ba2d890bbfd7d3815</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>wsgi: Fix handling partial writes on Python 3</title>
<updated>2016-02-09T10:20:05+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-02-09T01:24:47+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=b7380fdc70fde26777f4f141da3f01570e870cac'/>
<id>b7380fdc70fde26777f4f141da3f01570e870cac</id>
<content type='text'>
Closes https://github.com/eventlet/eventlet/issues/295 (in the wsgi
module we use a custom writelines implementation now).

Those write() calls might write only part of the data (and even if they
don't - it's more readable to make sure all data is written
explicitly).

I changed the test code so that the write() implementation returns the
number of characters logged and it cooperates nicely with writeall()
now.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Closes https://github.com/eventlet/eventlet/issues/295 (in the wsgi
module we use a custom writelines implementation now).

Those write() calls might write only part of the data (and even if they
don't - it's more readable to make sure all data is written
explicitly).

I changed the test code so that the write() implementation returns the
number of characters logged and it cooperates nicely with writeall()
now.
</pre>
</div>
</content>
</entry>
<entry>
<title>patcher: certain order of import subprocess and monkey_patch breaks .communicate()</title>
<updated>2016-01-24T18:36:38+00:00</updated>
<author>
<name>Sergey Shepelev</name>
<email>temotor@gmail.com</email>
</author>
<published>2016-01-24T18:27:52+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=8ea9df6b9f6a13bb47feb35bacc60968c5ce4b43'/>
<id>8ea9df6b9f6a13bb47feb35bacc60968c5ce4b43</id>
<content type='text'>
https://github.com/eventlet/eventlet/issues/290
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
https://github.com/eventlet/eventlet/issues/290
</pre>
</div>
</content>
</entry>
<entry>
<title>green selectors: Override DefaultSelector as well</title>
<updated>2016-01-11T21:25:15+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-01-11T21:25:15+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=16aa6cb370c44ea29f4781d9d0a2ee799ebc8c12'/>
<id>16aa6cb370c44ea29f4781d9d0a2ee799ebc8c12</id>
<content type='text'>
The original patch[1] missed one thing which I just discovered - there's
selectors.DefaultSelector alias for the "the most efficient implementation
available on the current platform"[2].

Before this patch a non-green selector class would be obtained when
DefaultSelector was used.

[1] 0d509ef7d2eea3ed4f8ade35d5d09918c811b56c
[2] https://docs.python.org/3.5/library/selectors.html#selectors.DefaultSelector
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The original patch[1] missed one thing which I just discovered - there's
selectors.DefaultSelector alias for the "the most efficient implementation
available on the current platform"[2].

Before this patch a non-green selector class would be obtained when
DefaultSelector was used.

[1] 0d509ef7d2eea3ed4f8ade35d5d09918c811b56c
[2] https://docs.python.org/3.5/library/selectors.html#selectors.DefaultSelector
</pre>
</div>
</content>
</entry>
<entry>
<title>green select: Make sure devpoll method is removed</title>
<updated>2016-01-11T21:21:12+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-01-11T21:21:12+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=2506a37fa3a73e6726d8d89bb0b0a1b6a593822c'/>
<id>2506a37fa3a73e6726d8d89bb0b0a1b6a593822c</id>
<content type='text'>
devpoll is another polling method, added in Python 3.3. We don't have
a green version of it so we better remove it too (see [1] for more
details).

[1] f63165c0e3c85699ebdb454878d1eaea13e90553
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
devpoll is another polling method, added in Python 3.3. We don't have
a green version of it so we better remove it too (see [1] for more
details).

[1] f63165c0e3c85699ebdb454878d1eaea13e90553
</pre>
</div>
</content>
</entry>
<entry>
<title>green selectors: Remove non-green selectors</title>
<updated>2016-01-11T21:14:57+00:00</updated>
<author>
<name>Jakub Stasiak</name>
<email>jakub@stasiak.at</email>
</author>
<published>2016-01-11T21:14:57+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/eventlet.git/commit/?id=ebdcce00d94d0ecc4a6179045a05779179d50a42'/>
<id>ebdcce00d94d0ecc4a6179045a05779179d50a42</id>
<content type='text'>
The code pretending that all the selector classes exist and
"implementing" them using SelectSelector was added in [1] because there
was no facility to easily remove things from green version of modules
(as far as I understand) - the facility has been added in [2] thus
allowing us to make this change.

As mentioned in the code I believe this approach is likely to be less
confusing than the previous one (SelectSelector doesn't expose fileno()
method that the other selectors have so it's not fully API-compatible
with them).

[1] 0d509ef7d2eea3ed4f8ade35d5d09918c811b56c
[2] f63165c0e3c85699ebdb454878d1eaea13e90553
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The code pretending that all the selector classes exist and
"implementing" them using SelectSelector was added in [1] because there
was no facility to easily remove things from green version of modules
(as far as I understand) - the facility has been added in [2] thus
allowing us to make this change.

As mentioned in the code I believe this approach is likely to be less
confusing than the previous one (SelectSelector doesn't expose fileno()
method that the other selectors have so it's not fully API-compatible
with them).

[1] 0d509ef7d2eea3ed4f8ade35d5d09918c811b56c
[2] f63165c0e3c85699ebdb454878d1eaea13e90553
</pre>
</div>
</content>
</entry>
</feed>
