diff options
| author | Nir Soffer <nirsof@gmail.com> | 2017-11-06 23:16:37 +0200 |
|---|---|---|
| committer | Victor Stinner <victor.stinner@gmail.com> | 2017-11-06 13:16:37 -0800 |
| commit | ad455cd9243319b896c86074ffeb3bf78a82f4ec (patch) | |
| tree | 48a4989023b45958c9fa7dfe090936506998906d /Misc | |
| parent | 30f4fa456ef626ad7a92759f492ec7a268f7af4e (diff) | |
| download | cpython-git-ad455cd9243319b896c86074ffeb3bf78a82f4ec.tar.gz | |
bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)
blocksize was hardcoded to 8192, preventing efficient upload when using
file-like body. Add blocksize argument to __init__, so users can
configure the blocksize to fit their needs.
I tested this uploading data from /dev/zero to a web server dropping the
received data, to test the overhead of the HTTPConnection.send() with a
file-like object.
Here is an example 10g upload with the default buffer size (8192):
$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 17.53 seconds (584.00m/s)
real 0m17.574s
user 0m8.887s
sys 0m5.971s
Same with 512k blocksize:
$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 6.60 seconds (1551.15m/s)
real 0m6.641s
user 0m3.426s
sys 0m2.162s
In real world usage the difference will be smaller, depending on the
local and remote storage and the network.
See https://github.com/nirs/http-bench for more info.
Diffstat (limited to 'Misc')
| -rw-r--r-- | Misc/NEWS.d/next/Library/2017-11-05-01-17-12.bpo-31945.TLPBtS.rst | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/Misc/NEWS.d/next/Library/2017-11-05-01-17-12.bpo-31945.TLPBtS.rst b/Misc/NEWS.d/next/Library/2017-11-05-01-17-12.bpo-31945.TLPBtS.rst new file mode 100644 index 0000000000..49b8395f28 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2017-11-05-01-17-12.bpo-31945.TLPBtS.rst @@ -0,0 +1,3 @@ +Add Configurable *blocksize* to ``HTTPConnection`` and +``HTTPSConnection`` for improved upload throughput. Patch by Nir +Soffer. |