From b4bd81dce17d23b3e402b7fd492bb2ebd30b284c Mon Sep 17 00:00:00 2001 From: Jeff Forcier Date: Mon, 7 Mar 2022 16:21:28 -0500 Subject: Massively speed up low-level SFTP read/write This doesn't impact most users who perform reads/writes using SFTPClient.get(fo)/put(fo) as those naturally perform chunking. However, users accessing the raw SFTPFile objects via SFTPClient.open() and then reading/writing large (more than a few MB) files, may experience severe slowdown due to inefficient slicing of the file being read/written. This change replaces the naive "slice a list of bytes" code with bytearray and memoryview, which are significantly more performant in these use cases, while remaining backwards compatible. Patch courtesy of Sevastian Tchernov. --- sites/www/changelog.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'sites') diff --git a/sites/www/changelog.rst b/sites/www/changelog.rst index 48119f00..9387aabf 100644 --- a/sites/www/changelog.rst +++ b/sites/www/changelog.rst @@ -2,6 +2,16 @@ Changelog ========= +- :bug:`892 major` Significantly speed up low-level read/write actions on + `~paramiko.sftp_file.SFTPFile` objects by using `bytearray`/`memoryview`. + This is unlikely to change anything for users of the higher level methods + like ``SFTPClient.get`` or ``SFTPClient.getfo``, but users of + `SFTPClient.open ` will likely see + orders of magnitude improvements for files larger than a few megabytes in + size. + + Thanks to ``@jkji`` for the original report and to Sevastian Tchernov for the + patch. - :support:`1985` Add ``six`` explicitly to install-requires; it snuck into active use at some point but has only been indicated by transitive dependency on ``bcrypt`` until they somewhat-recently dropped it. This will be -- cgit v1.2.1