summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #1001 from basak/add_description_parameterglacierMitch Garnaat2012-09-191-2/+3
|\ | | | | Add description to Vault.create_archive_from_file
| * Add description to Vault.create_archive_from_fileRobie Basak2012-09-191-2/+3
|/ | | | | Allow archive creation via create_archive_from_file to specify an archive description.
* Merge pull request #975 from tedder/8cd93058131df93932d8ea99e4e78ce0695a3ad8Mitch Garnaat2012-09-191-0/+18
|\ | | | | Add initial Glacier support in root boto object.
| * add connect_glacier to the main boto object.tedder2012-09-081-0/+18
| |
* | Merge pull request #994 from jamesls/glacier-concurrentMitch Garnaat2012-09-183-8/+248
|\ \ | | | | | | Add threaded multipart archive upload for glacier
| * | Add threaded multipart archive upload for glacierJames Saryerwinnie2012-09-183-8/+248
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will internally create a thread pool that concurrently uploads the various chunks of a file using the multipart API for glacier. When testing upload speeds, this was approximately 13 times faster than the single threaded archive upload. It also is much better at sustaining a constant upload rate, in the single threaded upload, the upload rate fluctuated quite a bit (which makes sense given that it part of the time it's reading from disk and computing hashes instead of sending data). For comparison, for a 1GB file: Operation | Time (secs) | Average Upload rate (MB/s) | --------------------------------------------------------------- s3multiput 216 6.0 glacier-single 1291 2.0 glacier-concurrent 98 11.5
* | Merge branch 'glacier-l1-bugfix' into glacierJames Saryerwinnie2012-09-181-15/+14
|\ \
| * | Bug fix: format strings had an extra '%s'James Saryerwinnie2012-09-181-3/+3
| | | | | | | | | | | | | | | This was for abort_multipart_upload and list_parts, and list_multipart_uploads
| * | Pep 8 cleanupJames Saryerwinnie2012-09-181-3/+1
| | |
| * | Bug fix: limit/marker should be keys in params dictJames Saryerwinnie2012-09-181-8/+8
| | |
| * | Bug fix: s/params_list/param_list/James Saryerwinnie2012-09-181-1/+1
| | |
| * | Bug fix: import urllibJames Saryerwinnie2012-09-181-0/+1
|/ / | | | | | | | | The make_request method uses urllib yet it is not imported.
* | Merge branch 'win-binary-file' into glacierJames Saryerwinnie2012-09-171-2/+3
|\ \
| * | Ensure file is opened in binary mode, fixes #988James Saryerwinnie2012-09-171-2/+3
|/ / | | | | | | | | Was able to repro the issue by trying to submit a binary file, verified the fix works with binary files on windows.
* | A few PEP8 cleanups.Mitch Garnaat2012-09-161-3/+4
| |
* | glacier: fix Vault.retrieve_archive result typeRobie Basak2012-09-161-1/+1
| | | | | | | | | | | | retrieve_archive is documented to return a Job object, but currently it returns a job id string. As this is layer 2, it would make more sense to return a full Job object, so do this instead.
* | Merge pull request #980 from almost/glacierMitch Garnaat2012-09-121-0/+28
|\ \ | | | | | | Support for retrieve-inventory by @yamatt
| * \ Merge pull request #1 from yamatt/glacierThomas Parslow2012-09-121-0/+28
| |\ \ | | | | | | | | Added retreive-inventory function
| | * | Added retreive-inventory functionMatthew Copperwaite2012-09-101-0/+28
| |/ /
* | | Merge branch 'glacier-closedfile' into glacierJames Saryerwinnie2012-09-111-1/+2
|\ \ \
| * | | Raise a ValueError when writing to a closed fileJames Saryerwinnie2012-09-111-1/+2
|/ / / | | | | | | | | | | | | | | | | | | | | | This is consistent with other file like objects including: * file * tempfile * StringIO
* | | Merge pull request #977 from jamesls/glacier-filereadMitch Garnaat2012-09-105-22/+64
|\ \ \ | | | | | | | | Allow upload_archive to accept file like objects
| * | | Allow upload_archive to accept file like objectsJames Saryerwinnie2012-09-105-18/+59
| | | | | | | | | | | | | | | | | | | | The entire file is never read into memory, only (configurable) single chunks are read at a single time.
| * | | Allow single op threshold to be configurableJames Saryerwinnie2012-09-061-4/+5
| | |/ | |/| | | | | | | | | | | | | This is the threshold at which the Vault class will use a multipart upload instead of a single operation upload.
* | | Merge pull request #968 from jamesls/glacier-perfMitch Garnaat2012-09-102-9/+37
|\ \ \ | |/ / |/| | Reduce memory usage for chunk_hashes
| * | Reduce memory usage for chunk_hashesJames Saryerwinnie2012-09-062-9/+37
|/ / | | | | | | | | | | | | | | | | | | On a 180MB file this reduced total memory usage by approximately 40%. This was also marginally faster (but not by much). I've also added the start of unittests for the writer module, and I've written some very basic unittests for the chunk_hashes function.
* | Remove leading whitespace in glacier modulesJames Saryerwinnie2012-09-062-3/+3
| |
* | Pep8 cleanup to glacer l2 unit testsJames Saryerwinnie2012-09-061-30/+47
| | | | | | | | | | Also switched the assert statements to use assertEqual for better error diagnostics.
* | Merge remote-tracking branch 'almost/glacier' into glacierJames Saryerwinnie2012-09-067-28/+192
|\ \ | |/
| * Added glacier to the readme :)Thomas Parslow2012-09-061-0/+1
| |
| * Fixed testsThomas Parslow2012-09-061-0/+1
| |
| * Merge branch 'glacier' of github.com:boto/boto into glacierThomas Parslow2012-09-063-67/+70
| |\ | |/ |/| | | | | Conflicts: boto/glacier/layer2.py
* | Merge branch 'delete-vault' into glacierJames Saryerwinnie2012-09-063-1/+74
|\ \
| * | Add Layer2.delete_vaultJames Saryerwinnie2012-09-063-0/+64
| | | | | | | | | | | | | | | | | | This allow layer2 to be used to both create and delete vaults. I've also added the start of integration tests for layer2 that will create, retrieve, and delete a vault.
| * | Update Layer2.create_vault to return a Vault objectJames Saryerwinnie2012-09-061-1/+10
|/ / | | | | | | This makes it consistent with Layer2.get_vault.
* | Merge branch 'glacier' of git://github.com/almost/boto into almost-glacierMitch Garnaat2012-09-062-2/+4
|\ \
* \ \ Merge pull request #964 from almost/glacierMitch Garnaat2012-09-053-10/+50
|\ \ \ | | | | | | | | Return the response and not the fully read body for non-json responses (...
* \ \ \ Merge pull request #963 from almost/glacierMitch Garnaat2012-09-051-8/+3
|\ \ \ \ | | | | | | | | | | Fixed a couple of bugs in archive retrieval for layer2 Glacier
| | | | * Added a very basic integration test for glacierThomas Parslow2012-09-062-0/+26
| | | | |
| | | | * Not-yet-working integration test (commented out)Thomas Parslow2012-09-061-0/+60
| | | | |
| | | | * Remove print statements, whoopsThomas Parslow2012-09-062-2/+0
| | | | |
| | | | * Added some tests for glacier layer 2Thomas Parslow2012-09-066-28/+176
| | | |/
| | | * Fixed the tests to work with GlacierResponseThomas Parslow2012-09-062-2/+4
| | |/
| | * Added a GlacierResponse objectThomas Parslow2012-09-053-10/+50
| | |
| | * Return the response and not the fully read body for non-json responses ↵Thomas Parslow2012-09-051-1/+1
| |/ | | | | | | (otherwise you get the whole output of a job read into memory)
| * describe_job needs an archive name as a well as a job idThomas Parslow2012-09-051-1/+1
| |
| * Archive retrieval job should not include a format (that's for listing ↵Thomas Parslow2012-09-051-7/+2
|/ | | | operations)
* Merge branch 'moliware-glacier' into glacierMitch Garnaat2012-09-034-23/+177
|\
| * Command line tool for amazon glaciermoliware2012-09-032-2/+156
| |
| * unicode constants in the response dict were not the responsible of the ↵moliware2012-09-031-15/+15
| | | | | | | | UnicodeDecodeError
View Lorry Controller Status