summaryrefslogtreecommitdiff
path: root/src/backends
Commit message (Collapse)AuthorAgeFilesLines
* Add build-time option to suffix library names sent to dlopenTim Burke2021-10-269-18/+18
| | | | | | This is useful when repacking libraries for python wheels, for example. Change-Id: Ie7b36584de5054c14a9b77d87a5c5fa5cc7a3719
* Stop using ceill() to compute padded data sizePete Zaitcev2017-06-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The well-known idiom to compute a required number of data blocks of size B to contain data of length d is: (d + (B-1))/B The code we use, with ceill(), computes the same value, but does it in an unorthodox way. This makes a reviewer to doubt himself and even run tests to make sure we're really computing the obvious thing. Apropos the reviewer confusion, the code in Phazr.IO looks weird. It uses (word_size - hamming_distance) to compute the necessary number of blocks... but then returns the amount of memory needed to store blocks of a different size (word_size). We left all of it alone and return exactly the same values that the old computation returned. All these computations were the only thing in the code that used -lm, so drop that too. Coincidentially, this patch solves the crash of distro-built packages of liberasurecode (see Red Hat bug #1454543). But it's a side effect. Expect a proper patch soon. Change-Id: Ib297f6df304abf5ca8c27d3392b1107a525e0be0
* Merge "Jerasure: Handle initialization errors correctly"Jenkins2017-05-101-3/+7
|\
| * Jerasure: Handle initialization errors correctlyTim Burke2017-03-161-3/+7
| | | | | | | | | | | | | | Otherwise we can get backtraces where we try to free something that was never initialized. Change-Id: Iaea427b977fd20819e2da5678cc4889d3a42dd65
* | Merge "ISA-L: Only calculate gf tables on init, not every encode"Jenkins2017-03-291-10/+18
|\ \
| * | ISA-L: Only calculate gf tables on init, not every encodeDaniel Axtens2017-02-131-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the Galois Field multiplication tables are recalcuated every time an encode is done. This is wasteful, as they are fixed by k and m, which is set on init. Calculate the tables only once, on init. This trades off a little bit of per-context memory and creation time for measurably faster encodes when using the same context. On powerpc64le, when repeatedly encoding a 4kB file with pyeclib, this increases the measured speed by over 10%. Change-Id: I2f025aaee2d13cb1717a331e443e179ad5a13302 Signed-off-by: Daniel Axtens <dja@axtens.net>
* | | Merge "Add Phazr.IO libphazr backend to liberasurecode"Jenkins2017-03-019-11/+412
|\ \ \ | |_|/ |/| |
| * | Add Phazr.IO libphazr backend to liberasurecodeJim Cheung2017-02-289-11/+412
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, there are several implementations of erasure codes that are available within OpenStack Swift. Most, if not all, of which are based on the Reed Solomon coding algorithm. Phazr.IO’s Erasure Coding technology uses a patented algorithm which are significantly more efficient and improves the speed of coding, decoding and reconstruction. In addition, Phazr.IO Erasure Code use a non-systematic algorithm which provides data protection at rest and in transport without the need to use encryption. Please contact support@phazr.io for more info on our technology. Change-Id: I4e40d02a8951e38409ad3c604c5dd6f050fa7ea0
* | | jerasure: plug memory leaksDaniel Axtens2017-02-212-1/+44
| |/ |/| | | | | | | | | | | | | | | | | Jerasure inits some global variables on init. We need to free them, or we'll leak memory. Partial-Bug: #1666674 Change-Id: Ie7073738428a71910016e910a66dbd92ca98eb92 Signed-off-by: Daniel Axtens <dja@axtens.net>
* | ISA-L: free matrix on exitDaniel Axtens2017-02-101-0/+1
|/ | | | | | | | | | | isa_l_common_init allocates desc->matrix, but this isn't freed in isa_l_exit. Instead, the entire isa_l_desc structure is freed, thus leaking the memory. Explicitly free desc->matrix. Change-Id: Ibf672d1a309498591b87d739632a90a1b3704f7e Signed-off-by: Daniel Axtens <dja@axtens.net>
* ISA-L Cauchy supportKota Tsuyuzaki2016-12-063-545/+638
| | | | | | | | | | | | | | | | | | | | | | | | This is for supporting ISA-L cauchy based matrix. The difference from isa_l_rs_vand is only the matrix to use the encode/decode calculation. As a known issue, isa_l_rs_vand backend has constraint for the combinations of the available fragment to be able to decode/reconstuct. (See related change in detail) To avoid the constraint, this patch adds another isa-l backend to use cauchy matrix and keep the backward compatibility, this is in another isa_l_rs_cauchy namespace. For implementation consieration, the code is almost same except the matrix generation fucntion so that this patch makes isa_l_common.c file for gathering common fucntions like init/encode/decode/reconstruct. And then the common init funciton takes an extra args "gen_matrix_func_name" for entry point to load the fucntion by dlsym from isa-l .so file. Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Related-Change: Icee788a0931fe692fe0de31fabc4ba450e338a87 Change-Id: I6eb150d9d0c3febf233570fa7729f9f72df2e9be
* Fix error handling on gf_ivnert_matrix in isa-l backendKota Tsuyuzaki2016-11-091-2/+8
| | | | | | | | | | | | | | | | | | | | | | | Current isa-l has possibility to return corrupted decoded data or corrupted reconstructed data on decode/reconstruct without error code. That is from the specification of isa-l rs vandermond matrix discussed at [1]. With many # of parities cases, we may hit the case above due to failing to get the inverse matrix from the encode matrix. The isa-l maintener gbtucker suggests a good way to detect the failing inverse matrix, that we should handle the return value gf_invert_matrix. If gf_invert_matrix returns not 0, we should stop to decode/reconstruct and return failure return code to the caller immediately. Otherwise, the caller regards the garbage data/fragment as correct one. And this patch adds the specific test case we can hit the issue (it happens not so general). 1: https://github.com/01org/isa-l/issues/10 Related-Change: I6eb150d9d0c3febf233570fa7729f9f72df2e9be Change-Id: Icee788a0931fe692fe0de31fabc4ba450e338a87
* Ensure ec backends can actually be loaded.James Page2015-11-147-7/+7
| | | | | | | Use the actual soname rather than the fully unversioned name, ensuring that systems that don't have -dev packages actually work. Signed-off-by: James Page <james.page@ubuntu.com>
* Split helpers.h include for backward compatibilityTushar Gohad2015-09-225-0/+5
| | | | | | | Users of liberasurecode <= 1.0.7 used alloc/free helpers (which they shouldn't have). This change is to make sure we are still able to those older revs of programs and they work with newer liberasurecode.
* Rename remaining INTERNAL_RS_VAND instancesTushar Gohad2015-07-201-14/+14
| | | | ... to LIBERASURECODE_RS_VAND
* Rename liberasurecode_rsvand -> liberasurecode_rs_vandliberasurecode_rs_vand-renameTushar Gohad2015-06-221-3/+3
|
* Rename rs_vand_internal to liberasurecode_rs_vandTushar Gohad2015-06-221-75/+75
|
* Properly set W in the new internal RS backend. Without this, the fragment ↵new_commits_for_1_0_8Kevin Greenan2015-06-181-1/+1
| | | | | | length passed up is incorrect.
* Plugging new internal RS backend into liberasurecode.Kevin Greenan2015-06-181-0/+312
|
* Fix nasty rebuild bug where partiy would be reconstructed incorrectlyKevin Greenan2015-03-302-44/+72
| | | | | | | | when both data and parity was missing. The fix is to just call decode when reconstructing parity, since it will have to do extra work anyway when data is missing. We did a little extra work in ISA-L to do better, but can save that for later, since 99% of the time decode will perform just fine.
* Rename metadata_adder on backend_commonKota Tsuyuzaki2015-03-056-6/+6
| | | | This patch renames the "metadata_adder" variable to "backend_metadata_size"
* Fix segmentation_fault on shss with no priv_argsmetadata_adder_new_patches2Kota Tsuyuzaki2015-02-271-1/+6
|
* Fix isa-l backend to use correct word sizeKota Tsuyuzaki2015-02-271-1/+3
|
* Fix small thingsKota Tsuyuzaki2015-02-271-12/+7
| | | | | | | | Small fixes as follows: - Add is_compatible_with function into shss backend - Remove encoded data check against to shss at liberasurecode_test.c - Decrease metadata_adder size on shss backend to be correct fixed value
* Fix shss to instantiate backend discriptorKota Tsuyuzaki2015-02-261-14/+24
|
* Update isa-l backend to v2.13Tushar Gohad2015-02-251-46/+47
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* Add NTT backend called "shss"ntt_backendKota Tsuyuzaki2015-02-101-0/+296
| | | | | | | | | | | | | | This introduces a new plug-able backend called "shss" made by Nippon Telegraph and Telephone corporation (NTT). Note that this produces a just plug-in to shss erasure coding binary so that users have to install a shss binary (i.e. libshss.so) aside from liberasurecode when using shss. Please contact us if you are insterested in the NTT backend (welcome!): Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp> Co-Author: Ryuta Kon <kon.ryuta@po.ntts.co.jp>
* fixed mem leaks in rs_vand and xor backendEric Lambert2014-10-101-1/+1
|
* Fixed memory leak: freeing jerasure_rs_cauchy_descriptor was not completely ↵Eric Lambert2014-10-101-23/+47
| | | | freeing the schedule array
* backends now support verion checksEric Lambert2014-10-025-38/+130
|
* Add the ability for backends to specifyTushar Gohad2014-09-305-0/+5
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* Plugging-in the ISA-L backend... Still needs more documentation and a ↵Kevin Greenan2014-09-291-0/+591
| | | | little clean-up, but the tests are passing...
* A little bit of code clean up based on feedback/reviewsEric Lambert2014-09-151-8/+10
|
* Compile in pedantic mode and fail compilation on warnings. Also cleanup all ↵Eric Lambert2014-09-104-82/+165
| | | | existing warnings.
* Add doxygen config. Update README, copyrights.Tushar Gohad2014-09-014-4/+4
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* Fixed bug where null_init was not correctly setting w param (was relying on ↵Eric Lambert2014-08-261-0/+2
| | | | un-initialized memory which resulted in non-deterministic behavior).
* Cleaned up the Cauchy backend. This has a bit more cleaned than theMark Storer2014-08-251-44/+67
| | | | | | | | | other backends because Jerasure allocs more memory under the hood that we need to check on init, and cleanup on exit. Minor cleanup wise we do fewer memory derefs by saving the value of k, m and w in a few places. This is, however, probably most effective at just improving readability a bit.
* Minor cleanup to null backend. Mostly to the exit cases. Free checks forMark W. Storer2014-08-232-44/+32
| | | | | | | | null, so as long we intialize the pointers to NULL, we can be a little terser (and cleaner?) in the exits. Cleanup in the Jerasure Vandermonde backend. Primary change here was to free the memory allocated by Jerasure in the erasures_to_erased method.
* Backend changes needed to honor "excluded fragments".Kevin Greenan2014-08-214-9/+12
|
* Minor bug in getting the required fragments for jerasureKevin Greenan2014-08-082-0/+2
|
* Adding reconstruction for XOR codes.Kevin Greenan2014-08-051-1/+4
|
* Fixed reconstruction.Kevin Greenan2014-08-052-20/+25
|
* Add HD parameter to generic params.Kevin Greenan2014-08-051-1/+1
|
* Implemented fragments needed for jerasure backendsKevin Greenan2014-07-232-4/+41
|
* Oops... Yet another build fix.Kevin Greenan2014-07-231-2/+2
|
* Plugged-in Cauchy RS.Kevin Greenan2014-07-231-0/+302
|
* null_code: First cutTushar Gohad2014-07-221-0/+220
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* Mac OS X support in xor and jerasure backendsTushar Gohad2014-07-222-0/+8
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* flat_xor_hd: Fix xor_desc pointer (encode/decodes tests pass now)Tushar Gohad2014-07-221-5/+18
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
* Add encode postprocessing, checksum helpersTushar Gohad2014-07-211-17/+1
| | | | Signed-off-by: Tushar Gohad <tushar.gohad@intel.com>
View Lorry Controller Status