summaryrefslogtreecommitdiff
path: root/lib/tdb
Commit message (Collapse)AuthorAgeFilesLines
* tdb: Release tdb 1.4.2Stefan Metzmacher2019-08-212-1/+74
| | | | | | | | | | | | | | | | * Build fixes * Improve the performance by inlining the tdb_oob() checks Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org> Autobuild-User(master): Stefan Metzmacher <metze@samba.org> Autobuild-Date(master): Tue Aug 20 14:45:41 UTC 2019 on sn-devel-184 (cherry picked from commit 60cba7b3a17104da1543d59609f50c6638880dd1) Autobuild-User(v4-11-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-11-test): Wed Aug 21 09:57:08 UTC 2019 on sn-devel-184
* tdb: Inline the common part of tdb_oobVolker Lendecke2019-08-212-9/+15
| | | | | | | | | | | | | | | When you set in tdbtorture.c to make it more similar to locking.tdb use, bin/tdbtorture -m -n 1 -l 100000 -s becomes twice as fast. This is a pretty extreme case, but all other tests that I did improve significantly as well. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> (cherry picked from commit f5735e2c666a5a494131c1d25f7ba5c7fbeae923)
* tdb: Speed up tdb_oob()Volker Lendecke2019-08-212-1/+17
| | | | | | | | | This is common between both implementations of tdb_oob(). It's faster if we don't have to dereference function pointers. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> (cherry picked from commit 897bffa8166f643eb9063a848bb0c02455663317)
* tdb: Introduce tdb_oob()Volker Lendecke2019-08-218-20/+26
| | | | | | | | Initially just encapsulate the pointer dereferences Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> (cherry picked from commit 5a388453e0cb038fa3ed5fb46f972470f7793566)
* tdb: Rename tdb_oob() to tdb_notrans_oob()Volker Lendecke2019-08-212-4/+4
| | | | | | | | | tdb_oob() will become a public function encapsulating the pointer dereferences. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> (cherry picked from commit 885ba572efaac6c20388b8e119315c837e8f5236)
* tdb: Adapt _tdb_transaction_cancel() to README.CodingVolker Lendecke2019-07-031-1/+1
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Noel Power <npower@samba.org>
* tdb: Adapt tdb_rescue() to README.CodingVolker Lendecke2019-07-031-1/+1
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Noel Power <npower@samba.org>
* lib tdb: memcmp ubsan warningGary Lockyer2019-07-011-5/+7
| | | | | | | | | | | | | | | | | Fix the ubsan warning lib/tdb/common/tdb.c:184:9: runtime error: null pointer passed as argument 2, which is declared to never be null" memcmp call now guarded by a length check. memcmp returns zero when called with a zero length parameter. Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org> Autobuild-User(master): Noel Power <npower@samba.org> Autobuild-Date(master): Mon Jul 1 14:50:54 UTC 2019 on sn-devel-184
* lib/tdb/common: Fix Array access results in a null pointer dereferenceNoel Power2019-07-011-1/+2
| | | | | | | | | | | | Fixes; lib/tdb/common/transaction.c:613:7: warning: Array access (via field 'blocks') results in a null pointer dereference <--[clang] if (tdb->transaction->blocks[i] != NULL) { ^ 1 warning generated. Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* lib/tdb/common: Fix warning: Null pointer passed as argument to paramNoel Power2019-07-011-2/+4
| | | | | | | | | | | Fixes: lib/tdb/common/rescue.c:299:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang] qsort(found.arr, found.num, sizeof(found.arr[0]), cmp_key); ^ ~~~~~~~~~ Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* clang: Fix Null pointer passed as argument warningNoel Power2019-06-261-1/+1
| | | | | | | | | | Fixes: lib/tdb/common/transaction.c:354:2: warning: Null pointer passed as an argument to a 'nonnull' parameter <--[clang] memcpy(tdb->transaction->blocks[blk] + off, buf, len); & Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* lib/tdb: clang: Fix warning: Dereference of null pointerNoel Power2019-06-261-0/+3
| | | | | | | | | | | Fixes: lib/tdb/common/lock.c:933:6: warning: Dereference of null pointer <--[clang] if (tdb->allrecord_lock.count) { ^~~~~~~~~~~~~~~~~~~~~~~~~ Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* py3: Remove PyStr_FromFormat() compatability macroAndrew Bartlett2019-06-241-4/+1
| | | | | | | | We no longer need Samba to be py2/py3 compatible so we choose to return to the standard function names. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Noel Power <noel.power@suse.com>
* py3: Remove PyStr_FromString() compatability macroAndrew Bartlett2019-06-241-3/+1
| | | | | | | | We no longer need Samba to be py2/py3 compatible so we choose to return to the standard function names. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Noel Power <noel.power@suse.com>
* lib/tdb/common: clang: Fix 'Value stored to 'last_ptr' is never read'Noel Power2019-06-111-1/+0
| | | | | | | | | | | | | | Fixes lib/tdb/common/freelistcheck.c:96:3: warning: Value stored to 'last_ptr' is never read <--[clang] last_ptr = rec_ptr; ^ ~~~~~~~ Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Gary Lockyer gary@catalyst.net.nz Autobuild-User(master): Noel Power <npower@samba.org> Autobuild-Date(master): Tue Jun 11 13:31:01 UTC 2019 on sn-devel-184
* tdb: Fix typo in READMEVolker Lendecke2019-05-231-2/+2
| | | | | | | | README is far from a complete, good and accurate document, but what's in there should at least not have obvious errors. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Release tdb 1.4.1tdb-1.4.1Andrew Bartlett2019-05-172-1/+74
| | | | | | | | | | | | | * Do not propogate any errors from tdb_repack() to to tdb_transaction_commit() The repack may fail due to lock ordering or memory limits, but the original data has been written so the call must succeed. (bug 13952) Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Fri May 17 08:21:52 UTC 2019 on sn-devel-184
* tdb: Do not return errors from tdb_repack() in the tail of ↵Andrew Bartlett2019-05-171-1/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tdb_transaction_commit() The call to tdb_repack() inside tdb_transaction_commit() is an optimization, not part of the transaction itself, so failing due to lock or other errors isn't a fatal error that should cause the caller to think the transaction was a failure by returning -1. The tdb transaction itself has finished and been committed onto stable storage via fsync and all locks released at the point tdb_repack() is called. tdb_repack() is only called here as it's a convenient point to attempt to reduce tdb fragmentation without having to add a timer call to repack in all users of tdb. This causes lock ordering issues in Samba, showing up as: ldb: ltdb: tdb(../private/sam.ldb.d/DC=SAMBA2008R2,DC=EXAMPLE,DC=COM.ldb): tdb_transaction_prepare_commit: failed to upgrade hash locks: Locking error This is because Samba has multiple tdb databases open, and the lock order between them is important. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13952 Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* lib/tdb: squash 'cast between incompatible function types' warningNoel Power2019-05-161-20/+40
| | | | | | | | | | | | | | squash 'cast between incompatible function types' warning To avoid warning above produced by using -Wcast-function-type we; + ensure PyCFunctions of type METH_NOARGS defined dummy arg + ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG macro Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andreas Schneider <asn@samba.org>
* tdbtool: avoid theoretical NULL dereference (CID 1361462)Douglas Bagnall2019-05-091-9/+11
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* tdb: Release tdb 1.4.0tdb-1.4.0Andrew Bartlett2019-03-212-1/+74
| | | | | | | | | | | * Remove Python 2.x support except to build just the bare C library * Remove --extra-python (a build time mode to produce Python2 and Python3 bindings at the same time) * New minor version to allow Samba 4.10 to release a tdb if required from that branch Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* build: Standardise on calling conf.SAMBA_CHECK_PYTHON() in librariesAndrew Bartlett2019-03-211-8/+2
| | | | | | | | | | | | | We do this by removing the confusing mandatory option to conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of --disable-python internally This follows the default minimum of Python 3.4 and keeps things consistent with the main Samba build where --disable-python is required to skip building python bindings. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* build: Remove bld.gen_python_environments()Andrew Bartlett2019-03-211-13/+11
| | | | | | | This was part of --extra-python support. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* tdb: version 1.3.18tdb-1.3.18Stefan Metzmacher2019-02-262-1/+74
| | | | | | | | * Fix build problems with older python versions. * C99 build fixes. * Fix standalone build of tdb. Signed-off-by: Stefan Metzmacher <metze@samba.org>
* Search for location of waf scriptDavid Mulder2019-02-261-1/+2
| | | | | | | | | | | | When calling make from the ldb, talloc, tdb, and tevent bundles, we need to first find the location of the waf script. Currently the build fails since it can't find waf. Fixes regression caused by a660b7f. Signed-off-by: David Mulder <dmulder@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* lib:tdb: Use C99 initializer for tdb_logging_contextAndreas Schneider2019-01-281-1/+3
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* lib:tdb: Use C99 initializer for tdb_headerAndreas Schneider2019-01-281-1/+3
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* lib:tdb: Use C99 initializer for PyGetSetDef in pytdbAndreas Schneider2019-01-281-18/+34
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* tdb: Fix compatibility of wscript with older pythonLukas Slebodnik2019-01-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Traceback (most recent call last): File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 158, in waf_entry_point run_commands() File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 251, in run_commands ctx = run_command(cmd_name) File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 235, in run_command ctx.execute() File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 204, in execute self.recurse([os.path.dirname(g_module.root_path)]) File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 286, in recurse user_function(self) File "tdb-1.3.17/wscript", line 225, in testonly cmd = "BINDIR={} {}".format(blddir, sh_test) ValueError: zero length field name in format Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Sun Jan 20 03:49:59 CET 2019 on sn-devel-144
* Fix typos and ban the rams from sambatdb-1.3.17Swen Schillig2019-01-101-1/+1
| | | | | | Signed-off-by: Swen Schillig <swen@linux.ibm.com> Reviewed-by: Jeremy Allison <jra@samba.org> Reviewed-by: Volker Lendecke <Volker.Lendecke@SerNet.DE>
* PY3: change shebang to python3 in lib dirJoe Guo2018-12-142-2/+2
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* build: Move python detection back into waf (instead of in configure and ↵Andrew Bartlett2018-12-142-2/+1
| | | | | | | | | | | | | | | | | | | Makefile) This avoids creating a mini-configure in the configure script. Users wishing to use python2 to build need to specify PYTHON= to both ./configure and make After we merged the python3 change, it became clear that relying on systems prefixing the correct python just causes trouble and make debugging harder, so only use $PYTHON for the override, not the default case This essentially reverts a660b7fb8e519bd3be558fd0425bff8f287fca1f but leaves the files more consistent. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Noel Power <npower@samba.org>
* build: Workaround python3 hash order issues (for now)Andrew Bartlett2018-12-131-1/+1
| | | | | | | | | | | This works around python3 having a new hash seed each time it starts to allow a second "make" not to rebuild the world. This should probably be reverted once we find the hash that is causing the issue, but should reduce frustration for now. Signed-off-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* PY3: switch current build to use python3Noel Power2018-12-102-2/+4
| | | | | | | | | | | | | | Make sure default make and configure for all now defaults to building with python3. To build a samba (or sub component e.g. talloc etc.) with python3 ./configure && make To build a samba (or sub component e.g. talloc etc.) with python2 PYTHON=python ./configure && PYTHON=python make Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Fix spelling mistakesOlly Betts2018-11-301-1/+1
| | | | | | Signed-off-by: Olly Betts <olly@survex.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* tdb: Use #ifdef instead of #if for config.h definitionsAndreas Schneider2018-11-281-2/+2
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* tdb: Align integer typesVolker Lendecke2018-11-061-1/+1
| | | | | | | | | | | | | tdb->max_dead_records is "int", as is the corresponding parameter to tdb_set_max_dead(). Not that a signed variable makes any sense, but this is old code and tdb_set_max_dead() is a public API which we should not change for this. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Tue Nov 6 21:52:32 CET 2018 on sn-devel-144
* tdb: Allow !CLEAR_IF_FIRST & MUTEX_LOCKINGVolker Lendecke2018-11-062-56/+42
| | | | | | | | This is a prerequisite to allow gencache to run on a non-transactioned database with mutexes. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Version 1.3.17 for tdb_traverse_chainVolker Lendecke2018-11-062-1/+74
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Add test for tdb_traverse_chainVolker Lendecke2018-11-062-0/+95
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Add tdb_traverse_chainVolker Lendecke2018-11-062-0/+177
| | | | | | | | This is a lightweight readonly traverse of a single chain, see the comment in the header file. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Make record deletion circular-chain safeVolker Lendecke2018-10-303-102/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this patch we had 3 loops walking a hash chain to delete records: tdb_do_delete() to find the predecessor of the record that was to be deleted. tdb_count_dead(), the name says it all and tdb_purge_dead() to give back all dead records from a chain to the freelist. This patch introduces tdb_trim_dead that walks a hash chain just once. While it does so it counts the number of dead records, and all records beyond tdb->max_dead_records are moved to the freelist. Normal record deletion now works by always marking a record as dead in step 1 and then calling tdb_trim_dead. This is made safe against circular chains by doing the slow chain walk only in the case when we did not delete a dead record during our walk. It changes our dynamics a bit: When deleting a record with non-zero max_dead_records, now we always leave that number of records around when deleting, doing a blocking lock on the freelist when we found too many dead records. Previously when exceeding max_dead_records we wiped all dead records to start accumulating them from scratch, assuming we could lock the freelist in a nonblocking fashion. The net effect for an uncontended freelist is the same: In tdb_allocate() we still completely hand over all dead records to the freelist when we could lock it, it just happens later than without this patch. This means for a lightly loaded system we will potentially leave more dead records around in databases like locking.tdb. However, on a heavily loaded system we become more predictable: If the freelist is so heavily contended that across many deletes we can't get hold of it, previously we accumulated more dead records than max_dead_records would allow. This is a really lowlevel tradeoff that is likely hard to measure, but to me becoming more deterministic without sacrificing too much parallelism (we keep more dead records around) is worth trying. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144
* tdb: Do early RDONLY error check for tdb_deleteVolker Lendecke2018-10-291-0/+5
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Purge dead records whenever we block the freelistVolker Lendecke2018-10-291-0/+6
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Don't delete dead records in traverseVolker Lendecke2018-10-291-6/+0
| | | | | | | | | | The next commit will change the handling of dead records, removing the "tdb_do_delete" function. As traverses should not happen in normal operations, dead records from them should be rare, and relying on traverses to remove them is a very bad idea IMHO. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Align an integer typeVolker Lendecke2018-10-291-1/+1
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Fix a typoVolker Lendecke2018-10-291-2/+2
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb/test/_tdbtext: remove unused importsDouglas Bagnall2018-10-251-1/+0
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* tdbtorture: Align integer typesVolker Lendecke2018-10-251-1/+1
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* tdbtorture: No transaction with mutexesVolker Lendecke2018-10-251-5/+16
| | | | | | | | | | Right now we don't do transactions with mutexed tdbs. tdbtorture -m locks up. I haven't really investigated why that is the case. The lockup confused me quite a bit until I figured out it works fine as long as it does not do transactions, which is all we need right now. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
View Lorry Controller Status