From f37439efd2fbd9a9e995d838da20d60337ca07f7 Mon Sep 17 00:00:00 2001 From: Michael Adam Date: Thu, 29 Oct 2009 00:01:45 +0100 Subject: s3:dbwrap_ctdb: fix race condition with concurrent transactions on the same node. In ctdb_transaction_commit(), when the trans2_commit control fails, there is a race condition in the 1 second sleep between the local transaction_cancel and the call to ctdb_replay_transaction(): The database is not locked, and neither is the transaction_lock record. So another client can start and possibly complete a new transaction in this gap, but only on the same node: The locking of the transaction_lock record on a different node which involves migration of the record to the other node has been disabled by introduction of the transaction_active flag on the db which closes precisely this gap from the start of the commit until the call to TRANS2_FINISH or TRANS2_ERROR. But this mechanism does not cover the case where a process on the same node tries to start a transaction: There is no obstacle to locking the transaction_lock record because the record does not need to be migrated. This commit closes this race condition in ctdb_transaction_fetch_start() by using the new ctdb_ctrl_transaction_active() call to ask the local ctdb daemon whether it has a transaction running on the database. If so, the check is repeated until the running transaction is done. This does introduce an additional call to the local ctdbd when starting transactions, but it does close the (hopefully) last race condition. Michael --- source3/lib/dbwrap_ctdb.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'source3/lib/dbwrap_ctdb.c') diff --git a/source3/lib/dbwrap_ctdb.c b/source3/lib/dbwrap_ctdb.c index ec52d766c06..fef984c8c31 100644 --- a/source3/lib/dbwrap_ctdb.c +++ b/source3/lib/dbwrap_ctdb.c @@ -347,6 +347,7 @@ static int db_ctdb_transaction_fetch_start(struct db_ctdb_transaction_handle *h) pid_t pid; NTSTATUS status; struct ctdb_ltdb_header header; + int32_t transaction_status; key.dptr = (uint8_t *)discard_const(keyname); key.dsize = strlen(keyname); @@ -362,6 +363,17 @@ again: } crec = talloc_get_type_abort(rh->private_data, struct db_ctdb_rec); + transaction_status = db_ctdb_transaction_active(ctx->db_id); + if (transaction_status == 1) { + unsigned long int usec = (1000 + random()) % 100000; + DEBUG(3, ("Transaction already active on db_id[0x%08x]." + "Re-trying after %lu microseconds...", + ctx->db_id, usec)); + talloc_free(tmp_ctx); + usleep(usec); + goto again; + } + /* * store the pid in the database: * it is not enought that the node is dmaster... -- cgit v1.2.1