dsdb/modules: a module to count attribute searches and results

The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.

The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:

          "subtree_delete",
          "repl_meta_data",
  +       "count_attrs",
          "group_audit_log",
          "encrypted_secrets",

and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:

./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb

will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).

A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.

When the module is included in the stack there will be a very small
decrease in performance.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

1 files changed, 27 insertions, 0 deletions

diff --git a/script/attr_count_read b/script/attr_count_read
new file mode 100755
index 00000000000..69c7c63e729
--- /dev/null
+++ b/script/attr_count_read
@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+
+import sys
+import struct
+sys.path.insert(0, "bin/python")
+import tdb
+
+
+def main(filename):
+    db = tdb.Tdb(filename)
+    pairs = []
+    longest = 0
+    for k in db:
+        v = struct.unpack("I", db[k])[0]
+        pairs.append((v, k.decode('utf-8')))
+        longest = max(len(k), longest)
+
+    pairs.sort()
+    for v, k in pairs:
+        print("%*s: %7d" % (longest, k,  v))
+
+
+if len(sys.argv) < 2:
+    print("Usage: attr_count_read <tdb-file>")
+    sys.exit(1)
+
+main(sys.argv[1])