Tune gc for 25% speed-up of include processor.

Set first parameter of gc.set_threshold to 10000 instead of default 700.

See comments in program text.


Notes
-----

I used a little script to find the include server timings as a function of this
parameter.

This script executes:

"DISTCC_PUMP_INSTALLATION=/tmp/distcc3.0 INCLUDE_SERVER_ARGS='-t --gc_threshold=700' \
   /usr/bin/time -p make-dbg -j120 -r -experimental-ld -g0 gws:gws"

with various values of gc_threshold.

The measurement were done on a four processor system (dual Opterons), loaded
with an nxserver process. Files were served through Fuse.


gc       include server user + sys time
setting

   700   155.1s 154.0s 164.7s 186.1s 154.3s Mean: 162.8
 10000   123.8s 125.4s 120.6s 120.4s 121.0s Mean: 122.2
100000   123.4s 123.3s 122.2s 123.3s 116.5s Mean: 121.7

   700   166.5s 156.4s 150.8s 153.4s 155.4s Mean: 156.5
 10000   120.7s 142.1s 118.3s 117.7s 119.2s Mean: 123.6
100000   140.3s 117.1s 149.7s 116.7s 141.6s Mean: 123.6


The corresponding elapsed times:

         371.0s 363.3s 361.9s 370.8s 369.0s Mean: 367.2
         341.8s 346.1s 336.3s 338.3s 340.5s Mean: 340.6
         345.9s 338.5s 342.9s 374.4s 340.0s Mean: 348.3

         377.4s 379.9s 387.1s 377.4s 373.2s Mean: 379.0
         332.1s 337.7s 336.5s 332.2s 314.7s Mean: 330.6
         341.1s 334.6s 357.8s 338.5s 352.4s Mean: 344.9

Note that the saving in include server time is around 35s, but the savings in
elapsed time is around 44s. These numbers are uncertain.  Even so, they strongly
indicate that the include server is a signifcant bottleneck.

BTW, for comparison with plain distcc:

         821.4s 822.0s 842.4s 821.5s 852.0s Mean: 831.9
         878.7s 857.2s 860.3s 880.1s 856.0s Mean: 866.5

Thus, pump-mode 2.5X is faster than plain distcc (with the set_threshold =
10000 of this CL).

Revievers: fergus@google.com, csilvers@google.com

2 files changed, 15 insertions, 0 deletions

diff --git a/include_server/basics.py b/include_server/basics.py
index 8f5d856..e86b218 100755
--- a/include_server/basics.py
+++ b/include_server/basics.py
@@ -184,6 +184,16 @@ SIMPLE = 0     # not implemented
 MEMOIZING = 1  # only one currently implemented
 ALGORITHMS = [SIMPLE, MEMOIZING]
 
+# PYTHON TUNING
+
+# The default for the first parameter of gc.set_threshold is 700; see
+# http://www.isi.edu/~chiang/python.html for a discussion of why this parameter
+# can be bumped up considerably for speed-up.  The new default of 10000 was
+# tested on a very large application, where include server time CPU time drops
+# from 151s to 118s (best times out of 10 runs). There was no seeming changes to
+# memory usage.  Trying with 100,000 did not speed up the application further.
+GC_THRESHOLD = 10000  
+
 
 # FLAGS FOR COMMAND LINE OPTIONS
 
diff --git a/include_server/include_server.py b/include_server/include_server.py
index 23befbe..45e348a 100755
--- a/include_server/include_server.py
+++ b/include_server/include_server.py
@@ -24,6 +24,7 @@ __author__ = "Nils Klarlund"
 # ppid is 0; if so, then abort.
 
 # Python imports
+import gc
 import os
 import re
 import sys
@@ -628,6 +629,10 @@ def Main():
     include_server_port_ready.Release()
     try:
       try:
+        gc.set_threshold(basics.GC_THRESHOLD)
+        # Use commented-out line below to have a message printed for each
+        # collection.
+        # gc.set_debug(gc.DEBUG_STATS + gc.DEBUG_COLLECTABLE)
         server.serve_forever()
       except KeyboardInterrupt:
         print >> sys.stderr, (