summaryrefslogtreecommitdiff
path: root/script
Commit message (Collapse)AuthorAgeFilesLines
* traffic_replay: use packets per second as primary scaleDouglas Bagnall2019-01-081-3/+16
| | | | | | | | The old -S/--scale-traffic is relative to the original model, which made its relationship to true traffic volumes quite opaque Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: --old-scale to mimic the old traffic_replayDouglas Bagnall2019-01-081-0/+10
| | | | | | | | | traffic_replay had a broken sense of traffic scale. That is fixed, but in order to compare old and new tests, it helps to be able to approximate the old behaviour. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: add option to reanimate dying conversationsDouglas Bagnall2019-01-081-1/+6
| | | | | | | | | | | | | | | The traffic model is generated from a window in time, which makes conversations appear to start and stop unnaturally at the window boundaries. When the window is short compared to the traffic replay time and the true expected conversation length, this has a significant distorting effect, leading to more conversations than would be expected to generate a given number of packets. To offset this slightly we add the --conversation-persistence option which tries to convert apparent death into a longish wait. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_model: don't report generation errors as parse errorsDouglas Bagnall2019-01-081-16/+10
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: improve debug messages in traffic_replayDouglas Bagnall2019-01-081-7/+6
| | | | | | | tracebacks and less nonsense at higher debug levels. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: traffic_replay --latency-timeout to control final waitDouglas Bagnall2019-01-081-0/+3
| | | | | | | | | Conversations that haven't finished within some acceptable margin of on-time can be said to have failed. This is where you specify that margin. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: --stop-on-any-error option to not ignore client troubleDouglas Bagnall2019-01-081-1/+4
| | | | | | | | Sometimes you want to know if any client is crashing for any reason. In those times use --stop-on-any-error for an early exit. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: rework conversation generation to better use memoryDouglas Bagnall2019-01-081-1/+1
| | | | | | | | Use less memory altogether and don't allocated shared mutable before the fork. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: generate sparser descriptions of conversationsDouglas Bagnall2019-01-081-6/+7
| | | | | | | | | | | | | Rather than building all the packets at this point, we stick to the barest details of the packets (which is all the model gives us anyway). The advantage is that will take a lot less memory, which matters because this process forks into many clients that were sharing and mutate the conversation list. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: new version of model with packet_rate, version numberDouglas Bagnall2019-01-081-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: delete empty temp directoriesDouglas Bagnall2019-01-081-1/+10
| | | | | | | | | even if someone asked to keep the temporary data, they don't want to see 5000 empty directries. Non-empty directories will remain. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic learner: avoid truncated output files on errorDouglas Bagnall2019-01-081-1/+8
| | | | | | | | add_argument(type=argparse.FileType('w'), ...) will open the file and leave it empty if the script fails. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_learner: use samba.logger, not print(file=sys.stderr)Douglas Bagnall2019-01-081-4/+8
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_learner: return an error codeDouglas Bagnall2019-01-081-4/+3
| | | | | | | And use it in tests, rather than expecting exact strings. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: use default value for --durationDouglas Bagnall2019-01-081-7/+3
| | | | | | | | We could not do this when we replayed traffic summaries (as opposed to models), but now this script does not do that. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: drop summary replayDouglas Bagnall2019-01-081-36/+22
| | | | | | | | | | The traffic_replay script has been able to replay a replay log as well as a model, which was not used in practice and complicated the script. If we want that feature, we can make a new script for it. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic replay: allow --random-seed=0Douglas Bagnall2019-01-081-2/+2
| | | | | | | Zero is a valid seed. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* PY3: change shebang to python3 in script dirJoe Guo2018-12-144-4/+4
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* autobuild: convert top_commit_msg to str for Py 3Douglas Bagnall2018-12-121-0/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: py3: cope with bytes when compiling system-info.txtDouglas Bagnall2018-12-121-3/+8
| | | | | | | The command output looks like b'foo\nbar' in string-space. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: We should run autobuild with python3 by defaultNoel Power2018-12-101-1/+1
| | | | | | | | | | | sn-devel autobuild runing autobuild.py (via git hooks I suppose) but if run directly (e.g. depending on script shebang) then 'python' aka python2 will run. This will cause an error when building some targets because the autobuild script itself sometimes builds paths based on the version of python executing the script e.g ${PYTHON_PREFIX}. Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Convert old py3 tasks to py2Noel Power2018-12-101-8/+8
| | | | | | | | | Now that we are building with python3 by default we need to convert the old python3 test tasks to python2 (e.g. reverse how we used do it) Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Adjust autobuild for PY2/PY3 get_python_lib behaviourNoel Power2018-12-101-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The resuls of get_python_lib are different between python2 & python3 and this results in autobuild generating the wrong PYTHONPATH with python3. python2 ======= print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix')) /my/prefix/lib64/python2.7 python3 print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix')) /my/prefix/lib/python3.6 But with addition of plat_specific param the results are the same python2 ======= print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix')) /my/prefix/lib64/python2.7/site-packages python3 ======= print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix')) /my/prefix/lib64/python3.6/site-packages Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Modify old samba_buildpy3_only job to python2Noel Power2018-12-101-6/+6
| | | | | | | | Since autobuild now builds python3 by default we need to change the previously buildpy3 only job to python2 Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script: Fix failing build_xc jobNoel Power2018-12-101-0/+14
| | | | | | | | | | | | | build_xc job uses compare_cc_results.py to compare cache files, the cache files are stringified hash maps, the results in python 3.4 don't compare well due to inconsistent order of dict key/value pairs when the cache files are created. While comparing the file contents works fine in python3.6 it fails with python3.4. This patch detects problematic dict lines and rewrites the value for comparison Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Remove temporary purepy3 tasksNoel Power2018-12-101-82/+0
| | | | | Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* CI: Add new (TEMP) pure python3 autobuild jobs for samba-build & samba-nt4Noel Power2018-12-101-0/+39
| | | | | Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script: PY3 port traffic_learnerNoel Power2018-12-101-5/+5
| | | | | | | Use python3 compatable print Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script: Add new (temporary) pure python3 ad-dc testNoel Power2018-12-101-0/+15
| | | | | | | | | | | Ideally we want all the tests to run under python3 by default (no special task for this) and then convert the existing '-py3' tasks to run the python tests with python3. However at the moment the convertion process is not ready to do this, for a while we need to run separate autobuild tasks for this. Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Add a max-members option to cap group sizeTim Beale2018-12-041-0/+4
| | | | | | | | | | | | | | | | | | | | | | | traffic_replay tries to distribute the users among the groups in a realistic manner - some groups will have almost all users in them. However, this becomes a problem when testing a really large database, e.g. we may want 100K users, but no more than 5K users in each group. This patch adds a max-member option so we can limit how big the groups actually get. If we detect that a group exceeds the max-members, we reset the group's probability (of getting selected) to zero, and then recalculate the cumulative distribution. The means that the group should no longer get selected by generate_random_membership(). (Note we can't completely remove the group from the list because that changes the list-index-to-group-ID mapping). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Tue Dec 4 12:22:50 CET 2018 on sn-devel-144
* autobuild: Add _FORTIFY_SOURCE=2 to the -O3 buildAndreas Schneider2018-11-281-1/+1
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* script: Add new (temporary) pure python3 ad-dc-2 testNoel Power2018-11-141-0/+18
| | | | | | | | | | | Ideally we want all the tests to run under python3 by default (no special task for this) and then convert the existing '-py3' tasks to run the python tests with python3. However at the moment the convertion process is not ready to do this, for a while we need to run separate autobuild tasks for this. Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Rework machine accounts to remove redundant codeTim Beale2018-11-071-0/+1
| | | | | | | | | | | | | | | generate_users_and_groups() now generates the machine acounts as well as the user accounts, so it seems there's no need to also have generate_traffic_accounts(), which does the same job. Instead, we can just pass through the number of machine acounts to generate_users_and_groups() and delete the other function. Also updated generate_users_and_groups() so that machine_accounts is no longer optional (we want to create machine accounts in all cases). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Move 'traffic account' flag up a levelTim Beale2018-11-071-2/+4
| | | | | | | | | | | | | | | | | | We create machine accounts for 2 different purposes: 1). For traffic generation, i.e. testing realistic network packets. 2). For generating a realistic large DB. Unfortunately, we want to use different userAccountControl flags for the 2 different cases. Commit 3338a3e257fa9f28 changed the flags used for case #2, but this breaks case #1. The problem is generate_users_and_groups() is called in both cases, so we want the 'traffic account' flag passed into that function. This ensures that the machine accounts get created with the appropriate userAccountControl flags for the particular case you want to test. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Move machine account creationTim Beale2018-11-071-1/+5
| | | | | | | | | | | | | I was assuming that generate_users_and_groups() only gets called in the --generate-users-only case. However, it also gets called in the default traffic replay case. This patch reworks the code so that the number of machine accounts to create gets passed in, and the 'create 25% more computers than users' assumption only applies to the --generate-users-only case. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: remove unused variablesDouglas Bagnall2018-11-011-2/+0
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <noel.power@suse.com>
* traffic_replay: Exception has no .messageDouglas Bagnall2018-11-011-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <noel.power@suse.com>
* script/show_test_time: attempt py3 compatDouglas Bagnall2018-11-011-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <noel.power@suse.com>
* traffic_replay: logger was ignoring smb.conf log-levelTim Beale2018-10-311-1/+1
| | | | | | | | | We were trying to access the debug-level (in python C bindings) before the smb.conf had been loaded and actually set the debug-level. So it would default to zero, regardless of what was in the smb.conf. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* traffic_replay: Generate users faster by writing to local DBTim Beale2018-10-311-1/+11
| | | | | | | | | | | | | | | | | | | | We can create user accounts much faster if the LDB connection talks directly to the local sam.ldb file rather than going via LDAP. This patch allows the 'host' argument to the tool to be a .ldb file (e.g. "/usr/local/samba/private/sam.ldb") instead of a server name/IP. In most cases, the traffic_replay tool wants to run on a remote device (because the point of it is to send traffic to the DC). However, the --generate-users-only is one case where the tool can be run locally, directly on the test DC. (The traffic_replay user generation is handy for standalone testing, because it also handles assigning group memberships to the generated user accounts). Note that you also need to use '--option="ldb:nosync = true"' to get the improvement in performance. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* script/show_test_time: approach python 3 compatibilityDouglas Bagnall2018-10-251-3/+3
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* script/generate_param.py: remove unused importsDouglas Bagnall2018-10-251-4/+0
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* CI/Autobuild: Remove samba-none-env-py3 testNoel Power2018-10-231-1/+0
| | | | | | | | | | | | We now run a purepython3 none-env test, later when the whole build is running under python3 we will resurrect this job but as (samba-none-env-py2) for python2 Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144
* script: Add new (temporary) autobuild task for none-env tests.Noel Power2018-10-231-0/+13
| | | | | | | | | | | Ideally we want all the tests to run under python3 by default (no special task for this) and then convert the existing '-py3' tasks to run the python tests with python3. However at the moment the convertion process is not ready to do this, for a while we need to run separate autobuild tasks for this. Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/traffic_replay: apply new logger to replace printJoe Guo2018-10-101-51/+57
| | | | | | | | Use logger to replace print Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/traffic_replay: get debug level via apiJoe Guo2018-10-101-4/+5
| | | | | | | | | | | The -d option will set samba global debug level automatically. We should not parse and use the passed in value. Use samba.get_debug_level instead. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/traffic_replay: print packets data to stderrJoe Guo2018-10-101-2/+2
| | | | | | | | | This is debug info, should print to stderr. Otherwise it will flood stdout. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* third_party: Add pam_set_items.so from pam_wrapperMathieu Parent2018-10-021-0/+1
| | | | | | | Pair-Programmed-With: Andreas Schneider <asn@samba.org> Signed-off-by: Mathieu Parent <math.parent@gmail.com> Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Guenther Deschner <gd@samba.org>
* script: add new autobuild task for building pure python3Noel Power2018-09-281-0/+8
| | | | | Signed-off-by: Noel Power <noel.power@suse.com> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* turn --with-json-audit into global --with-jsonPhilipp Gesang2018-09-261-1/+1
| | | | | | | | | | Fold the build option --with-json-audit into the toplevel wscript to reflect the fact that JSON support is no longer local to the audit subsystem. Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
View Lorry Controller Status