summaryrefslogtreecommitdiff
path: root/script
Commit message (Collapse)AuthorAgeFilesLines
* script/compare_cc_results: print usage on too few argsDouglas Bagnall2019-07-051-3/+9
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/bisect-test: fix for py3Douglas Bagnall2019-07-021-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: make code more pythonicJoe Guo2019-06-201-43/+31
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: avoid nested try except blockJoe Guo2019-06-201-11/+12
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: rm redundant empty str in optionsJoe Guo2019-06-201-20/+20
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: define cwd for builder and avoid chdirJoe Guo2019-06-201-5/+3
| | | | | | | | We can pass cwd in Popen, no need to chdir for each cmd. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: improve run_cmd with check_outputJoe Guo2019-06-201-4/+4
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: add helper functions to simplify make test cmdJoe Guo2019-06-201-120/+131
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: replace random-sleep.sh with python functionJoe Guo2019-06-202-58/+39
| | | | | | | | | | | | | The random-sleep.sh script is tricky, and there is no need to rely on another script just for random sleep. Using the python function, the random number generation will happen when load autobuild.py other than execute cmd, but shouldn't affect the result. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: rm unused mime type text/plain in tasksJoe Guo2019-06-201-294/+288
| | | | | | | | The mime type `text/plain` is repeated everywhere but not used any more. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: fix tasks indentJoe Guo2019-06-201-430/+467
| | | | | | | | Indent tasks at same level, make it easier to copy/move lines arround. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Fix autobuild for python3Volker Lendecke2019-06-181-1/+1
| | | | | | | | | "keys()" in python3 returns an unmodifiable view object. Two lines down we might want to modify it, which python3 does not allow. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Noel Power <npower@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* script/autobuild.py: define LCOV_CMD and run it after each make testJoe Guo2019-05-101-2/+35
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: add ENABLE_COVERAGE placeholder in configure cmd optionJoe Guo2019-05-101-10/+10
| | | | | | | The value will be "--enable-coverage" or "". Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: replace more placeholders in cmds for coverageJoe Guo2019-05-101-0/+4
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: add --enable-coverage optionJoe Guo2019-05-101-0/+3
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: mv optionparse to topJoe Guo2019-05-101-53/+53
| | | | | | | so the rest of the code can use the option values directly. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: mv find_git_root and gitroot to topJoe Guo2019-05-101-14/+16
| | | | | | | so the reset of the code can use gitroot directly. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: rename sdir to test_source_dirJoe Guo2019-05-101-7/+7
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/attr_count_read: load and correlate all dataDouglas Bagnall2019-05-011-11/+183
| | | | | | | | | | | | | | | | | | | | | | | This changes script/attr_count_read to take the samba private directory as an argument and load all the databases at once, printing them as one big table. It isn't extremely clear what it all means, but it *tries* to tell you. With --plot, it will attempt to load matplotlib and plot the number of requested attributes against the number returned, with colour of each point indicating its relative frequency. It is a scatterplot that wants to be a heatmap. With --no-casefold, you can get an extra confusing table where, for instance, something repeatedly asks for "attributeId" which is not accounted for, while in a completely different row an unrequested "attributeID" is found many times over. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 1 06:46:36 UTC 2019 on sn-devel-184
* dsdb/modules: a module to count attribute searches and resultsDouglas Bagnall2019-05-011-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The dsdb module stack can turn a simple search request into a complicated tree of sub-queries that include attributes not originally asked for and excluding those that were. The corresponding replies might contain unrequested attributes or (for good reasons, according to some module) hide requested ones. The entire stack is there to meddle and that is what is does. Except *this* module. It just counts. To understand dsdb performance it helps to have some idea what requests and replies are flying too and fro. This module, when inserted anywhere in the stack, counts the requests and replies passing through and the attributes they contain. This data is stored in on-disk tdbs in the private/debug directory. The module is not loaded by default. To load it you need to patch the source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs" somewhere in the module lists in the samba_dsdb_init() function. For example, to examine the traffic between repl_meta_data and group_audit_log, you would do something like this around line 316: "subtree_delete", "repl_meta_data", + "count_attrs", "group_audit_log", "encrypted_secrets", and recompile. Samba will then write to a number of tdb files in the debug directory as requests and replies pass through. A simple script is included to read these files. Doing this: ./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb will print a table showing how often various attritbutes were requested but not found (from the point of view of the module). A more sophisticated version of the script is coming in the next commit, but this one is included first because in its simplicity it documents the storage format reasonably well. The tdb keys are attribute names, and the values are uint32_t in machine native order. When the module is included in the stack there will be a very small decrease in performance. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: load dns query from file and write stats to fileJoe Guo2019-05-011-0/+3
| | | | | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 1 01:10:42 UTC 2019 on sn-devel-184
* traffic_replay: Avoid Exception if no packet rate is specifiedTim Beale2019-04-301-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | traffic_replay would throw an exception if you didn't specify some sort of packet rate. We can avoid this by using --scale-traffic=1.0 as the default if nothing else was specified. script/traffic_replay model.txt $SERVER.$REALM --duration=10 --fixed-password=blahblah12# -U$USERNAME%$PASSWORD INFO 2019-04-10 01:03:01,809 pid:47755 script/traffic_replay #280: Using the specified model file to generate conversations Traceback (most recent call last): File "script/traffic_replay", line 438, in <module> main() File "script/traffic_replay", line 293, in main opts.conversation_persistence) File "bin/python/samba/emulate/traffic.py", line 1295, in generate_conversation_sequences target_packets = int(packet_rate * duration) TypeError: unsupported operand type(s) for *: 'NoneType' and 'float' Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* traffic_replay: Assign users to groups by defaultTim Beale2019-04-301-1/+10
| | | | | | | | | | | | | | | | The traffic_replay script has a myriad of options, but by default when it creates user accounts it does not assign these users to any groups (you have to specify extra options to do that). This isn't really a fair test of samba performance, because it's unlikely that real world setups will have users that are in no groups (other than the default ones). This patch changes the default behaviour so that it will assign the new users to groups automatically, if no other group options were specified. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: Build also Samba AD with MIT KerberosAndreas Schneider2019-04-251-1/+1
| | | | | Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Ralph Boehme <slow@samba.org>
* waf: build: Respect --disable-python for third_party modulesLutz Justen2019-04-242-0/+11
| | | | | | | | | | | | | | | | Skips installation of samba/third_party stuff into the python directory if --disable-python is set. Added test after install that confirms no python modules installed. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13905 Signed-off-by: Lutz Justen <ljusten@google.com> Reviewed-by: Jeremy Allison <jra@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed Apr 24 07:32:31 UTC 2019 on sn-devel-184
* selftest: rename schemaupgrade_dc (+pair) to schema_dcGarming Sam2019-04-121-4/+4
| | | | | | | | | | | | | | | | This is needed because the name of the autobuild job and the name of the selftest env end up in the socket path for ncalrpc sockets. The challenge is that (for example) /memdisk/autobuild/fl/b2424063/samba-schemaupgrade/bin/ab/schemaupgrade_pair_dc/ncalrpc/np/protected_storage does not fit in a struct sockaddr_un. Signed-off-by: Garming Sam <garming@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Fri Apr 12 05:41:36 UTC 2019 on sn-devel-144
* selftest: split schemaupgrade testenv outAaron Haslett2019-04-111-0/+14
| | | | | | | | | | Schemaupgrade tests are particularly resource intensive and are causing runners to hit their memory and CPU limits, so we need to split them out. Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* autobuild: attempt authenticated email if environment suggests itDouglas Bagnall2019-04-051-0/+6
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: try to distribute the tasks a bit moreStefan Metzmacher2019-02-271-21/+21
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: spread ad-dc tests over 6 autobuild/ci separate tasks/jobsStefan Metzmacher2019-02-271-12/+62
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: add samba-ad-member taskStefan Metzmacher2019-02-271-3/+15
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: run ad_dc_backup tests in samba-ad-dc-backupStefan Metzmacher2019-02-271-3/+2
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: move maptoguest and simpleserver to 'samba-fileserver'Stefan Metzmacher2019-02-271-1/+7
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* autobuild: move nt4_dc_schannel out of 'samba'Stefan Metzmacher2019-02-271-1/+6
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* selftest:Samba4: add ad_dc_backup alias to ad_dcStefan Metzmacher2019-02-271-0/+2
| | | | | | | | This will allow us to run really most tests in an isolated autobuild/ci task later. Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* selftest:Samba4: add ad_dc_default alias to ad_dc_ntvfsStefan Metzmacher2019-02-271-0/+2
| | | | | | | | | | | This will allow us to run really most tests in an isolated autobuild/ci task later. This will apply to tests, which may not rely on the ntvfs backend, so the ad_dc_default alias can point to another environment in future. Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* selftest:Samba4: add ad_dc_slowtests alias to ad_dc_ntvfsStefan Metzmacher2019-02-271-0/+2
| | | | | | | | This will allow us to run really slow tests in an isolated autobuild/ci task later. Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* selftest:Samba4: add fl2008dc as alias to ad_dc_ntvfsStefan Metzmacher2019-02-271-2/+5
| | | | | | | | | Using aliases it will be possible to split the large amount of tests which use ad_dc_ntvfs into multiple autobuild/ci tasks/jobs later. Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* selftest: Add basic sanity-check tests for nopython targetTim Beale2019-02-201-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously we were only checking samba compiled OK with --disable-python, not that it actually ran. The main problem is all the make test framework is based around subunit/smbtorture, neither of which we seem to build with disable-python. However, for just a simple sanity-check, we can just bypass all the subunit-filter work and just call the Perl code directly. This works OK as long as it's just simple shell script tests that we're running, as we can check the script's exit code directly. The main thing that we really want to test is that we can start up the smbd testenv and connect to it (i.e. a simple smbclient test). This patch adds a new 'make test-nopython' target. This disables the subunit filtering, and runs a small test-list that was generated manually. Note that currently this has the limitation that it doesn't support known failures or flapping tests. However, just checking that smd starts up OK is probably OK for now. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed Feb 20 02:10:00 CET 2019 on sn-devel-144
* autobuild: Drop 'py2' flagTim Beale2019-02-151-9/+2
| | | | | | | This isn't used any more. It was only being set, never referenced. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Replace samba-buildpy2-only with samba-nopython-py2Tim Beale2019-02-151-8/+32
| | | | | | | | | | | | | For Samba 4.11, the minimum python2 functionality we will support (for now, at least - we may change our minds) is for the --disable-python target, i.e. if you're excluding all the python functionality from samba, then WAF should still support being built with python2. The use case here is old unix platforms that want to use smbd, but don't have python3 support. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Remove ${EXTRA_PYTHON} variableTim Beale2019-02-151-17/+8
| | | | | | | | We no longer build the python2 bindings, only python3. So we can get rid of this variable now. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Tidy up unnecessary line-breaks in 'TESTS='Tim Beale2019-02-151-21/+10
| | | | | | | | Now that we've dropped the {PY3_ONLY} variable, there's no need for line-breaks in some of the 'TESTS=' values. We can tidy this up a bit. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Remove the PY3_ONLY variableTim Beale2019-02-151-13/+10
| | | | | | | | This variable is no longer needed as all the tests run using python3 now. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Update variable name to make more senseTim Beale2019-02-151-4/+4
| | | | | | | | | When we switched from python2 being the default to python3, we didn't update this variable name. It's now handling the python2 case, but it's a boolean flag named 'py3', which is rather confusing. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* autobuild: Drop py2 autobuild jobsTim Beale2019-02-151-6/+0
| | | | | | | | | | | | | | | | Samba v4.11 will no longer support python2, so let's drop the autobuild jobs. This will save some gitlab/sn-devel time and money, as it's less work for CI to do. Note that this highlights some previous inconsistencies: - samba-none-env-py2 was being built for gitlab but not sn-devel. - samba-nt4-py2 was being built for sn-devel but not gitlab I've left samba-buildpy2-only for now, which will be addressed in a subsequent patch. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/autobuild.py: add 'lsb_release -a' and 'mount' to system-info.txtStefan Metzmacher2019-02-141-0/+2
| | | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* CI: split out "samba-ad-dc-ntvfs[-py2]" test targetsRalph Boehme2019-02-111-6/+20
| | | | | | | | | | | | | | | | | | | Many AD tests currently use the "samba" target. Split out a new target "samba-ad-dc-ntvfs" and have all tests that use the "ad_dc_ntvfs" env use the new target. This should greatly speed up the runtime for the "samba" target and avoid swapping. This reduces the total CI time by ~ 55%, I got an autobuild and a gitlab pipeline finished in just ~ 100 mins! Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Signed-off-by: Ralph Boehme <slow@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Ralph Böhme <slow@samba.org> Autobuild-Date(master): Mon Feb 11 14:10:12 CET 2019 on sn-devel-144
* autobuild: Split backup/restore testenvs out into separate jobTim Beale2019-02-051-0/+12
| | | | | | | | | | | | | | | | The samba-ad-dc-2 job was reaching its limits with the number of testenvs and what the resource-limited CI machines can handle. Samba processes were getting swapped out of memory, causing CI runs to fail. This patch splits the backup/restore testenv targets into a separate autobuild job: samba-ad-dc-backup. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Stefan Metzmacher <metze@samba.org> Autobuild-User(master): Stefan Metzmacher <metze@samba.org> Autobuild-Date(master): Tue Feb 5 12:23:31 CET 2019 on sn-devel-144
View Lorry Controller Status