summaryrefslogtreecommitdiff
path: root/script/traffic_replay
Commit message (Collapse)AuthorAgeFilesLines
* traffic replay: Store the instance id in the replay contextGary Lockyer2019-07-241-1/+2
| | | | | | | | Store the traffic runner instance id in the replay context. Will be used in subsequent commits. Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: load dns query from file and write stats to fileJoe Guo2019-05-011-0/+3
| | | | | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 1 01:10:42 UTC 2019 on sn-devel-184
* traffic_replay: Avoid Exception if no packet rate is specifiedTim Beale2019-04-301-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | traffic_replay would throw an exception if you didn't specify some sort of packet rate. We can avoid this by using --scale-traffic=1.0 as the default if nothing else was specified. script/traffic_replay model.txt $SERVER.$REALM --duration=10 --fixed-password=blahblah12# -U$USERNAME%$PASSWORD INFO 2019-04-10 01:03:01,809 pid:47755 script/traffic_replay #280: Using the specified model file to generate conversations Traceback (most recent call last): File "script/traffic_replay", line 438, in <module> main() File "script/traffic_replay", line 293, in main opts.conversation_persistence) File "bin/python/samba/emulate/traffic.py", line 1295, in generate_conversation_sequences target_packets = int(packet_rate * duration) TypeError: unsupported operand type(s) for *: 'NoneType' and 'float' Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* traffic_replay: Assign users to groups by defaultTim Beale2019-04-301-1/+10
| | | | | | | | | | | | | | | | The traffic_replay script has a myriad of options, but by default when it creates user accounts it does not assign these users to any groups (you have to specify extra options to do that). This isn't really a fair test of samba performance, because it's unlikely that real world setups will have users that are in no groups (other than the default ones). This patch changes the default behaviour so that it will assign the new users to groups automatically, if no other group options were specified. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Andreas Schneider <asn@samba.org>
* traffic: rm --scale-traffic default valueJoe Guo2019-01-231-1/+1
| | | | | | | | | | | | | | -S, --scale--traffic defaults to 1.0 when we switch to new option -T, both -T and -S are set, which raise an error: script/traffic_replay #234: --scale-traffic and --packets-per-second are incompatible. Use one or the other. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz> Reviewed-by: Tim Beale <timbeale@catalyst.net.nz> Autobuild-User(master): Tim Beale <timbeale@samba.org> Autobuild-Date(master): Wed Jan 23 03:09:41 CET 2019 on sn-devel-144
* traffic_replay: use packets per second as primary scaleDouglas Bagnall2019-01-081-3/+16
| | | | | | | | The old -S/--scale-traffic is relative to the original model, which made its relationship to true traffic volumes quite opaque Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: --old-scale to mimic the old traffic_replayDouglas Bagnall2019-01-081-0/+10
| | | | | | | | | traffic_replay had a broken sense of traffic scale. That is fixed, but in order to compare old and new tests, it helps to be able to approximate the old behaviour. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: add option to reanimate dying conversationsDouglas Bagnall2019-01-081-1/+6
| | | | | | | | | | | | | | | The traffic model is generated from a window in time, which makes conversations appear to start and stop unnaturally at the window boundaries. When the window is short compared to the traffic replay time and the true expected conversation length, this has a significant distorting effect, leading to more conversations than would be expected to generate a given number of packets. To offset this slightly we add the --conversation-persistence option which tries to convert apparent death into a longish wait. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_model: don't report generation errors as parse errorsDouglas Bagnall2019-01-081-16/+10
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: improve debug messages in traffic_replayDouglas Bagnall2019-01-081-7/+6
| | | | | | | tracebacks and less nonsense at higher debug levels. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: traffic_replay --latency-timeout to control final waitDouglas Bagnall2019-01-081-0/+3
| | | | | | | | | Conversations that haven't finished within some acceptable margin of on-time can be said to have failed. This is where you specify that margin. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: --stop-on-any-error option to not ignore client troubleDouglas Bagnall2019-01-081-1/+4
| | | | | | | | Sometimes you want to know if any client is crashing for any reason. In those times use --stop-on-any-error for an early exit. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: rework conversation generation to better use memoryDouglas Bagnall2019-01-081-1/+1
| | | | | | | | Use less memory altogether and don't allocated shared mutable before the fork. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: generate sparser descriptions of conversationsDouglas Bagnall2019-01-081-6/+7
| | | | | | | | | | | | | Rather than building all the packets at this point, we stick to the barest details of the packets (which is all the model gives us anyway). The advantage is that will take a lot less memory, which matters because this process forks into many clients that were sharing and mutate the conversation list. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: new version of model with packet_rate, version numberDouglas Bagnall2019-01-081-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: delete empty temp directoriesDouglas Bagnall2019-01-081-1/+10
| | | | | | | | | even if someone asked to keep the temporary data, they don't want to see 5000 empty directries. Non-empty directories will remain. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic: use default value for --durationDouglas Bagnall2019-01-081-7/+3
| | | | | | | | We could not do this when we replayed traffic summaries (as opposed to models), but now this script does not do that. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: drop summary replayDouglas Bagnall2019-01-081-36/+22
| | | | | | | | | | The traffic_replay script has been able to replay a replay log as well as a model, which was not used in practice and complicated the script. If we want that feature, we can make a new script for it. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic replay: allow --random-seed=0Douglas Bagnall2019-01-081-2/+2
| | | | | | | Zero is a valid seed. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* PY3: change shebang to python3 in script dirJoe Guo2018-12-141-1/+1
| | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Noel Power <npower@samba.org>
* traffic_replay: Add a max-members option to cap group sizeTim Beale2018-12-041-0/+4
| | | | | | | | | | | | | | | | | | | | | | | traffic_replay tries to distribute the users among the groups in a realistic manner - some groups will have almost all users in them. However, this becomes a problem when testing a really large database, e.g. we may want 100K users, but no more than 5K users in each group. This patch adds a max-member option so we can limit how big the groups actually get. If we detect that a group exceeds the max-members, we reset the group's probability (of getting selected) to zero, and then recalculate the cumulative distribution. The means that the group should no longer get selected by generate_random_membership(). (Note we can't completely remove the group from the list because that changes the list-index-to-group-ID mapping). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Tue Dec 4 12:22:50 CET 2018 on sn-devel-144
* traffic_replay: Rework machine accounts to remove redundant codeTim Beale2018-11-071-0/+1
| | | | | | | | | | | | | | | generate_users_and_groups() now generates the machine acounts as well as the user accounts, so it seems there's no need to also have generate_traffic_accounts(), which does the same job. Instead, we can just pass through the number of machine acounts to generate_users_and_groups() and delete the other function. Also updated generate_users_and_groups() so that machine_accounts is no longer optional (we want to create machine accounts in all cases). Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Move 'traffic account' flag up a levelTim Beale2018-11-071-2/+4
| | | | | | | | | | | | | | | | | | We create machine accounts for 2 different purposes: 1). For traffic generation, i.e. testing realistic network packets. 2). For generating a realistic large DB. Unfortunately, we want to use different userAccountControl flags for the 2 different cases. Commit 3338a3e257fa9f28 changed the flags used for case #2, but this breaks case #1. The problem is generate_users_and_groups() is called in both cases, so we want the 'traffic account' flag passed into that function. This ensures that the machine accounts get created with the appropriate userAccountControl flags for the particular case you want to test. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Move machine account creationTim Beale2018-11-071-1/+5
| | | | | | | | | | | | | I was assuming that generate_users_and_groups() only gets called in the --generate-users-only case. However, it also gets called in the default traffic replay case. This patch reworks the code so that the number of machine accounts to create gets passed in, and the 'create 25% more computers than users' assumption only applies to the --generate-users-only case. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic_replay: Exception has no .messageDouglas Bagnall2018-11-011-1/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Noel Power <noel.power@suse.com>
* traffic_replay: logger was ignoring smb.conf log-levelTim Beale2018-10-311-1/+1
| | | | | | | | | We were trying to access the debug-level (in python C bindings) before the smb.conf had been loaded and actually set the debug-level. So it would default to zero, regardless of what was in the smb.conf. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* traffic_replay: Generate users faster by writing to local DBTim Beale2018-10-311-1/+11
| | | | | | | | | | | | | | | | | | | | We can create user accounts much faster if the LDB connection talks directly to the local sam.ldb file rather than going via LDAP. This patch allows the 'host' argument to the tool to be a .ldb file (e.g. "/usr/local/samba/private/sam.ldb") instead of a server name/IP. In most cases, the traffic_replay tool wants to run on a remote device (because the point of it is to send traffic to the DC). However, the --generate-users-only is one case where the tool can be run locally, directly on the test DC. (The traffic_replay user generation is handy for standalone testing, because it also handles assigning group memberships to the generated user accounts). Note that you also need to use '--option="ldb:nosync = true"' to get the improvement in performance. Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* script/traffic_replay: apply new logger to replace printJoe Guo2018-10-101-51/+57
| | | | | | | | Use logger to replace print Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/traffic_replay: get debug level via apiJoe Guo2018-10-101-4/+5
| | | | | | | | | | | The -d option will set samba global debug level automatically. We should not parse and use the passed in value. Use samba.get_debug_level instead. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* script/traffic_replay: print packets data to stderrJoe Guo2018-10-101-2/+2
| | | | | | | | | This is debug info, should print to stderr. Otherwise it will flood stdout. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* traffic-replay: add extra checkJoe Guo2018-08-151-0/+6
| | | | | | | | Make sure --average-groups-per-user is not more than --number-of-users Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* script: Add 'random-seed' option to traffic_replayTim Beale2018-05-161-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When using a traffic-model file to generate traffic, there is some randomness in the actual packets that get generated. This means it's hard to use the tool to detect an increase/decrease in Samba performance - we don't know whether a decrease in packets sent is due to a regression in the Samba codebase, or just due to the tool sending different types of packets (i.e. ones that take longer to process). This patch adds an option to seed the python random number generator. This means that exactly the same traffic can be generated across multiple test runs. (Previously we were using the '--traffic-summary' option to avoid this problem - we can generate a summary-file based on the model, and then use the same summary file across multiple runs. However, this proved impractical when you want to run multiple combinations of scale/rate parameters, e.g. 21 x 8 different permutations just fills up disk space with summary-files.) Signed-off-by: Tim Beale <timbeale@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Alexander Bokovoy <ab@samba.org> Reviewed-by: William Brown <william@blackhats.net.au> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Wed May 16 13:53:26 CEST 2018 on sn-devel-144
* traffic_replay: fetch domain from creds other than optsJoe Guo2018-05-121-1/+1
| | | | | | | | | | | For traffic_replay script, when user provides `--workgroup` or `-W` option, it will be set on the creds option group, not the default opts one. The previous code will not work properly when smb.conf file is missing. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* traffic_replay: fix typo in message stringJoe Guo2018-05-121-1/+1
| | | | | | Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* traffic_replay: set gensec features to encrypt credentialsJoe Guo2018-05-121-0/+3
| | | | | | | | | | | | While running traffic_replay script against windows dc, it will fail with a `LDAP_UNWILLING_TO_PERFORM` error for adding user. Windows requires the credentials to be encrypted before sending. `set_gensec_features` will fix it. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* traffic_relay: bulk port print to modern py3 styleJoe Guo2018-04-131-50/+55
| | | | | | | | | | | Change print to function and avoid the ugly `print >>sys.stderr`. Signed-off-by: Joe Guo <joeg@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz> Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org> Autobuild-Date(master): Fri Apr 13 10:36:32 CEST 2018 on sn-devel-144
* scripts: Scripts to replay and generate samba trafficGary Lockyer2017-08-171-0/+362
Scripts to generate representative network traffic and replay this to a samba instance. For load testing, performance profiling and capacity planning. traffic_learner process a file generated by traffic_summary and generate a model that can be used by traffic_replay to generate samba network traffic. traffic_replay Replay a summary file generated by traffic_summary, or use a model created by traffic_learner to generate network traffic. Signed-off-by: Gary Lockyer <gary@catalyst.net.nz> Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz> Pair-programmed-with: Garming Sam <garming@catalyst.net.nz> Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org> Pair-Programmed-With: Tim Beale <timbeale@catalyst.net.nz>
View Lorry Controller Status