summaryrefslogtreecommitdiff
path: root/ctdb
Commit message (Collapse)AuthorAgeFilesLines
* ctdb-scripts: Fix tcp_tw_recycle existence checkRafael David Tinoco via samba-technical2019-06-211-2/+2
| | | | | | | | | | | | | | | | | net.ipv4.tcp_tw_recycle has been removed from Linux 4.12 but, still, makes sense to check its existence. Unfortunately, current check does not test for the procfs file existence. This commit fixes the issue. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13984 Signed-off-by: Rafael David Tinoco <rafaeldtinoco@ubuntu.com> Reviewed-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org> Autobuild-Date(master): Tue Jun 4 23:31:24 UTC 2019 on sn-devel-184 (cherry picked from commit 843fbb1207ee7ac84f3282974b66b9290d8da0ac)
* ctdb-common: Fix memory leak in run_procAmitay Isaacs2019-05-171-2/+5
| | | | | | | | | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943 Signed-off-by: Amitay Isaacs <amitay@gmail.com> Reviewed-by: Martin Schwenke <martin@meltin.net> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Tue May 14 08:59:03 UTC 2019 on sn-devel-184 (cherry picked from commit b1f4c86eea022999d5439e4a6ef3494fe41479b6) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Fri May 17 10:56:19 UTC 2019 on sn-devel-144
* ctdb-common: Fix memory leakMartin Schwenke2019-05-171-1/+2
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 30bc6e2529cdd444d4ec7902844c3a6fb0858090)
* ctdb-recoverd: Fix memory leakMartin Schwenke2019-05-171-1/+1
| | | | | | | | | | | state is always freed before exiting this function, so allocate fde off it instead of long-lived ctdb context. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13943 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 6a2941e2a9fd6ab2d5b8dbac042b61a7b1b0b914)
* ctdb:common: Do not print NULL if we don't get a sockpathAndreas Schneider2019-05-171-1/+1
| | | | | | | | | | | | sock_socket_start_recv() might not fill sockpath if we return early. Found by GCC 9. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13937 Signed-off-by: Andreas Schneider <asn@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> (cherry picked from commit 830cb7e67568de5f3ce359cb6af3be8ab545c824)
* ctdb-daemon: Never use 0 as a client IDMartin Schwenke2019-05-171-1/+47
| | | | | | | | | | | | ctdb_control_db_attach() and ctdb_control_db_detach() assume that any control with client ID 0 comes from another daemon and treat it specially. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13930 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 8663e0a64fbdb9ea16babbfe87d6f5d7a7b72bbd)
* ctdb-tests: Fix logic error in simple ctdb reloadips testMartin Schwenke2019-05-171-17/+20
| | | | | | | | | | | | | | | | There is a chance that restoring IP addresses to the test node will result in different IP addresses being assigned to that node. Removing a single IP address may then fail (or be a no-op) if it is done after the restore. So, swap the single IP address removal to happen first, then restore, then remove all IP addresses. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit dc89db8ca6aadd4a9f7e8a85843c53709d04587c)
* ctdb-tests: Make ctdb reloadips tests more reliableMartin Schwenke2019-05-172-7/+61
| | | | | | | | | | | | | ctdb reloadips will fail if it can't disable takover runs. The most likely reason for this is that there is already a takeover run in progress. We can't predict when this will happen, so retry if this occurs. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 8be4ee1a28d5c037955832b6f827d40f28f02796)
* ctdb-tests: Capture output in $out on failure as wellMartin Schwenke2019-05-171-3/+5
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit cf00db40355b49443263187f9d97934f91287e51)
* ctdb-tests: Don't clean up test var directory in autotest targetMartin Schwenke2019-05-171-1/+1
| | | | | | | | | | | | | | | | | | If the directory is always cleaned up then it is not possible to look at daemon logs to debug test failures. This target is only really used by autobuild.py, which (optionally) cleans up the parent directory anyway. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Tue May 7 06:56:01 UTC 2019 on sn-devel-184 (cherry picked from commit 5a9e338330fe136908a3a17a5df81c054c5cc5b0)
* ctdb-tests: Fix usage messageMartin Schwenke2019-05-171-1/+1
| | | | | | | | | | | | Since commit 0e9ead8f28fced3ebfa888786a1dc5bb59e734a3 daemons have been shut down after each test, so this option no longer has anything to do with killing daemons. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit a2ab6485e027ebb13871c7d83b7626ac5c9b98c0)
* ctdb-tests: Wait to allow database attach/detach to take effectMartin Schwenke2019-05-172-36/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes the detach test fails: Check detaching single test database detach_test1.tdb BAD: database detach_test1.tdb is still attached Number of databases:4 dbid:0x5ae995ee name:detach_test4.tdb path:tests/var/simple/node.0/db/volatile/detach_test4.tdb.0 dbid:0xd84cc13c name:detach_test3.tdb path:tests/var/simple/node.0/db/volatile/detach_test3.tdb.0 dbid:0x8e8e8cef name:detach_test2.tdb path:tests/var/simple/node.0/db/volatile/detach_test2.tdb.0 dbid:0xc62491f4 name:detach_test1.tdb path:tests/var/simple/node.0/db/volatile/detach_test1.tdb.0 Number of databases:3 dbid:0x5ae995ee name:detach_test4.tdb path:tests/var/simple/node.1/db/volatile/detach_test4.tdb.1 dbid:0xd84cc13c name:detach_test3.tdb path:tests/var/simple/node.1/db/volatile/detach_test3.tdb.1 dbid:0x8e8e8cef name:detach_test2.tdb path:tests/var/simple/node.1/db/volatile/detach_test2.tdb.1 Number of databases:4 dbid:0x5ae995ee name:detach_test4.tdb path:tests/var/simple/node.2/db/volatile/detach_test4.tdb.2 dbid:0xd84cc13c name:detach_test3.tdb path:tests/var/simple/node.2/db/volatile/detach_test3.tdb.2 dbid:0x8e8e8cef name:detach_test2.tdb path:tests/var/simple/node.2/db/volatile/detach_test2.tdb.2 dbid:0xc62491f4 name:detach_test1.tdb path:tests/var/simple/node.2/db/volatile/detach_test1.tdb.2 *** TEST COMPLETED (RC=1) AT 2019-04-27 03:35:40, CLEANING UP... When issued from a client, the detach control re-broadcasts itself asynchronously to all nodes and then returns success. The controls to some nodes to do the actual detach may still be in flight when success is returned to the client. Therefore, the test should wait for a few seconds to allow the asynchronous controls to complete. The same is true for the attach control, so workaround the problem in the attach test too. An alternative is to make the attach and detach controls synchronous by avoiding the broadcast and waiting for the results of the individual controls sent to the nodes. However, a simple implementation would involve adding new nested event loops. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 3cb53a7a05409925024d6a67bcfaeb962d896e0b)
* ctdb-tests: Avoid bulk output in $out, prefer $outfileMartin Schwenke2019-05-1738-198/+167
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 066cc5b0c561464ed08890d9aa1a1a55b545e9cc)
* ctdb-tests: Make try_command_on_node less error-proneMartin Schwenke2019-05-171-8/+22
| | | | | | | | | | | | | | | | | | | | This sometimes fails, apparently due to a cat process in onnode getting EAGAIN. The conclusion is that tests that process large amounts of output should not depend on a sub-shell delivering that output into a shell variable. Change try_command_on_node() to leave all of the output in file $outfile and just put the first 1KB into $out. $outfile is removed after each test completes. Change the implementation of sanity_check_output() to use $outfile instead of $out. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 9d02452a24625df5f62fd6d45a16effe2fa45fbe)
* ctdb-tests: Change sanity_check_output() to internally use $outMartin Schwenke2019-05-1711-21/+15
| | | | | | | | | | | | All callers are currently passed $out. Global variable $out is used in many other places so use it here to simplify the interface and make future changes simpler. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13924 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 7c3819d1ac264acf998f426e0cef7f6211e0ddee)
* ctdb-tests: Extend test to cover ctdb rddumpmemoryMartin Schwenke2019-05-171-2/+5
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13923 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 8108b3134c017c22d245fc5b2207a88d44ab0dd2)
* ctdb-tools: Fix ctdb dumpmemory to avoid printing trailing NULMartin Schwenke2019-05-171-4/+6
| | | | | | | | | | Fix ctdb rddumpmemory too. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13923 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit f78d9388fb459dc83fafb4da6e683e3137ad40e1)
* ctdb-common: Avoid race between fd and signal eventsAmitay Isaacs2019-04-151-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13895 In run_proc, there was an implicit assumption that when a process exits, fd event (pipe between parent and child) would be processed first and signal event (SIGCHLD for the child) would be processed later. However, that is not the case. SIGCHLD can be received asynchronously any time even when the pipe data has not fully been read. This causes run_proc to miss some of the output from child process in tests. When SIGCHLD is being processed, if the pipe between parent and child is still open, then do an explict read from the pipe to ensure we read any data still in the pipe before closing the pipe. Signed-off-by: Amitay Isaacs <amitay@gmail.com> Reviewed-by: Martin Schwenke <martin@meltin.net> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Fri Apr 12 08:19:29 UTC 2019 on sn-devel-144 (cherry picked from commit 289201277cd983b27cdfd5376c607eab112b4082) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Mon Apr 15 12:55:46 UTC 2019 on sn-devel-144
* ctdb-daemon: Revert "We can not assume that just because we could complete a ↵Martin Schwenke2019-04-151-0/+3
| | | | | | | | | | | | | | | | | | | | | | TCP handshake" We also can not assume that nodes can be marked as connected via only the keepalive mechanism. Keepalives are not sent to disconnected nodes so, in the absence of other packets (e.g. broadcasts), 2 nodes may never become marked as connected to each other. Revert to marking nodes as connected in the TCP transport code. If a connection is to a non(-operational) ctdbd then it will revert to disconnected after a short while and may actually flap. This should be rare. This reverts commit 66919db3d7ab1e091223faf515b183af8bfddc83. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13888 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 38dc6d11a26c2e9a2cae7927321f2216ceb1c5ec)
* ctdb-scripts: Update statd-callout to try several configuration filesMartin Schwenke2019-04-121-1/+2
| | | | | | | | | | | | The alternative seems to be to try something via CTDB_NFS_CALLOUT. That would be complicated and seems like overkill for something this simple. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit a2bd4085896804ee2da811e17f18c78a5bf4e658)
* ctdb-scripts: Allow load_system_config() to take multiple alternativesMartin Schwenke2019-04-121-9/+10
| | | | | | | | | | | | | The situation for NFS config has got more complicated and is probably broken in statd-callout on Debian-like systems at the moment. Allow several alternative configuration names to be tried. Stop after the first that is found and loaded. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 0d67ea5fcca766734ecc73ad6b0139f7c13a15c5)
* ctdb-tests: Update NFS test infrastructure to support systemd servicesMartin Schwenke2019-04-121-6/+93
| | | | | | | | | | | The tests are written around the default of sysvinit-redhat. Add support for systemd-redhat. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 2833ddcfcb780497264e0f412a9ad6e26a9bc657)
* ctdb-scripts: Add systemd services to NFS call-outMartin Schwenke2019-04-121-0/+23
| | | | | | | | | | | | | At least Red Hat and Debian appear to use (a variant of?) the upstream systemd units for NFS, so adding support for these services is relatively easy. Distributions using Sys-V init can patch the call-out to use the relevant Sys-V init services. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit a8fafd377ff0cb07ab161e437c5fe024704345eb)
* ctdb-scripts: Start NFS quota service if definedMartin Schwenke2019-04-121-0/+10
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 708c04071af8d6ddc3bf2bddbde4d5847f440c0e)
* ctdb-scripts: Stop/start mount/rquotad/status via NFS call-outMartin Schwenke2019-04-124-6/+88
| | | | | | | | | | | | | | | | | When an NFS check restarts a failed service by hand then systemd will be unable to stop or start this service again because (at least) the PID file will be wrong. Do this via the NFS Linux kernel call-out instead. Allow the call-out to use the services instead of doing manual restarts. Add variables for mount, status and rquotad services to support this. Adding systemd NFS services to the call-out will follow. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 42103b568698d8087d27f0848b402ccb7cfac86b)
* ctdb-scripts: Factor out nfs_load_config()Martin Schwenke2019-04-121-3/+11
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 8de0a339b550e7363d265af04ad69f2179af75c6)
* ctdb-scripts: Add test variable CTDB_NFS_DISTRO_STYLEMartin Schwenke2019-04-121-14/+32
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit e72c3c800a50fe746164e319e21180c44d041619)
* ctdb-scripts: Rename variable nfslock_service to nfs_lock_serviceMartin Schwenke2019-04-121-11/+11
| | | | | | | | | | | There will be more of these variable for other services so, for readability, it makes sense for them to start with "nfs_". BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 9981353ab79dce81b698c535977be4a681119d1e)
* ctdb-scripts: Reindent some functions prior to making changesMartin Schwenke2019-04-121-44/+44
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13860 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit d7e187c1a7046196ec96637bdc14cc6b042eafcc)
* ctdb-tests: Add some testing for IPv4-mapped IPv6 address parsingMartin Schwenke2019-03-221-0/+19
| | | | | | | | | | | | | | ctdb_sock_addr values are hashed in some contexts. This means that all of the memory used for the ctdb_sock_addr should be consistent regardless of how parsing is done. The first 2 cases are just sanity checks but the 3rd case involving an IPv4-mapped IPv6 address is the real target of this test addition. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13839 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit d9286701cd9253bf3b42cac3d850ae8c23743e6d)
* ctdb: Initialize addr struct to zero before reparsing as IPV4Zhu Shangzhong2019-03-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Failed to kill the tcp connection that using IPv4-mapped IPv6 address (e.g. ctdb_killtcp eth0 ::ffff:192.168.200.44:2049 ::ffff:192.168.200.45:863). When the ctdb_killtcp is used to kill the tcp connection, the IPs and ports in the connection will be parsed to conn.client and conn.server (call stack: main->ctdb_sock_addr_from_string->ip_from_string). In the ip_from_string, as we are using IPv4-mapped IPv6 addresses, the ipv6_from_string will be used to parse ip to addr.ip6 first. The next step the ipv4_from_string will be used to reparse ip to addr.ip. As a result, the data that dump from conn.server is "2 0 8 1 192 168 200 44 0 0 0 0 0 0 0 0 0 0 255 255 192 168 200 44 0 0 0 0", the data from conn.client is "2 0 3 95 192 168 200 45 0 0 0 0 0 0 0 0 0 0 255 255 192 168 200 45 0 0 0 0". The connection will be add to conn_list by ctdb_connection_list_add. Then the reset_connections_send uses conn_list as parameter to start to reset connections in the conn_list. In the reset_connections_send, the database "connections" will be created. The connections from conn_list will be written to the database(call db_hash_add), and use the data that dump from conn_client and conn_server as key. In the reset_connections_capture_tcp_handler, the ctdb_sys_read_tcp_packet will receive data on the raw socket. And extract the IPs and ports from the tcp packet. when extracting IP and port, the tcp4_extract OR tcp6_extract will be used. Then we got the new conn.client and conn.server. the data that dump from the conn.server is "2 0 8 1 192 168 200 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0", the data from conn.client is "2 0 3 95 192 168 200 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0". Finally, we use the data as key to check if this connection is one being reset(call db_hash_delete). The db_hash_delete will return ENOENT. Because the two key that being used by db_hash_delete and db_hash_add are different. So, the TCP RST will be NOT sent for the connection forever. We should initialize addr struct to zero before reparsing as IPV4 in the ip_from_string. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13839 Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn> Reviewed-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 539b5ff32b32b7c75dfaaa119e41f5af6ff1e6fc)
* ctdb-packaging: Test package requires tcpdumpMartin Schwenke2019-03-221-1/+1
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13838 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit 957c38b65ca060eabe1e676f8dfb54839d706155)
* ctdb-packaging: ctdb package should not own system library directoryMartin Schwenke2019-03-221-1/+1
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13838 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@samba.org> (cherry picked from commit b2b8dce4fc56c27ef0131104b316346565369dd7)
* ctdb-cluster-mutex: Separate out command and file handlingMartin Schwenke2019-03-041-42/+71
| | | | | | | | | | | | | | | | | | This code is difficult to read and there really is no common code between the 2 cases. For example, there is no need to split a filename into words. Separating each of the 2 cases into its own function makes the logic much easier to understand. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Mon Feb 25 03:40:16 CET 2019 on sn-devel-144 (cherry picked from commit c93430fe8fe530a55b9a04cf6cc660c3d420e333) (cherry picked from commit d5131afc533102ed5adfb147bf1a316e51810729)
* ctdb-recoverd: Time out attempt to take recovery lock after 120sMartin Schwenke2019-03-041-1/+1
| | | | | | | | | | | Currently this will wait forever. It really needs a timeout in case the cluster filesystem (or other lock mechanism) is completely wedged. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 13a1a4808935290dceb219daccd7aac3fda4e184)
* ctdb-recoverd: Ban node on unknown error when taking recovery lockMartin Schwenke2019-03-041-0/+11
| | | | | | | | | | | We really shouldn't see unknown errors. They probably represent a misconfigured recovery lock or similar. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 45a77d65b2e39b4af94da4ab99575f4ee08a7ebd)
* ctdb-recoverd: Make recoverd context available in recovery lock handleMartin Schwenke2019-03-041-0/+3
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit c0fb62ed3954fc6e8667480aba92003fc270f257)
* ctdb-recoverd: Clean up logging on failure to take recovery lockMartin Schwenke2019-03-041-3/+6
| | | | | | | | | | Add an explicit case for a timeout and clean up the other messages. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 7e4aae6943291c3144c8a3ff97537e8d4c7dc7c9)
* ctdb-recoverd: Free cluster mutex handler on failure to take lockMartin Schwenke2019-03-041-1/+10
| | | | | | | | | | | | | If nested events occur while the file descriptor handler is still active then chaos can ensue. For example, if a node is banned and the lock is explicitly cancelled (e.g. due to election loss) then double-talloc-free()s abound. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 621658cbed5d91d7096fc208bac2ff93a1880e7d)
* ctdb-config: Change example recovery lock setting to one that failsMartin Schwenke2019-03-041-4/+9
| | | | | | | | | | | | | | | | | | | ctdbd will start without a recovery lock configured. It will log a message saying that this is not optimal. However, a careless user may overlook both this message and the importance of setting a recovery lock. If the existing example configuration is uncommented then the directory containing it will be created (by 01.reclock.script) and the failure (i.e. multiple nodes able to take the lock) will be confusing. Instead, change the example setting to one that will result in banned nodes, encouraging users to consciously configure (or deconfigure) the recovery lock. Tweak the corresponding comment. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13790 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit 82e7f38214896c2c200132bc6dde3348cfac16cc)
* ctdb: Print locks latency in machinereadable statsVolker Lendecke2019-02-011-0/+5
| | | | | | | | | | | | | | Bug: https://bugzilla.samba.org/show_bug.cgi?id=13742 Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Martin Schwenke <martin@meltin.net> Autobuild-User(master): Volker Lendecke <vl@samba.org> Autobuild-Date(master): Wed Jan 16 05:34:17 CET 2019 on sn-devel-144 (cherry picked from commit 193a0d6f01372604b925d1972591062a0bb2400f) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Fri Feb 1 15:18:15 CET 2019 on sn-devel-144
* ctdb-daemon: Exit with error if a database directory does not existMartin Schwenke2018-12-051-0/+20
| | | | | | | | | | | | | | | | | | | Since 4.9.0, the log messages can be confusing if a required database directory does not exist. Explicitly check for database directories, logging a clear error and exiting if one is missing. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13696 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Mon Dec 3 06:56:41 CET 2018 on sn-devel-144 (cherry picked from commit dd7574afd1b2fb6a88defa154bc3d15e94f9ce0d) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Wed Dec 5 13:01:52 CET 2018 on sn-devel-144
* ctdb-tests: Make the debug hung script test cope with unreadable stacksMartin Schwenke2018-11-201-3/+14
| | | | | | | | | | | | | | | | | | | | | | | Ideally this would just involve using "test -r". However, operating system security features may mean that kernel stacks are not readable even though they appear to be. Instead, try reading that stack of a process on the test node. If that succeeds then so should reading the stack of the "stuck" sleep process in the test. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13684 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Tim Beale <timbeale@catalyst.net.nz> Autobuild-User(master): Tim Beale <timbeale@samba.org> Autobuild-Date(master): Thu Nov 15 08:15:32 CET 2018 on sn-devel-144 (cherry picked from commit c1dd6382e3211792e313f7d559b943f55c9cb0e1) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Tue Nov 20 15:50:33 CET 2018 on sn-devel-144
* ctdb-recovery: Ban a node that causes recovery failureMartin Schwenke2018-11-061-15/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... instead of applying banning credits. There have been a couple of cases where recovery repeatedly takes just over 2 minutes to fail. Therefore, banning credits expire between failures and a continuously problematic node is never banned, resulting in endless recoveries. This is because it takes 2 applications of banning credits before a node is banned, which generally involves 2 recovery failures. The recovery helper makes up to 3 attempts to recover each database during a single run. If a node causes 3 failures then this is really equivalent to 3 recovery failures in the model that existed before the recovery helper added retries. In that case the node would have been banned after 2 failures. So, instead of applying banning credits to the "most failing" node, simply ban it directly from the recovery helper. If multiple nodes are causing recovery failures then this can cause a node to be banned more quickly than it might otherwise have been, even pre-recovery-helper. However, 90 seconds (i.e. 3 failures) is a long time to be in recovery, so banning earlier seems like the best approach. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13670 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Mon Nov 5 06:52:33 CET 2018 on sn-devel-144 (cherry picked from commit 27df4f002a594dbb2f2a38afaccf3e22f19818e1)
* ctdb-daemon: Fix valgrind hit in event codeMartin Schwenke2018-10-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ==25741== Syscall param write(buf) points to uninitialised byte(s) ==25741== at 0x4939291: write (write.c:27) ==25741== by 0x4868285: sys_write (sys_rw.c:68) ==25741== by 0x13915D: sock_queue_trigger (sock_io.c:316) ==25741== by 0x4DE6478: tevent_common_invoke_immediate_handler (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37) ==25741== by 0x4DE64A2: tevent_common_loop_immediate (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37) ==25741== by 0x4DEBE5A: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37) ==25741== by 0x4DEA2D6: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37) ==25741== by 0x4DE57E3: _tevent_loop_once (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37) ==25741== by 0x15D1BA: ctdb_event_script_args (eventscript.c:821) ==25741== by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315) ==25741== by 0x110642: main (ctdbd.c:393) ==25741== Address 0x57888a4 is 100 bytes inside a block of size 144 alloc'd ==25741== at 0x48357BF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==25741== by 0x4B9B7C0: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.14) ==25741== by 0x15CCC6: eventd_client_write (eventscript.c:430) ==25741== by 0x15CCC6: eventd_client_run (eventscript.c:556) ==25741== by 0x15CCC6: ctdb_event_script_run (eventscript.c:649) ==25741== by 0x15D198: ctdb_event_script_args (eventscript.c:812) ==25741== by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315) ==25741== by 0x110642: main (ctdbd.c:393) ==25741== BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659 Pair-programmed-with: Amitay Isaacs <amitay@gmail.com> Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> Autobuild-User(master): Amitay Isaacs <amitay@samba.org> Autobuild-Date(master): Mon Oct 22 09:27:15 CEST 2018 on sn-devel-144 (cherry picked from commit fbea9d36996f248ba2b077f12ad16c199b853134) Autobuild-User(v4-9-test): Karolin Seeger <kseeger@samba.org> Autobuild-Date(v4-9-test): Thu Oct 25 11:05:48 CEST 2018 on sn-devel-144
* ctdb-event: Check the return status of sock_daemon_set_startup_fdAmitay Isaacs2018-10-251-1/+6
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659 Signed-off-by: Amitay Isaacs <amitay@gmail.com> Reviewed-by: Martin Schwenke <martin@meltin.net> (cherry picked from commit a1909603808b994b7822b697494e39e8da4aaa66)
* ctdb-common: Set close-on-exec for startup fdAmitay Isaacs2018-10-252-2/+9
| | | | | | | | | | | | | The startup_fd should not be propagated to the child processes created from a daemon. It should only be used in the daemon code to return the status of the startup. Another use of startup_fd is to notify the parent if the daemon process has exited. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659 Signed-off-by: Amitay Isaacs <amitay@gmail.com> Reviewed-by: Martin Schwenke <martin@meltin.net> (cherry picked from commit 80549927bc1741a4b8af8b8e830de4d37fa0c4a8)
* ctdb-daemon: Exit if eventd goes awayMartin Schwenke2018-10-251-7/+2
| | | | | | | | | | | | | ctdbd enters a broken state if eventd goes away. A clean shutdown is not possible because that involves running events. Restarting eventd is possible but this might mask a serious problem and it is possible that eventd might keep on disappearing. Just exit. BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit c9e1603a5d0c1a216439d4a2b0e7cdc05181e898)
* ctdb-daemon: Return early when refusing to run an event scriptMartin Schwenke2018-10-251-0/+1
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659 Signed-off-by: Martin Schwenke <martin@meltin.net> Reviewed-by: Amitay Isaacs <amitay@gmail.com> (cherry picked from commit a3d12252fa8e0a7e900b819dec30bdb9da458254)
* ctdb-tests: Drop code for RECEIVE_RECORDS controlAmitay Isaacs2018-10-101-20/+0
| | | | | | | | BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641 Signed-off-by: Amitay Isaacs <amitay@gmail.com> Reviewed-by: Martin Schwenke <martin@meltin.net> (cherry picked from commit 83b3c5670d85c607c1cf1ab8cfc2c967d4d16721)
View Lorry Controller Status