diff options
Diffstat (limited to 'ctdb')
| -rw-r--r-- | ctdb/doc/Makefile | 10 | ||||
| -rw-r--r-- | ctdb/doc/ctdb-tunables.7.xml | 708 | ||||
| -rw-r--r-- | ctdb/doc/ctdb.1.xml | 1861 | ||||
| -rw-r--r-- | ctdb/doc/ctdb.7.xml | 1001 | ||||
| -rw-r--r-- | ctdb/doc/ctdbd.1.xml | 2215 | ||||
| -rw-r--r-- | ctdb/doc/ctdbd.conf.5.xml | 1598 | ||||
| -rw-r--r-- | ctdb/doc/ctdbd_wrapper.1.xml | 106 | ||||
| -rw-r--r-- | ctdb/doc/ltdbtool.1.xml | 248 | ||||
| -rw-r--r-- | ctdb/doc/onnode.1.xml | 331 | ||||
| -rw-r--r-- | ctdb/doc/ping_pong.1.xml | 169 |
10 files changed, 5389 insertions, 2858 deletions
diff --git a/ctdb/doc/Makefile b/ctdb/doc/Makefile index 2f7d41d239e..34303a542aa 100644 --- a/ctdb/doc/Makefile +++ b/ctdb/doc/Makefile @@ -1,15 +1,19 @@ DOCS = ctdb.1 ctdb.1.html \ ctdbd.1 ctdbd.1.html \ + ctdbd_wrapper.1 ctdbd_wrapper.1.html \ onnode.1 onnode.1.html \ ltdbtool.1 ltdbtool.1.html \ - ping_pong.1 ping_pong.1.html + ping_pong.1 ping_pong.1.html \ + ctdbd.conf.5 ctdbd.conf.5.html \ + ctdb.7 ctdb.7.html \ + ctdb-tunables.7 ctdb-tunables.7.html all: $(DOCS) -%.1: %.1.xml +%: %.xml xsltproc -o $@ http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl $< -%.1.html: %.1.xml +%.html: %.xml xsltproc -o $@ http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl $< distclean: diff --git a/ctdb/doc/ctdb-tunables.7.xml b/ctdb/doc/ctdb-tunables.7.xml new file mode 100644 index 00000000000..456e856355c --- /dev/null +++ b/ctdb/doc/ctdb-tunables.7.xml @@ -0,0 +1,708 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE refentry + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> + +<refentry id="ctdb-tunables.7"> + + <refmeta> + <refentrytitle>ctdb-tunables</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ctdb-tunables</refname> + <refpurpose>CTDB tunable configuration variables</refpurpose> + </refnamediv> + + <refsect1> + <title>DESCRIPTION</title> + + <para> + CTDB's behaviour can be configured by setting run-time tunable + variables. This lists and describes all tunables. See the + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> + <command>listvars</command>, <command>setvar</command> and + <command>getvar</command> commands for more details. + </para> + + <refsect2> + <title>MaxRedirectCount</title> + <para>Default: 3</para> + <para> + If we are not the DMASTER and need to fetch a record across the network + we first send the request to the LMASTER after which the record + is passed onto the current DMASTER. If the DMASTER changes before + the request has reached that node, the request will be passed onto the + "next" DMASTER. For very hot records that migrate rapidly across the + cluster this can cause a request to "chase" the record for many hops + before it catches up with the record. + + this is how many hops we allow trying to chase the DMASTER before we + switch back to the LMASTER again to ask for new directions. + </para> + <para> + When chasing a record, this is how many hops we will chase the record + for before going back to the LMASTER to ask for new guidance. + </para> + </refsect2> + + <refsect2> + <title>SeqnumInterval</title> + <para>Default: 1000</para> + <para> + Some databases have seqnum tracking enabled, so that samba will be able + to detect asynchronously when there has been updates to the database. + Everytime a database is updated its sequence number is increased. + </para> + <para> + This tunable is used to specify in 'ms' how frequently ctdb will + send out updates to remote nodes to inform them that the sequence + number is increased. + </para> + </refsect2> + + <refsect2> + <title>ControlTimeout</title> + <para>Default: 60</para> + <para> + This is the default + setting for timeout for when sending a control message to either the + local or a remote ctdb daemon. + </para> + </refsect2> + + <refsect2> + <title>TraverseTimeout</title> + <para>Default: 20</para> + <para> + This setting controls how long we allow a traverse process to run. + After this timeout triggers, the main ctdb daemon will abort the + traverse if it has not yet finished. + </para> + </refsect2> + + <refsect2> + <title>KeepaliveInterval</title> + <para>Default: 5</para> + <para> + How often in seconds should the nodes send keepalives to eachother. + </para> + </refsect2> + + <refsect2> + <title>KeepaliveLimit</title> + <para>Default: 5</para> + <para> + After how many keepalive intervals without any traffic should a node + wait until marking the peer as DISCONNECTED. + </para> + <para> + If a node has hung, it can thus take KeepaliveInterval*(KeepaliveLimit+1) + seconds before we determine that the node is DISCONNECTED and that we + require a recovery. This limitshould not be set too high since we want + a hung node to be detectec, and expunged from the cluster well before + common CIFS timeouts (45-90 seconds) kick in. + </para> + </refsect2> + + <refsect2> + <title>RecoverTimeout</title> + <para>Default: 20</para> + <para> + This is the default setting for timeouts for controls when sent from the + recovery daemon. We allow longer control timeouts from the recovery daemon + than from normal use since the recovery dameon often use controls that + can take a lot longer than normal controls. + </para> + </refsect2> + + <refsect2> + <title>RecoverInterval</title> + <para>Default: 1</para> + <para> + How frequently in seconds should the recovery daemon perform the + consistency checks that determine if we need to perform a recovery or not. + </para> + </refsect2> + + <refsect2> + <title>ElectionTimeout</title> + <para>Default: 3</para> + <para> + When electing a new recovery master, this is how many seconds we allow + the election to take before we either deem the election finished + or we fail the election and start a new one. + </para> + </refsect2> + + <refsect2> + <title>TakeoverTimeout</title> + <para>Default: 9</para> + <para> + This is how many seconds we allow controls to take for IP failover events. + </para> + </refsect2> + + <refsect2> + <title>MonitorInterval</title> + <para>Default: 15</para> + <para> + How often should ctdb run the event scripts to check for a nodes health. + </para> + </refsect2> + + <refsect2> + <title>TickleUpdateInterval</title> + <para>Default: 20</para> + <para> + How often will ctdb record and store the "tickle" information used to + kickstart stalled tcp connections after a recovery. + </para> + </refsect2> + + <refsect2> + <title>EventScriptTimeout</title> + <para>Default: 20</para> + <para> + How long should ctdb let an event script run before aborting it and + marking the node unhealthy. + </para> + </refsect2> + + <refsect2> + <title>EventScriptTimeoutCount</title> + <para>Default: 1</para> + <para> + How many events in a row needs to timeout before we flag the node UNHEALTHY. + This setting is useful if your scripts can not be written so that they + do not hang for benign reasons. + </para> + </refsect2> + + <refsect2> + <title>EventScriptUnhealthyOnTimeout</title> + <para>Default: 0</para> + <para> + This setting can be be used to make ctdb never become UNHEALTHY if your + eventscripts keep hanging/timing out. + </para> + </refsect2> + + <refsect2> + <title>RecoveryGracePeriod</title> + <para>Default: 120</para> + <para> + During recoveries, if a node has not caused recovery failures during the + last grace period, any records of transgressions that the node has caused + recovery failures will be forgiven. This resets the ban-counter back to + zero for that node. + </para> + </refsect2> + + <refsect2> + <title>RecoveryBanPeriod</title> + <para>Default: 300</para> + <para> + If a node becomes banned causing repetitive recovery failures. The node will + eventually become banned from the cluster. + This controls how long the culprit node will be banned from the cluster + before it is allowed to try to join the cluster again. + Don't set to small. A node gets banned for a reason and it is usually due + to real problems with the node. + </para> + </refsect2> + + <refsect2> + <title>DatabaseHashSize</title> + <para>Default: 100001</para> + <para> + Size of the hash chains for the local store of the tdbs that ctdb manages. + </para> + </refsect2> + + <refsect2> + <title>DatabaseMaxDead</title> + <para>Default: 5</para> + <para> + How many dead records per hashchain in the TDB database do we allow before + the freelist needs to be processed. + </para> + </refsect2> + + <refsect2> + <title>RerecoveryTimeout</title> + <para>Default: 10</para> + <para> + Once a recovery has completed, no additional recoveries are permitted + until this timeout has expired. + </para> + </refsect2> + + <refsect2> + <title>EnableBans</title> + <para>Default: 1</para> + <para> + When set to 0, this disables BANNING completely in the cluster and thus + nodes can not get banned, even it they break. Don't set to 0 unless you + know what you are doing. You should set this to the same value on + all nodes to avoid unexpected behaviour. + </para> + </refsect2> + + <refsect2> + <title>DeterministicIPs</title> + <para>Default: 0</para> + <para> + When enabled, this tunable makes ctdb try to keep public IP addresses + locked to specific nodes as far as possible. This makes it easier for + debugging since you can know that as long as all nodes are healthy + public IP X will always be hosted by node Y. + </para> + <para> + The cost of using deterministic IP address assignment is that it + disables part of the logic where ctdb tries to reduce the number of + public IP assignment changes in the cluster. This tunable may increase + the number of IP failover/failbacks that are performed on the cluster + by a small margin. + </para> + + </refsect2> + <refsect2> + <title>LCP2PublicIPs</title> + <para>Default: 1</para> + <para> + When enabled this switches ctdb to use the LCP2 ip allocation + algorithm. + </para> + </refsect2> + + <refsect2> + <title>ReclockPingPeriod</title> + <para>Default: x</para> + <para> + Obsolete + </para> + </refsect2> + + <refsect2> + <title>NoIPFailback</title> + <para>Default: 0</para> + <para> + When set to 1, ctdb will not perform failback of IP addresses when a node + becomes healthy. Ctdb WILL perform failover of public IP addresses when a + node becomes UNHEALTHY, but when the node becomes HEALTHY again, ctdb + will not fail the addresses back. + </para> + <para> + Use with caution! Normally when a node becomes available to the cluster + ctdb will try to reassign public IP addresses onto the new node as a way + to distribute the workload evenly across the clusternode. Ctdb tries to + make sure that all running nodes have approximately the same number of + public addresses it hosts. + </para> + <para> + When you enable this tunable, CTDB will no longer attempt to rebalance + the cluster by failing IP addresses back to the new nodes. An unbalanced + cluster will therefore remain unbalanced until there is manual + intervention from the administrator. When this parameter is set, you can + manually fail public IP addresses over to the new node(s) using the + 'ctdb moveip' command. + </para> + </refsect2> + + <refsect2> + <title>DisableIPFailover</title> + <para>Default: 0</para> + <para> + When enabled, ctdb will not perform failover or failback. Even if a + node fails while holding public IPs, ctdb will not recover the IPs or + assign them to another node. + </para> + <para> + When you enable this tunable, CTDB will no longer attempt to recover + the cluster by failing IP addresses over to other nodes. This leads to + a service outage until the administrator has manually performed failover + to replacement nodes using the 'ctdb moveip' command. + </para> + </refsect2> + + <refsect2> + <title>NoIPTakeover</title> + <para>Default: 0</para> + <para> + When set to 1, ctdb will not allow IP addresses to be failed over + onto this node. Any IP addresses that the node currently hosts + will remain on the node but no new IP addresses can be failed over + to the node. + </para> + </refsect2> + + <refsect2> + <title>NoIPHostOnAllDisabled</title> + <para>Default: 0</para> + <para> + If no nodes are healthy then by default ctdb will happily host + public IPs on disabled (unhealthy or administratively disabled) + nodes. This can cause problems, for example if the underlying + cluster filesystem is not mounted. When set to 1 on a node and + that node is disabled it, any IPs hosted by this node will be + released and the node will not takeover any IPs until it is no + longer disabled. + </para> + </refsect2> + + <refsect2> + <title>DBRecordCountWarn</title> + <para>Default: 100000</para> + <para> + When set to non-zero, ctdb will log a warning when we try to recover a + database with more than this many records. This will produce a warning + if a database grows uncontrollably with orphaned records. + </para> + </refsect2> + + <refsect2> + <title>DBRecordSizeWarn</title> + <para>Default: 10000000</para> + <para> + When set to non-zero, ctdb will log a warning when we try to recover a + database where a single record is bigger than this. This will produce + a warning if a database record grows uncontrollably with orphaned + sub-records. + </para> + </refsect2> + + <refsect2> + <title>DBSizeWarn</title> + <para>Default: 1000000000</para> + <para> + When set to non-zero, ctdb will log a warning when we try to recover a + database bigger than this. This will produce + a warning if a database grows uncontrollably. + </para> + </refsect2> + + <refsect2> + <title>VerboseMemoryNames</title> + <para>Default: 0</para> + <para> + This feature consumes additional memory. when used the talloc library + will create more verbose names for all talloc allocated objects. + </para> + </refsect2> + + <refsect2> + <title>RecdPingTimeout</title> + <para>Default: 60</para> + <para> + If the main dameon has not heard a "ping" from the recovery dameon for + this many seconds, the main dameon will log a message that the recovery + daemon is potentially hung. + </para> + </refsect2> + + <refsect2> + <title>RecdFailCount</title> + <para>Default: 10</para> + <para> + If the recovery daemon has failed to ping the main dameon for this many + consecutive intervals, the main daemon will consider the recovery daemon + as hung and will try to restart it to recover. + </para> + </refsect2> + + <refsect2> + <title>LogLatencyMs</title> + <para>Default: 0</para> + <para> + When set to non-zero, this will make the main daemon log any operation that + took longer than this value, in 'ms', to complete. + These include "how long time a lockwait child process needed", + "how long time to write to a persistent database" but also + "how long did it take to get a response to a CALL from a remote node". + </para> + </refsect2> + + <refsect2> + <title>RecLockLatencyMs</title> + <para>Default: 1000</para> + <para> + When using a reclock file for split brain prevention, if set to non-zero + this tunable will make the recovery dameon log a message if the fcntl() + call to lock/testlock the recovery file takes longer than this number of + ms. + </para> + </refsect2> + + <refsect2> + <title>RecoveryDropAllIPs</title> + <para>Default: 120</para> + <para> + If we have been stuck in recovery, or stopped, or banned, mode for + this many seconds we will force drop all held public addresses. + </para> + </refsect2> + + <refsect2> + <title>VerifyRecoveryLock</title> + <para>Default: 1</para> + <para> + Should we take a fcntl() lock on the reclock file to verify that we are the + sole recovery master node on the cluster or not. + </para> + </refsect2> + + <refsect2> + <title>VacuumInterval</title> + <para>Default: 10</para> + <para> + Periodic interval in seconds when vacuuming is triggered for + volatile databases. + </para> + </refsect2> + + <refsect2> + <title>VacuumMaxRunTime</title> + <para>Default: 120</para> + <para> + The maximum time in seconds for which the vacuuming process is + allowed to run. If vacuuming process takes longer than this + value, then the vacuuming process is terminated. + </para> + </refsect2> + + <refsect2> + <title>RepackLimit</title> + <para>Default: 10000</para> + <para> + During vacuuming, if the number of freelist records are more + than <varname>RepackLimit</varname>, then databases are + repacked to get rid of the freelist records to avoid + fragmentation. + </para> + <para> + Databases are repacked only if both + <varname>RepackLimit</varname> and + <varname>VacuumLimit</varname> are exceeded. + </para> + </refsect2> + + <refsect2> + <title>VacuumLimit</title> + <para>Default: 5000</para> + <para> + During vacuuming, if the number of deleted records are more + than <varname>VacuumLimit</varname>, then databases are + repacked to avoid fragmentation. + </para> + <para> + Databases are repacked only if both + <varname>RepackLimit</varname> and + <varname>VacuumLimit</varname> are exceeded. + </para> + </refsect2> + + <refsect2> + <title>VacuumFastPathCount</title> + <para>Default: 60</para> + <para> + When a record is deleted, it is marked for deletion during + vacuuming. Vacuuming process usually processes this list to purge + the records from the database. If the number of records marked + for deletion are more than VacuumFastPathCount, then vacuuming + process will scan the complete database for empty records instead + of using the list of records marked for deletion. + </para> + </refsect2> + + <refsect2> + <title>DeferredAttachTO</title> + <para>Default: 120</para> + <para> + When databases are frozen we do not allow clients to attach to the + databases. Instead of returning an error immediately to the application + the attach request from the client is deferred until the database + becomes available again at which stage we respond to the client. + </para> + <para> + This timeout controls how long we will defer the request from the client + before timing it out and returning an error to the client. + </para> + </refsect2> + + <refsect2> + <title>HopcountMakeSticky</title> + <para>Default: 50</para> + <para> + If the database is set to 'STICKY' mode, using the 'ctdb setdbsticky' + command, any record that is seen as very hot and migrating so fast that + hopcount surpasses 50 is set to become a STICKY record for StickyDuration + seconds. This means that after each migration the record will be kept on + the node and prevented from being migrated off the node. + </para> + <para> + This setting allows one to try to identify such records and stop them from + migrating across the cluster so fast. This will improve performance for + certain workloads, such as locking.tdb if many clients are opening/closing + the same file concurrently. + </para> + </refsect2> + + <refsect2> + <title>StickyDuration</title> + <para>Default: 600</para> + <para> + Once a record has been found to be fetch-lock hot and has been flagged to + become STICKY, this is for how long, in seconds, the record will be + flagged as a STICKY record. + </para> + </refsect2> + + <refsect2> + <title>StickyPindown</title> + <para>Default: 200</para> + <para> + Once a STICKY record has been migrated onto a node, it will be pinned down + on that node for this number of ms. Any request from other nodes to migrate + the record off the node will be deferred until the pindown timer expires. + </para> + </refsect2> + + <refsect2> + <title>StatHistoryInterval</title> + <para>Default: 1</para> + <para> + Granularity of the statistics collected in the statistics history. + </para> + </refsect2> + + <refsect2> + <title>AllowClientDBAttach</title> + <para>Default: 1</para> + <para> + When set to 0, clients are not allowed to attach to any databases. + This can be used to temporarily block any new processes from attaching + to and accessing the databases. + </para> + </refsect2> + + <refsect2> + <title>RecoverPDBBySeqNum</title> + <para>Default: 1</para> + <para> + When set to zero, database recovery for persistent databases + is record-by-record and recovery process simply collects the + most recent version of every individual record. + </para> + <para> + When set to non-zero, persistent databases will instead be + recovered as a whole db and not by individual records. The + node that contains the highest value stored in the record + "__db_sequence_number__" is selected and the copy of that + nodes database is used as the recovered database. + </para> + <para> + By default, recovery of persistent databses is done using + __db_sequence_number__ record. + </para> + </refsect2> + + <refsect2> + <title>FetchCollapse</title> + <para>Default: 1</para> + <para> + When many clients across many nodes try to access the same record at the + same time this can lead to a fetch storm where the record becomes very + active and bounces between nodes very fast. This leads to high CPU + utilization of the ctdbd daemon, trying to bounce that record around + very fast, and poor performance. + </para> + <para> + This parameter is used to activate a fetch-collapse. A fetch-collapse + is when we track which records we have requests in flight so that we only + keep one request in flight from a certain node, even if multiple smbd + processes are attemtping to fetch the record at the same time. This + can improve performance and reduce CPU utilization for certain + workloads. + </para> + <para> + This timeout controls if we should collapse multiple fetch operations + of the same record into a single request and defer all duplicates or not. + </para> + </refsect2> + + <refsect2> + <title>Samba3AvoidDeadlocks</title> + <para>Default: 0</para> + <para> + Enable code that prevents deadlocks with Samba (only for Samba 3.x). + </para> + <para> + This should be set to 1 when using Samba version 3.x to enable special + code in CTDB to avoid deadlock with Samba version 3.x. This code + is not required for Samba version 4.x and must not be enabled for + Samba 4.x. + </para> + </refsect2> + </refsect1> + + <refsect1> + <title>SEE ALSO</title> + <para> + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="http://ctdb.samba.org/"/> + </para> + </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Ronnie Sahlberg, + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + +</refentry> diff --git a/ctdb/doc/ctdb.1.xml b/ctdb/doc/ctdb.1.xml index ebb9c8e30b6..27e52cd2da1 100644 --- a/ctdb/doc/ctdb.1.xml +++ b/ctdb/doc/ctdb.1.xml @@ -1,320 +1,278 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry - PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" - "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> <refentry id="ctdb.1"> -<refmeta> - <refentrytitle>ctdb</refentrytitle> - <manvolnum>1</manvolnum> - <refmiscinfo class="source">ctdb</refmiscinfo> - <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> -</refmeta> - - -<refnamediv> - <refname>ctdb</refname> - <refpurpose>clustered tdb database management utility</refpurpose> -</refnamediv> - -<refsynopsisdiv> - <cmdsynopsis> - <command>ctdb [ OPTIONS ] COMMAND ...</command> - </cmdsynopsis> - - <cmdsynopsis> - <command>ctdb</command> - <arg choice="opt">-n <node></arg> - <arg choice="opt">-Y</arg> - <arg choice="opt">-t <timeout></arg> - <arg choice="opt">-T <timelimit></arg> - <arg choice="opt">-? --help</arg> - <arg choice="opt">--usage</arg> - <arg choice="opt">-d --debug=<INTEGER></arg> - <arg choice="opt">--socket=<filename></arg> - <arg choice="opt">--print-emptyrecords</arg> - <arg choice="opt">--print-datasize</arg> - <arg choice="opt">--print-lmaster</arg> - <arg choice="opt">--print-hash</arg> - <arg choice="opt">--print-recordflags</arg> - </cmdsynopsis> - -</refsynopsisdiv> - - <refsect1><title>DESCRIPTION</title> + <refmeta> + <refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ctdb</refname> + <refpurpose>CTDB management utility</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>ctdb</command> + <arg rep="repeat"><replaceable>OPTION</replaceable></arg> + <arg choice="req"><replaceable>COMMAND</replaceable></arg> + <arg choice="opt"><replaceable>COMMAND-ARGS</replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1> + <title>DESCRIPTION</title> <para> - ctdb is a utility to view and manage a ctdb cluster. + ctdb is a utility to view and manage a CTDB cluster. </para> - </refsect1> + <para> + The following terms are used when referring to nodes in a + cluster: + <variablelist> + <varlistentry> + <term>PNN</term> + <listitem> + <para> + Physical Node Number. The physical node number is an + integer that describes the node in the cluster. The + first node has physical node number 0. in a cluster. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term>PNN-LIST</term> + <listitem> + <para> + This is either a single PNN, a comma-separate list of PNNs + or "all". + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + </refsect1> <refsect1> <title>OPTIONS</title> <variablelist> - <varlistentry><term>-n <pnn></term> - <listitem> - <para> - This specifies the physical node number on which to execute the - command. Default is to run the command on the daemon running on - the local host. - </para> - <para> - The physical node number is an integer that describes the node in the - cluster. The first node has physical node number 0. - </para> - </listitem> + <varlistentry><term>-n <parameter>PNN-LIST</parameter></term> + <listitem> + <para> + The nodes specified by PNN-LIST should be queried for the + requested information. Default is to query the daemon + running on the local host. + </para> + </listitem> </varlistentry> <varlistentry><term>-Y</term> - <listitem> - <para> - Produce output in machine readable form for easier parsing by scripts. Not all commands support this option. - </para> - </listitem> + <listitem> + <para> + Produce output in machine readable form for easier parsing + by scripts. Not all commands support this option. + </para> + </listitem> </varlistentry> - <varlistentry><term>-t <timeout></term> - <listitem> - <para> - How long should ctdb wait for the local ctdb daemon to respond to a command before timing out. Default is 3 seconds. - </para> - </listitem> + <varlistentry><term>-t <parameter>TIMEOUT</parameter></term> + <listitem> + <para> + Indicates that ctdb should wait up to TIMEOUT seconds for + a response to most commands sent to the CTDB daemon. The + default is 10 seconds. + </para> + </listitem> </varlistentry> - <varlistentry><term>-T <timelimit></term> - <listitem> - <para> - A limit on how long the ctdb command will run for before it will - be aborted. When this timelimit has been exceeded the ctdb command will - terminate. - </para> - </listitem> + <varlistentry><term>-T <parameter>TIMELIMIT</parameter></term> + <listitem> + <para> + Indicates that TIMELIMIT is the maximum run time (in + seconds) for the ctdb command. When TIMELIMIT is exceeded + the ctdb command will terminate with an error. The default + is 120 seconds. + </para> + </listitem> </varlistentry> <varlistentry><term>-? --help</term> - <listitem> - <para> - Print some help text to the screen. - </para> - </listitem> + <listitem> + <para> + Print some help text to the screen. + </para> + </listitem> </varlistentry> <varlistentry><term>--usage</term> - <listitem> - <para> - Print useage information to the screen. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>-d --debug=<debuglevel></term> - <listitem> - <para> - Change the debug level for the command. Default is 0. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>--socket=<filename></term> - <listitem> - <para> - Specify the socketname to use when connecting to the local ctdb - daemon. The default is /tmp/ctdb.socket . - </para> - <para> - You only need to specify this parameter if you run multiple ctdb - daemons on the same physical host and thus can not use the default - name for the domain socket. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>--print-emptyrecords</term> - <listitem> - <para> - This enables printing of empty records when dumping databases - with the catdb, cattbd and dumpdbbackup commands. Records with - empty data segment are considered deleted by ctdb and cleaned - by the vacuuming mechanism, so this switch can come in handy for - debugging the vacuuming behaviour. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>--print-datasize</term> - <listitem> - <para> - This lets database dumps (catdb, cattdb, dumpdbbackup) print the - size of the record data instead of dumping the data contents. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>--print-lmaster</term> - <listitem> - <para> - This lets catdb print the lmaster for each record. - </para> - </listitem> + <listitem> + <para> + Print useage information to the screen. + </para> + </listitem> </varlistentry> - <varlistentry><term>--print-hash</term> - <listitem> - <para> - This lets database dumps (catdb, cattdb, dumpdbbackup) print the - hash for each record. - </para> - </listitem> + <varlistentry><term>-d --debug=<parameter>DEBUGLEVEL</parameter></term> + <listitem> + <para> + Change the debug level for the command. Default is ERR (0). + </para> + </listitem> </varlistentry> - <varlistentry><term>--print-recordflags</term> - <listitem> - <para> - This lets catdb and dumpdbbackup print the - record flags for each record. Note that cattdb always - prints the flags. - </para> - </listitem> + <varlistentry><term>--socket=<parameter>FILENAME</parameter></term> + <listitem> + <para> + Specify that FILENAME is the name of the Unix domain + socket to use when connecting to the local CTDB + daemon. The default is + <filename>/tmp/ctdb.socket</filename>. + </para> + </listitem> </varlistentry> - </variablelist> </refsect1> - <refsect1><title>Administrative Commands</title> + <refsect1> + <title>Administrative Commands</title> <para> - These are commands used to monitor and administrate a CTDB cluster. + These are commands used to monitor and administer a CTDB cluster. </para> - <refsect2><title>pnn</title> - <para> - This command displays the pnn of the current node. - </para> - </refsect2> - - <refsect2><title>status</title> - <para> - This command shows the current status of the ctdb node. - </para> - - <refsect3><title>node status</title> - <para> - Node status reflects the current status of the node. There are five possible states: - </para> - <para> - OK - This node is fully functional. - </para> - <para> - DISCONNECTED - This node could not be connected through the network and is currently not participating in the cluster. If there is a public IP address associated with this node it should have been taken over by a different node. No services are running on this node. - </para> - <para> - DISABLED - This node has been administratively disabled. This node is still functional and participates in the CTDB cluster but its IP addresses have been taken over by a different node and no services are currently being hosted. - </para> - <para> - UNHEALTHY - A service provided by this node is malfunctioning and should be investigated. The CTDB daemon itself is operational and participates in the cluster. Its public IP address has been taken over by a different node and no services are currnetly being hosted. All unhealthy nodes should be investigated and require an administrative action to rectify. - </para> - <para> - BANNED - This node failed too many recovery attempts and has been banned from participating in the cluster for a period of RecoveryBanPeriod seconds. Any public IP address has been taken over by other nodes. This node does not provide any services. All banned nodes should be investigated and require an administrative action to rectify. This node does not perticipate in the CTDB cluster but can still be communicated with. I.e. ctdb commands can be sent to it. - </para> - <para> - STOPPED - A node that is stopped does not host any public ip addresses, - nor is it part of the VNNMAP. A stopped node can not become LVSMASTER, - RECMASTER or NATGW. - This node does not perticipate in the CTDB cluster but can still be - communicated with. I.e. ctdb commands can be sent to it. - </para> - <para> - PARTIALLYONLINE - A node that is partially online participates - in a cluster like a node that is ok. Some interfaces to serve - public ip addresses are down, but at least one interface is up. - See also "ctdb ifaces". - </para> + <refsect2> + <title>pnn</title> + <para> + This command displays the PNN of the current node. + </para> + </refsect2> + + <refsect2> + <title>xpnn</title> + <para> + This command displays the PNN of the current node without + contacting the CTDB daemon. It parses the nodes file + directly, so can produce unexpected output if the nodes file + has been edited but has not been reloaded. + </para> + </refsect2> + + <refsect2> + <title>status</title> + <para> + This command shows the current status of all CTDB nodes based + on information from the queried node. + </para> + + <para> + Note: If the the queried node is INACTIVE then the status + might not be current. + </para> + + <refsect3> + <title>Node status</title> + <para> + This includes the number of physical nodes and the status of + each node. See <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for information + about node states. + </para> </refsect3> - <refsect3><title>generation</title> - <para> - The generation id is a number that indicates the current generation - of a cluster instance. Each time a cluster goes through a - reconfiguration or a recovery its generation id will be changed. - </para> - <para> - This number does not have any particular meaning other than to keep - track of when a cluster has gone through a recovery. It is a random - number that represents the current instance of a ctdb cluster - and its databases. - CTDBD uses this number internally to be able to tell when commands - to operate on the cluster and the databases was issued in a different - generation of the cluster, to ensure that commands that operate - on the databases will not survive across a cluster database recovery. - After a recovery, all old outstanding commands will automatically - become invalid. - </para> - <para> + <refsect3> + <title>Generation</title> + <para> + The generation id is a number that indicates the current generation + of a cluster instance. Each time a cluster goes through a + reconfiguration or a recovery its generation id will be changed. + </para> + <para> + This number does not have any particular meaning other than + to keep track of when a cluster has gone through a + recovery. It is a random number that represents the current + instance of a ctdb cluster and its databases. The CTDB + daemon uses this number internally to be able to tell when + commands to operate on the cluster and the databases was + issued in a different generation of the cluster, to ensure + that commands that operate on the databases will not survive + across a cluster database recovery. After a recovery, all + old outstanding commands will automatically become invalid. + </para> + <para> Sometimes this number will be shown as "INVALID". This only means that the ctdbd daemon has started but it has not yet merged with the cluster through a recovery. All nodes start with generation "INVALID" and are not assigned a real generation id until they have successfully been merged with a cluster through a recovery. - </para> + </para> </refsect3> - <refsect3><title>VNNMAP</title> - <para> - The list of Virtual Node Numbers. This is a list of all nodes that actively participates in the cluster and that share the workload of hosting the Clustered TDB database records. - Only nodes that are participating in the vnnmap can become lmaster or dmaster for a database record. - </para> + <refsect3> + <title>Virtual Node Number (VNN) map</title> + <para> + Consists of the number of virtual nodes and mapping from + virtual node numbers to physical node numbers. Virtual + nodes host CTDB databases. Only nodes that are + participating in the VNN map can become lmaster or dmaster + for database records. + </para> </refsect3> - <refsect3><title>Recovery mode</title> - <para> - This is the current recovery mode of the cluster. There are two possible modes: - </para> - <para> - NORMAL - The cluster is fully operational. - </para> - <para> - RECOVERY - The cluster databases have all been frozen, pausing all services while the cluster awaits a recovery process to complete. A recovery process should finish within seconds. If a cluster is stuck in the RECOVERY state this would indicate a cluster malfunction which needs to be investigated. - </para> + <refsect3> + <title>Recovery mode</title> <para> - Once the recovery master detects an inconsistency, for example a node - becomes disconnected/connected, the recovery daemon will trigger a - cluster recovery process, where all databases are remerged across the - cluster. When this process starts, the recovery master will first - "freeze" all databases to prevent applications such as samba from - accessing the databases and it will also mark the recovery mode as - RECOVERY. + This is the current recovery mode of the cluster. There are two possible modes: </para> <para> - When CTDBD starts up, it will start in RECOVERY mode. - Once the node has been merged into a cluster and all databases - have been recovered, the node mode will change into NORMAL mode - and the databases will be "thawed", allowing samba to access the - databases again. + NORMAL - The cluster is fully operational. + </para> + <para> + RECOVERY - The cluster databases have all been frozen, pausing all services while the cluster awaits a recovery process to complete. A recovery process should finish within seconds. If a cluster is stuck in the RECOVERY state this would indicate a cluster malfunction which needs to be investigated. + </para> + <para> + Once the recovery master detects an inconsistency, for example a node + becomes disconnected/connected, the recovery daemon will trigger a + cluster recovery process, where all databases are remerged across the + cluster. When this process starts, the recovery master will first + "freeze" all databases to prevent applications such as samba from + accessing the databases and it will also mark the recovery mode as + RECOVERY. + </para> + <para> + When the CTDB daemon starts up, it will start in RECOVERY + mode. Once the node has been merged into a cluster and all + databases have been recovered, the node mode will change into + NORMAL mode and the databases will be "thawed", allowing samba + to access the databases again. </para> </refsect3> - - <refsect3><title>Recovery master</title> - <para> - This is the cluster node that is currently designated as the recovery master. This node is responsible of monitoring the consistency of the cluster and to perform the actual recovery process when reqired. - </para> - <para> - Only one node at a time can be the designated recovery master. Which - node is designated the recovery master is decided by an election - process in the recovery daemons running on each node. - </para> + <refsect3> + <title>Recovery master</title> + <para> + This is the cluster node that is currently designated as the recovery master. This node is responsible of monitoring the consistency of the cluster and to perform the actual recovery process when reqired. + </para> + <para> + Only one node at a time can be the designated recovery master. Which + node is designated the recovery master is decided by an election + process in the recovery daemons running on each node. + </para> </refsect3> - <para> - Example: ctdb status - </para> - <para>Example output:</para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb status Number of nodes:4 -pnn:0 11.1.2.200 OK (THIS NODE) -pnn:1 11.1.2.201 OK -pnn:2 11.1.2.202 OK -pnn:3 11.1.2.203 OK +pnn:0 192.168.2.200 OK (THIS NODE) +pnn:1 192.168.2.201 OK +pnn:2 192.168.2.202 OK +pnn:3 192.168.2.203 OK Generation:1362079228 Size:4 hash:0 lmaster:0 @@ -323,14 +281,16 @@ hash:2 lmaster:2 hash:3 lmaster:3 Recovery mode:NORMAL (0) Recovery master:0 - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>nodestatus [<nodespec>]</title> + <refsect2> + <title>nodestatus <optional><parameter>PNN-LIST</parameter></optional></title> <para> - This command is similar to the <command>status</command> - command. It displays the "node status" subset of output. The - main differences are: + This command is similar to the <command>status</command> + command. It displays the "node status" subset of output. The + main differences are: </para> <itemizedlist> @@ -347,7 +307,7 @@ Recovery master:0 <command>ctdb status</command> provides status information for all nodes. <command>ctdb nodestatus</command> defaults to providing status for only the current node. - If <nodespec> is provided then status is given for + If PNN-LIST is provided then status is given for the indicated node(s). </para> @@ -370,163 +330,145 @@ Recovery master:0 healthy. </para> - <para> - Example: ctdb nodestatus - </para> - <para>Example output:</para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> # ctdb nodestatus pnn:0 10.0.0.30 OK (THIS NODE) - </screen> - <para> - Example: ctdb nodestatus all - </para> - <para>Example output:</para> - <screen format="linespecific"> # ctdb nodestatus all Number of nodes:2 pnn:0 10.0.0.30 OK (THIS NODE) pnn:1 10.0.0.31 OK - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>recmaster</title> + <refsect2> + <title>recmaster</title> <para> - This command shows the pnn of the node which is currently the recmaster. + This command shows the pnn of the node which is currently the recmaster. </para> - </refsect2> - <refsect2><title>uptime</title> <para> - This command shows the uptime for the ctdb daemon. When the last recovery or ip-failover completed and how long it took. If the "duration" is shown as a negative number, this indicates that there is a recovery/failover in progress and it started that many seconds ago. + Note: If the the queried node is INACTIVE then the status + might not be current. </para> + </refsect2> + <refsect2> + <title>uptime</title> <para> - Example: ctdb uptime + This command shows the uptime for the ctdb daemon. When the last recovery or ip-failover completed and how long it took. If the "duration" is shown as a negative number, this indicates that there is a recovery/failover in progress and it started that many seconds ago. </para> - <para>Example output:</para> - <screen format="linespecific"> + + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb uptime Current time of node : Thu Oct 29 10:38:54 2009 Ctdbd start time : (000 16:54:28) Wed Oct 28 17:44:26 2009 Time of last recovery/failover: (000 16:53:31) Wed Oct 28 17:45:23 2009 Duration of last recovery/failover: 2.248552 seconds - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>listnodes</title> + <refsect2> + <title>listnodes</title> <para> - This command shows lists the ip addresses of all the nodes in the cluster. + This command shows lists the ip addresses of all the nodes in the cluster. </para> - <para> - Example: ctdb listnodes - </para> - <para>Example output:</para> - <screen format="linespecific"> -10.0.0.71 -10.0.0.72 -10.0.0.73 -10.0.0.74 - </screen> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb listnodes +192.168.2.200 +192.168.2.201 +192.168.2.202 +192.168.2.203 + </screen> + </refsect3> </refsect2> - <refsect2><title>ping</title> - <para> - This command will "ping" specified CTDB node in the cluster to - verify that they are running. - </para> + <refsect2> + <title>natgwlist</title> <para> - Example: ctdb ping -n all + Show the current NAT gateway master and the status of all + nodes in the current NAT gateway group. See the + <citetitle>NAT GATEWAY</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more details. </para> + + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb natgwlist +0 192.168.2.200 +Number of nodes:4 +pnn:0 192.168.2.200 OK (THIS NODE) +pnn:1 192.168.2.201 OK +pnn:2 192.168.2.202 OK +pnn:3 192.168.2.203 OK + </screen> + </refsect3> + </refsect2> + + <refsect2> + <title>ping</title> <para> - Example output: + This command will "ping" specified CTDB nodes in the cluster + to verify that they are running. </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb ping -n all response from 0 time=0.000054 sec (3 clients) response from 1 time=0.000144 sec (2 clients) response from 2 time=0.000105 sec (2 clients) response from 3 time=0.000114 sec (2 clients) - </screen> - </refsect2> - - <refsect2><title>runstate [setup|first_recovery|startup|running]</title> - <para> - Print the runstate of the specified node. Runstates are used - to serialise important state transitions in CTDB, particularly - during startup. - </para> - <para> - If one or more optional runstate arguments are specified then - the node must be in one of these runstates for the command to - succeed. - </para> - <para> - Example: ctdb runstate - </para> - <para> - Example output: - </para> - <screen format="linespecific"> -RUNNING - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>ifaces</title> + <refsect2> + <title>ifaces</title> <para> This command will display the list of network interfaces, which could host public addresses, along with their status. </para> - <para> - Example: ctdb ifaces - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb ifaces Interfaces on node 0 name:eth5 link:up references:2 name:eth4 link:down references:0 name:eth3 link:up references:1 name:eth2 link:up references:1 - </screen> - <para> - Example: ctdb ifaces -Y - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + +# ctdb ifaces -Y :Name:LinkStatus:References: :eth5:1:2 :eth4:0:0 :eth3:1:1 :eth2:1:1 - </screen> - </refsect2> - - <refsect2><title>setifacelink <iface> <status></title> - <para> - This command will set the status of a network interface. - The status needs to be "up" or "down". This is typically - used in the 10.interfaces script in the "monitor" event. - </para> - <para> - Example: ctdb setifacelink eth0 up - </para> + </screen> + </refsect3> </refsect2> - <refsect2><title>ip</title> + <refsect2> + <title>ip</title> <para> - This command will display the list of public addresses that are provided by the cluster and which physical node is currently serving this ip. By default this command will ONLY show those public addresses that are known to the node itself. To see the full list of all public ips across the cluster you must use "ctdb ip -n all". + This command will display the list of public addresses that are provided by the cluster and which physical node is currently serving this ip. By default this command will ONLY show those public addresses that are known to the node itself. To see the full list of all public ips across the cluster you must use "ctdb ip -n all". </para> - <para> - Example: ctdb ip - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb ip Public IPs on node 0 172.31.91.82 node[1] active[] available[eth2,eth3] configured[eth2,eth3] 172.31.91.83 node[0] active[eth3] available[eth2,eth3] configured[eth2,eth3] @@ -536,14 +478,8 @@ Public IPs on node 0 172.31.92.83 node[0] active[eth5] available[eth5] configured[eth4,eth5] 172.31.92.84 node[1] active[] available[eth5] configured[eth4,eth5] 172.31.92.85 node[0] active[eth5] available[eth5] configured[eth4,eth5] - </screen> - <para> - Example: ctdb ip -Y - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + +# ctdb ip -Y :Public IP:Node:ActiveInterface:AvailableInterfaces:ConfiguredInterfaces: :172.31.91.82:1::eth2,eth3:eth2,eth3: :172.31.91.83:0:eth3:eth2,eth3:eth2,eth3: @@ -553,40 +489,38 @@ Public IPs on node 0 :172.31.92.83:0:eth5:eth5:eth4,eth5: :172.31.92.84:1::eth5:eth4,eth5: :172.31.92.85:0:eth5:eth5:eth4,eth5: - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>ipinfo <ip></title> - <para> - This command will display details about the specified public addresses. - </para> + <refsect2> + <title>ipinfo <parameter>IP</parameter></title> <para> - Example: ctdb ipinfo 172.31.92.85 + This command will display details about the specified public addresses. </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb ipinfo 172.31.92.85 Public IP[172.31.92.85] info on node 0 IP:172.31.92.85 CurrentNode:0 NumInterfaces:2 Interface[1]: Name:eth4 Link:down References:0 Interface[2]: Name:eth5 Link:up References:2 (active) - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>scriptstatus</title> - <para> - This command displays which scripts where run in the previous monitoring cycle and the result of each script. If a script failed with an error, causing the node to become unhealthy, the output from that script is also shown. - </para> + <refsect2> + <title>scriptstatus</title> <para> - Example: ctdb scriptstatus + This command displays which scripts where run in the previous monitoring cycle and the result of each script. If a script failed with an error, causing the node to become unhealthy, the output from that script is also shown. </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb scriptstatus 7 scripts were executed last monitoring cycle 00.ctdb Status:OK Duration:0.056 Tue Mar 24 18:56:57 2009 10.interface Status:OK Duration:0.077 Tue Mar 24 18:56:57 2009 @@ -596,11 +530,13 @@ Interface[2]: Name:eth5 Link:up References:2 (active) 40.vsftpd Status:OK Duration:0.045 Tue Mar 24 18:56:57 2009 41.httpd Status:OK Duration:0.039 Tue Mar 24 18:56:57 2009 50.samba Status:ERROR Duration:0.082 Tue Mar 24 18:56:57 2009 - OUTPUT:ERROR: Samba tcp port 445 is not responding +OUTPUT:ERROR: Samba tcp port 445 is not responding </screen> + </refsect3> </refsect2> - <refsect2><title>disablescript <script></title> + <refsect2> + <title>disablescript <parameter>SCRIPT</parameter></title> <para> This command is used to disable an eventscript. </para> @@ -609,7 +545,8 @@ Interface[2]: Name:eth5 Link:up References:2 (active) </para> </refsect2> - <refsect2><title>enablescript <script></title> + <refsect2> + <title>enablescript <parameter>SCRIPT</parameter></title> <para> This command is used to enable an eventscript. </para> @@ -618,43 +555,17 @@ Interface[2]: Name:eth5 Link:up References:2 (active) </para> </refsect2> - <refsect2><title>getvar <name></title> - <para> - Get the runtime value of a tuneable variable. - </para> - <para> - Example: ctdb getvar MaxRedirectCount - </para> - <para> - Example output: - </para> - <screen format="linespecific"> -MaxRedirectCount = 3 - </screen> - </refsect2> - - <refsect2><title>setvar <name> <value></title> + <refsect2> + <title>listvars</title> <para> - Set the runtime value of a tuneable variable. + List all tuneable variables, except the values of the obsolete tunables + like VacuumMinInterval. The obsolete tunables can be retrieved only + explicitly with the "ctdb getvar" command. </para> - <para> - Example: ctdb setvar MaxRedirectCount 5 - </para> - </refsect2> - - <refsect2><title>listvars</title> - <para> - List all tuneable variables, except the values of the obsolete tunables - like VacuumMinInterval. The obsolete tunables can be retrieved only - explicitly with the "ctdb getvar" command. - </para> - <para> - Example: ctdb listvars - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb listvars MaxRedirectCount = 3 SeqnumInterval = 1000 ControlTimeout = 60 @@ -699,38 +610,65 @@ StatHistoryInterval = 1 DeferredAttachTO = 120 AllowClientDBAttach = 1 RecoverPDBBySeqNum = 0 - </screen> + </screen> + </refsect3> + </refsect2> + + <refsect2> + <title>getvar <parameter>NAME</parameter></title> + <para> + Get the runtime value of a tuneable variable. + </para> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb getvar MaxRedirectCount +MaxRedirectCount = 3 + </screen> + </refsect3> + </refsect2> + + <refsect2> + <title>setvar <parameter>NAME</parameter> <parameter>VALUE</parameter></title> + <para> + Set the runtime value of a tuneable variable. + </para> + <para> + Example: ctdb setvar MaxRedirectCount 5 + </para> </refsect2> - <refsect2><title>lvsmaster</title> + <refsect2> + <title>lvsmaster</title> <para> - This command shows which node is currently the LVSMASTER. The - LVSMASTER is the node in the cluster which drives the LVS system and - which receives all incoming traffic from clients. + This command shows which node is currently the LVSMASTER. The + LVSMASTER is the node in the cluster which drives the LVS system and + which receives all incoming traffic from clients. </para> <para> - LVS is the mode where the entire CTDB/Samba cluster uses a single - ip address for the entire cluster. In this mode all clients connect to - one specific node which will then multiplex/loadbalance the clients - evenly onto the other nodes in the cluster. This is an alternative to using - public ip addresses. See the manpage for ctdbd for more information - about LVS. + LVS is the mode where the entire CTDB/Samba cluster uses a single + ip address for the entire cluster. In this mode all clients connect to + one specific node which will then multiplex/loadbalance the clients + evenly onto the other nodes in the cluster. This is an alternative to using + public ip addresses. See the manpage for ctdbd for more information + about LVS. </para> </refsect2> - <refsect2><title>lvs</title> + <refsect2> + <title>lvs</title> <para> - This command shows which nodes in the cluster are currently active in the - LVS configuration. I.e. which nodes we are currently loadbalancing - the single ip address across. + This command shows which nodes in the cluster are currently active in the + LVS configuration. I.e. which nodes we are currently loadbalancing + the single ip address across. </para> <para> - LVS will by default only loadbalance across those nodes that are both - LVS capable and also HEALTHY. Except if all nodes are UNHEALTHY in which - case LVS will loadbalance across all UNHEALTHY nodes as well. - LVS will never use nodes that are DISCONNECTED, STOPPED, BANNED or - DISABLED. + LVS will by default only loadbalance across those nodes that are both + LVS capable and also HEALTHY. Except if all nodes are UNHEALTHY in which + case LVS will loadbalance across all UNHEALTHY nodes as well. + LVS will never use nodes that are DISCONNECTED, STOPPED, BANNED or + DISABLED. </para> <para> @@ -744,24 +682,14 @@ RecoverPDBBySeqNum = 0 </refsect2> - <refsect2><title>getcapabilities</title> - <para> - This command shows the capabilities of the current node. - Please see manpage for ctdbd for a full list of all capabilities and - more detailed description. - </para> + <refsect2> + <title>getcapabilities</title> <para> - RECMASTER and LMASTER capabilities are primarily used when CTDBD - is used to create a cluster spanning across WAN links. In which case - ctdbd acts as a WAN accelerator. - </para> - - <para> - LVS capabile means that the node is participating in LVS, a mode - where the entire CTDB cluster uses one single ip address for the - entire cluster instead of using public ip address failover. - This is an alternative to using a loadbalancing layer-4 switch. + This command shows the capabilities of the current node. See + the <citetitle>CAPABILITIES</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more details. </para> <para> @@ -771,69 +699,97 @@ RecoverPDBBySeqNum = 0 RECMASTER: YES LMASTER: YES LVS: NO +NATGW: YES </screen> </refsect2> - <refsect2><title>statistics</title> - <para> - Collect statistics from the CTDB daemon about how many calls it has served. - </para> + <refsect2> + <title>statistics</title> <para> - Example: ctdb statistics + Collect statistics from the CTDB daemon about how many calls it has served. </para> - <para> - Example output: - </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb statistics CTDB version 1 - num_clients 3 - frozen 0 - recovering 0 - client_packets_sent 360489 - client_packets_recv 360466 - node_packets_sent 480931 - node_packets_recv 240120 - keepalive_packets_sent 4 - keepalive_packets_recv 3 - node - req_call 2 - reply_call 2 - req_dmaster 0 - reply_dmaster 0 - reply_error 0 - req_message 42 - req_control 120408 - reply_control 360439 - client - req_call 2 - req_message 24 - req_control 360440 - timeouts - call 0 - control 0 - traverse 0 - total_calls 2 - pending_calls 0 - lockwait_calls 0 - pending_lockwait_calls 0 - memory_used 5040 - max_hop_count 0 - max_call_latency 4.948321 sec - max_lockwait_latency 0.000000 sec - </screen> +num_clients 3 +frozen 0 +recovering 0 +client_packets_sent 360489 +client_packets_recv 360466 +node_packets_sent 480931 +node_packets_recv 240120 +keepalive_packets_sent 4 +keepalive_packets_recv 3 +node +req_call 2 +reply_call 2 +req_dmaster 0 +reply_dmaster 0 +reply_error 0 +req_message 42 +req_control 120408 +reply_control 360439 +client +req_call 2 +req_message 24 +req_control 360440 +timeouts +call 0 +control 0 +traverse 0 +total_calls 2 +pending_calls 0 +lockwait_calls 0 +pending_lockwait_calls 0 +memory_used 5040 +max_hop_count 0 +max_call_latency 4.948321 sec +max_lockwait_latency 0.000000 sec + </screen> + </refsect3> </refsect2> - <refsect2><title>statisticsreset</title> + <refsect2> + <title>statisticsreset</title> <para> - This command is used to clear all statistics counters in a node. + This command is used to clear all statistics counters in a node. </para> <para> Example: ctdb statisticsreset </para> </refsect2> - <refsect2><title>getreclock</title> + <refsect2> + <title>dbstatistics <parameter>DBNAME</parameter>|<parameter>HASH</parameter></title> + <para> + Display statistics about the specified database. + </para> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb dbstatistics locking.tdb +DB Statistics: locking.tdb + ro_delegations 0 + ro_revokes 0 + locks + total 14356 + failed 0 + current 0 + pending 0 + hop_count_buckets: 28087 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 + lock_buckets: 0 14188 38 76 32 19 3 0 0 0 0 0 0 0 0 0 + locks_latency MIN/AVG/MAX 0.001066/0.012686/4.202292 sec out of 14356 + Num Hot Keys: 1 + Count:8 Key:ff5bd7cb3ee3822edc1f0000000000000000000000000000 + </screen> + </refsect3> + </refsect2> + + <refsect2> + <title>getreclock</title> <para> This command is used to show the filename of the reclock file that is used. </para> @@ -842,12 +798,13 @@ CTDB version 1 Example output: </para> <screen format="linespecific"> -Reclock file:/gpfs/.ctdb/shared + Reclock file:/gpfs/.ctdb/shared </screen> </refsect2> - <refsect2><title>setreclock [filename]</title> + <refsect2> + <title>setreclock [filename]</title> <para> This command is used to modify, or clear, the file that is used as the reclock file at runtime. When this command is used, the reclock file checks are disabled. To re-enable the checks the administrator needs to activate the "VerifyRecoveryLock" tunable using "ctdb setvar". </para> @@ -863,9 +820,10 @@ Reclock file:/gpfs/.ctdb/shared - <refsect2><title>getdebug</title> + <refsect2> + <title>getdebug</title> <para> - Get the current debug level for the node. the debug level controls what information is written to the log file. + Get the current debug level for the node. the debug level controls what information is written to the log file. </para> <para> The debug levels are mapped to the corresponding syslog levels. @@ -880,40 +838,45 @@ Reclock file:/gpfs/.ctdb/shared </para> </refsect2> - <refsect2><title>setdebug <debuglevel></title> + <refsect2> + <title>setdebug <parameter>DEBUGLEVEL</parameter></title> <para> - Set the debug level of a node. This controls what information will be logged. + Set the debug level of a node. This controls what information will be logged. </para> <para> The debuglevel is one of EMERG ALERT CRIT ERR WARNING NOTICE INFO DEBUG </para> </refsect2> - <refsect2><title>getpid</title> + <refsect2> + <title>getpid</title> <para> - This command will return the process id of the ctdb daemon. + This command will return the process id of the ctdb daemon. </para> </refsect2> - <refsect2><title>disable</title> + <refsect2> + <title>disable</title> <para> - This command is used to administratively disable a node in the cluster. - A disabled node will still participate in the cluster and host - clustered TDB records but its public ip address has been taken over by - a different node and it no longer hosts any services. + This command is used to administratively disable a node in the cluster. + A disabled node will still participate in the cluster and host + clustered TDB records but its public ip address has been taken over by + a different node and it no longer hosts any services. </para> </refsect2> - <refsect2><title>enable</title> + <refsect2> + <title>enable</title> <para> - Re-enable a node that has been administratively disabled. + Re-enable a node that has been administratively disabled. </para> </refsect2> - <refsect2><title>stop</title> + <refsect2> + <title>stop</title> <para> - This command is used to administratively STOP a node in the cluster. - A STOPPED node is connected to the cluster but will not host any + This command is used to administratively STOP a node in the cluster. + A STOPPED node is connected to the cluster but will not host any public ip addresse, nor does it participate in the VNNMAP. The difference between a DISABLED node and a STOPPED node is that a STOPPED node does not host any parts of the database which means @@ -921,13 +884,15 @@ Reclock file:/gpfs/.ctdb/shared </para> </refsect2> - <refsect2><title>continue</title> + <refsect2> + <title>continue</title> <para> - Re-start a node that has been administratively stopped. + Re-start a node that has been administratively stopped. </para> </refsect2> - <refsect2><title>addip <public_ip/mask> <iface></title> + <refsect2> + <title>addip <parameter>IPADDR</parameter>/<parameter>mask</parameter> <parameter>IFACE</parameter></title> <para> This command is used to add a new public ip to a node during runtime. This allows public addresses to be added to a cluster without having @@ -935,59 +900,50 @@ Reclock file:/gpfs/.ctdb/shared </para> <para> Note that this only updates the runtime instance of ctdb. Any changes will be lost next time ctdb is restarted and the public addresses file is re-read. - If you want this change to be permanent you must also update the public addresses file manually. + If you want this change to be permanent you must also update the public addresses file manually. </para> </refsect2> - <refsect2><title>delip <public_ip></title> + <refsect2> + <title>delip <parameter>IPADDR</parameter></title> <para> This command is used to remove a public ip from a node during runtime. If this public ip is currently hosted by the node it being removed from, the ip will first be failed over to another node, if possible, before it is removed. </para> <para> Note that this only updates the runtime instance of ctdb. Any changes will be lost next time ctdb is restarted and the public addresses file is re-read. - If you want this change to be permanent you must also update the public addresses file manually. + If you want this change to be permanent you must also update the public addresses file manually. </para> </refsect2> - <refsect2><title>moveip <public_ip> <node></title> + <refsect2> + <title>moveip <parameter>IPADDR</parameter> <parameter>PNN</parameter></title> <para> - This command can be used to manually fail a public ip address to a - specific node. + This command can be used to manually fail a public ip address to a + specific node. </para> <para> - In order to manually override the "automatic" distribution of public - ip addresses that ctdb normally provides, this command only works - when you have changed the tunables for the daemon to: + In order to manually override the "automatic" distribution of public + ip addresses that ctdb normally provides, this command only works + when you have changed the tunables for the daemon to: </para> <para> - DeterministicIPs = 0 + DeterministicIPs = 0 </para> <para> - NoIPFailback = 1 + NoIPFailback = 1 </para> </refsect2> - <refsect2><title>shutdown</title> + <refsect2> + <title>shutdown</title> <para> - This command will shutdown a specific CTDB daemon. + This command will shutdown a specific CTDB daemon. </para> </refsect2> - <refsect2><title>recover</title> - <para> - This command will trigger the recovery daemon to do a cluster - recovery. - </para> - </refsect2> - - <refsect2><title>ipreallocate</title> - <para> - This command will force the recovery master to perform a full ip reallocation process and redistribute all ip addresses. This is useful to "reset" the allocations back to its default state if they have been changed using the "moveip" command. While a "recover" will also perform this reallocation, a recovery is much more hevyweight since it will also rebuild all the databases. - </para> - </refsect2> - - <refsect2><title>setlmasterrole <on|off></title> + <refsect2> + <title>setlmasterrole on|off</title> <para> This command is used ot enable/disable the LMASTER capability for a node at runtime. This capability determines whether or not a node can be used as an LMASTER for records in the database. A node that does not have the LMASTER capability will not show up in the vnnmap. </para> @@ -1003,7 +959,8 @@ Reclock file:/gpfs/.ctdb/shared </para> </refsect2> - <refsect2><title>setrecmasterrole <on|off></title> + <refsect2> + <title>setrecmasterrole on|off</title> <para> This command is used ot enable/disable the RECMASTER capability for a node at runtime. This capability determines whether or not a node can be used as an RECMASTER for the cluster. A node that does not have the RECMASTER capability can not win a recmaster election. A node that already is the recmaster for the cluster when the capability is stripped off the node will remain the recmaster until the next cluster election. </para> @@ -1016,232 +973,144 @@ Reclock file:/gpfs/.ctdb/shared </para> </refsect2> - <refsect2><title>killtcp [<srcip:port> <dstip:port>]</title> - <para> - This command will kill the specified TCP connections by - issuing a TCP RST to the srcip:port endpoint. A single - connection can be specified on the command-line, otherwise - connections are read one-per-line from standard input. This - is a command used by the ctdb eventscripts. - </para> - </refsect2> - - <refsect2><title>gratiousarp <ip> <interface></title> - <para> - This command will send out a gratious arp for the specified interface - through the specified interface. This command is mainly used by the - ctdb eventscripts. - </para> - </refsect2> - - <refsect2><title>reloadnodes</title> + <refsect2> + <title>reloadnodes</title> <para> - This command is used when adding new nodes, or removing existing nodes from an existing cluster. + This command is used when adding new nodes, or removing existing nodes from an existing cluster. </para> <para> - Procedure to add a node: + Procedure to add a node: </para> <para> - 1, To expand an existing cluster, first ensure with 'ctdb status' that - all nodes are up and running and that they are all healthy. - Do not try to expand a cluster unless it is completely healthy! + 1, To expand an existing cluster, first ensure with 'ctdb status' that + all nodes are up and running and that they are all healthy. + Do not try to expand a cluster unless it is completely healthy! </para> <para> - 2, On all nodes, edit /etc/ctdb/nodes and add the new node as the last - entry to the file. The new node MUST be added to the end of this file! + 2, On all nodes, edit /etc/ctdb/nodes and add the new node as the last + entry to the file. The new node MUST be added to the end of this file! </para> <para> - 3, Verify that all the nodes have identical /etc/ctdb/nodes files after you edited them and added the new node! + 3, Verify that all the nodes have identical /etc/ctdb/nodes files after you edited them and added the new node! </para> <para> - 4, Run 'ctdb reloadnodes' to force all nodes to reload the nodesfile. + 4, Run 'ctdb reloadnodes' to force all nodes to reload the nodesfile. </para> <para> - 5, Use 'ctdb status' on all nodes and verify that they now show the additional node. + 5, Use 'ctdb status' on all nodes and verify that they now show the additional node. </para> <para> - 6, Install and configure the new node and bring it online. + 6, Install and configure the new node and bring it online. </para> <para> - Procedure to remove a node: + Procedure to remove a node: </para> <para> - 1, To remove a node from an existing cluster, first ensure with 'ctdb status' that - all nodes, except the node to be deleted, are up and running and that they are all healthy. - Do not try to remove a node from a cluster unless the cluster is completely healthy! + 1, To remove a node from an existing cluster, first ensure with 'ctdb status' that + all nodes, except the node to be deleted, are up and running and that they are all healthy. + Do not try to remove a node from a cluster unless the cluster is completely healthy! </para> <para> - 2, Shutdown and poweroff the node to be removed. + 2, Shutdown and poweroff the node to be removed. </para> <para> - 3, On all other nodes, edit the /etc/ctdb/nodes file and comment out the node to be removed. Do not delete the line for that node, just comment it out by adding a '#' at the beginning of the line. + 3, On all other nodes, edit the /etc/ctdb/nodes file and comment out the node to be removed. Do not delete the line for that node, just comment it out by adding a '#' at the beginning of the line. </para> <para> - 4, Run 'ctdb reloadnodes' to force all nodes to reload the nodesfile. + 4, Run 'ctdb reloadnodes' to force all nodes to reload the nodesfile. </para> <para> - 5, Use 'ctdb status' on all nodes and verify that the deleted node no longer shows up in the list.. + 5, Use 'ctdb status' on all nodes and verify that the deleted node no longer shows up in the list.. </para> <para> </para> - - </refsect2> - <refsect2><title>reloadips</title> - <para> - This command is used to reload the public addresses file and update the - ip configuration of the running daemon. - </para> - <para> - Procedure to update the public address configuration on a single node: - </para> - <para> - 1, Update the /etc/ctdb/public_addresses file on the node - </para> - <para> - 2, Run 'ctdb reloadips' on the node. - </para> - <para> - The file will then be reloaded on the node and addresses will be added - or removed as required to match the newly loaded file. When updating - a single node it may take a little while before any newly added - addresses are failed onto the node. - </para> - - <para> - </para> - - <para> - Procedure to update the public address configuration on whole cluster: - </para> - <para> - 1, Update the /etc/ctdb/public_addresses file on all nodes - </para> - <para> - 2, Run 'ctdb reloadips -n all'. - </para> - <para> - This command will then force all nodes to reload and update the - addresses. This process is controlled and synchronized by the recovery - master to ensure that all addresses are added to all nodes as one - single operation, after which any required ip node rebalancing may - may take place. - </para> - </refsect2> - - <refsect2><title>tickle <srcip:port> <dstip:port></title> + <refsect2> + <title> + reloadips + <optional><parameter>PNN-LIST</parameter></optional> + </title> <para> - This command will will send a TCP tickle to the source host for the - specified TCP connection. - A TCP tickle is a TCP ACK packet with an invalid sequence and - acknowledge number and will when received by the source host result - in it sending an immediate correct ACK back to the other end. - </para> - <para> - TCP tickles are useful to "tickle" clients after a IP failover has - occured since this will make the client immediately recognize the - TCP connection has been disrupted and that the client will need - to reestablish. This greatly speeds up the time it takes for a client - to detect and reestablish after an IP failover in the ctdb cluster. + This command reloads the public addresses configuration file + on the specified nodes. When it completes addresses will be + reconfigured and reassigned across the cluster as necessary. </para> </refsect2> - <refsect2><title>gettickles <ip></title> - <para> - This command is used to show which TCP connections are registered with - CTDB to be "tickled" if there is a failover. - </para> - </refsect2> - <refsect2><title>repack [max_freelist]</title> - <para> - Over time, when records are created and deleted in a TDB, the TDB list of free space will become fragmented. This can lead to a slowdown in accessing TDB records. - This command is used to defragment a TDB database and pruning the freelist. - </para> - - <para> - If [max_freelist] is specified, then a database will only be repacked if it has more than this number of entries in the freelist. - </para> + <refsect2> + <title>getdbmap</title> <para> - During repacking of the database, the entire TDB database will be locked to prevent writes. If samba tries to write to a record in the database during a repack operation, samba will block until the repacking has completed. + This command lists all clustered TDB databases that the CTDB daemon has attached to. Some databases are flagged as PERSISTENT, this means that the database stores data persistently and the data will remain across reboots. One example of such a database is secrets.tdb where information about how the cluster was joined to the domain is stored. </para> - <para> - This command can be disruptive and can cause samba to block for the duration of the repack operation. In general, a repack operation will take less than one second to complete. + If a PERSISTENT database is not in a healthy state the database is + flagged as UNHEALTHY. If there's at least one completely healthy node running in + the cluster, it's possible that the content is restored by a recovery + run automaticly. Otherwise an administrator needs to analyze the + problem. </para> - <para> - A repack operation will only defragment the local TDB copy of the CTDB database. You need to run this command on all of the nodes to repack a CTDB database completely. - </para> - - <para> - Example: ctdb repack 1000 + See also "ctdb getdbstatus", "ctdb backupdb", "ctdb restoredb", + "ctdb dumpbackup", "ctdb wipedb", "ctdb setvar AllowUnhealthyDBRead 1" + and (if samba or tdb-utils are installed) "tdbtool check". </para> - <para> - By default, this operation is issued from the 00.ctdb event script every 5 minutes. + Most databases are not persistent and only store the state information that the currently running samba daemons need. These databases are always wiped when ctdb/samba starts and when a node is rebooted. </para> - - </refsect2> - <refsect2><title>vacuum [max_records]</title> - <para> - Over time CTDB databases will fill up with empty deleted records which will lead to a progressive slow down of CTDB database access. - This command is used to prune all databases and delete all empty records from the cluster. - </para> - - <para> - By default, vacuum will delete all empty records from all databases. - If [max_records] is specified, the command will only delete the first - [max_records] empty records for each database. - </para> - - <para> - Vacuum only deletes records where the local node is the lmaster. - To delete all records from the entire cluster you need to run a vacuum from each node. + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb getdbmap +Number of databases:10 +dbid:0x435d3410 name:notify.tdb path:/var/ctdb/notify.tdb.0 +dbid:0x42fe72c5 name:locking.tdb path:/var/ctdb/locking.tdb.0 +dbid:0x1421fb78 name:brlock.tdb path:/var/ctdb/brlock.tdb.0 +dbid:0x17055d90 name:connections.tdb path:/var/ctdb/connections.tdb.0 +dbid:0xc0bdde6a name:sessionid.tdb path:/var/ctdb/sessionid.tdb.0 +dbid:0x122224da name:test.tdb path:/var/ctdb/test.tdb.0 +dbid:0x2672a57f name:idmap2.tdb path:/var/ctdb/persistent/idmap2.tdb.0 PERSISTENT +dbid:0xb775fff6 name:secrets.tdb path:/var/ctdb/persistent/secrets.tdb.0 PERSISTENT +dbid:0xe98e08b6 name:group_mapping.tdb path:/var/ctdb/persistent/group_mapping.tdb.0 PERSISTENT +dbid:0x7bbbd26c name:passdb.tdb path:/var/ctdb/persistent/passdb.tdb.0 PERSISTENT - This command is not disruptive. Samba is unaffected and will still be able to read/write records normally while the database is being vacuumed. - </para> +# ctdb getdbmap # example for unhealthy database +Number of databases:1 +dbid:0xb775fff6 name:secrets.tdb path:/var/ctdb/persistent/secrets.tdb.0 PERSISTENT UNHEALTHY - <para> - Example: ctdb vacuum - </para> - - <para> - By default, this operation is issued from the 00.ctdb event script every 5 minutes. - </para> +# ctdb -Y getdbmap +:ID:Name:Path:Persistent:Unhealthy: +:0x7bbbd26c:passdb.tdb:/var/ctdb/persistent/passdb.tdb.0:1:0: + </screen> + </refsect3> </refsect2> - <refsect2><title>backupdb <dbname> <file></title> + <refsect2> + <title>backupdb <parameter>DBNAME</parameter> <parameter>FILE</parameter></title> <para> - This command can be used to copy the entire content of a database out to a file. This file can later be read back into ctdb using the restoredb command. -This is mainly useful for backing up persistent databases such as secrets.tdb and similar. + This command can be used to copy the entire content of a database out to a file. This file can later be read back into ctdb using the restoredb command. + This is mainly useful for backing up persistent databases such as secrets.tdb and similar. </para> </refsect2> - <refsect2><title>restoredb <file> [<dbname>]</title> + <refsect2> + <title>restoredb <parameter>FILE</parameter> [<parameter>DBNAME</parameter>]</title> <para> - This command restores a persistent database that was previously backed up using backupdb. + This command restores a persistent database that was previously backed up using backupdb. By default the data will be restored back into the same database as it was created from. By specifying dbname you can restore the data into a different database. </para> </refsect2> - <refsect2><title>wipedb <dbname></title> + <refsect2> + <title>getlog [<parameter>LEVEL</parameter>] [recoverd]</title> <para> - This command can be used to remove all content of a database. - </para> - </refsect2> - </refsect1> - - - <refsect2><title>getlog [<level>] [recoverd]</title> - <para> - In addition to the normal logging to a log file, - CTDBD also keeps a in-memory ringbuffer containing the most recent - log entries for all log levels (except DEBUG). + In addition to the normal logging to a log file, CTDB also + keeps a in-memory ringbuffer containing the most recent log + entries for all log levels (except DEBUG). </para> <para> This is useful since it allows for keeping continuous logs to a file @@ -1262,7 +1131,8 @@ This is mainly useful for backing up persistent databases such as secrets.tdb an </para> </refsect2> - <refsect2><title>clearlog [recoverd]</title> + <refsect2> + <title>clearlog [recoverd]</title> <para> This command clears the in-memory logging ringbuffer. </para> @@ -1273,257 +1143,532 @@ This is mainly useful for backing up persistent databases such as secrets.tdb an </para> </refsect2> - <refsect2><title>setdbreadonly <dbname|hash></title> + <refsect2> + <title>setdbreadonly <parameter>DBNAME</parameter>|<parameter>HASH</parameter></title> + <para> + This command will enable the read-only record support for a + database. This is an experimental feature to improve + performance for contended records primarily in locking.tdb and + brlock.tdb. When enabling this feature you must set it on all + nodes in the cluster. + </para> + </refsect2> + + <refsect2> + <title>setdbsticky <parameter>DBNAME</parameter>|<parameter>HASH</parameter></title> <para> - This command will enable the ReadOnly record support for a database. - This is an experimental feature to improve performance for contended - records primarily in locking.tdb and brlock.tdb. - When enabling this feature you must set it on all nodes in the cluster. - For now, this feature requires a special patch to samba in order to - use it. + This command will enable the sticky record support for the + specified database. This is an experimental feature to + improve performance for contended records primarily in + locking.tdb and brlock.tdb. When enabling this feature you + must set it on all nodes in the cluster. </para> </refsect2> + </refsect1> + + <refsect1> + <title>Internal commands</title> - <refsect1><title>Debugging Commands</title> <para> - These commands are primarily used for CTDB development and testing and - should not be used for normal administration. + Internal commands are used by CTDB's scripts and are not + required for managing a CTDB cluster. Their parameters and + behaviour are subject to change. </para> - <refsect2><title>process-exists <pid></title> + + <refsect2> + <title>runstate [setup|first_recovery|startup|running]</title> + <para> + Print the runstate of the specified node. Runstates are used + to serialise important state transitions in CTDB, particularly + during startup. + </para> <para> - This command checks if a specific process exists on the CTDB host. This is mainly used by Samba to check if remote instances of samba are still running or not. + If one or more optional runstate arguments are specified then + the node must be in one of these runstates for the command to + succeed. </para> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb runstate +RUNNING + </screen> + </refsect3> </refsect2> - <refsect2><title>getdbmap</title> + <refsect2> + <title>setifacelink <parameter>IFACE</parameter> <parameter>STATUS</parameter></title> <para> - This command lists all clustered TDB databases that the CTDB daemon has attached to. Some databases are flagged as PERSISTENT, this means that the database stores data persistently and the data will remain across reboots. One example of such a database is secrets.tdb where information about how the cluster was joined to the domain is stored. + This command will set the status of a network interface. + The status needs to be "up" or "down". This is typically + used in the 10.interfaces script in the "monitor" event. </para> <para> - If a PERSISTENT database is not in a healthy state the database is - flagged as UNHEALTHY. If there's at least one completely healthy node running in - the cluster, it's possible that the content is restored by a recovery - run automaticly. Otherwise an administrator needs to analyze the - problem. + Example: ctdb setifacelink eth0 up </para> + </refsect2> + + <refsect2> + <title>killtcp</title> <para> - See also "ctdb getdbstatus", "ctdb backupdb", "ctdb restoredb", - "ctdb dumpbackup", "ctdb wipedb", "ctdb setvar AllowUnhealthyDBRead 1" - and (if samba or tdb-utils are installed) "tdbtool check". - </para> - <para> - Most databases are not persistent and only store the state information that the currently running samba daemons need. These databases are always wiped when ctdb/samba starts and when a node is rebooted. + This command reads a list of TCP connections, one per line, + from standard input and terminates each connection. A connection + is specified as: </para> + <synopsis> + <parameter>SRC-IPADDR</parameter>:<parameter>SRC-PORT</parameter> <parameter>DST-IPADDR</parameter>:<parameter>DST-PORT</parameter> + </synopsis> <para> - Example: ctdb getdbmap + A connection is terminated by issuing a TCP RST to the + SRC-IPADDR:SRC-PORT endpoint. </para> <para> - Example output: + A single connection can be specified on the command-line + rather than on standard input. </para> - <screen format="linespecific"> -Number of databases:10 -dbid:0x435d3410 name:notify.tdb path:/var/ctdb/notify.tdb.0 -dbid:0x42fe72c5 name:locking.tdb path:/var/ctdb/locking.tdb.0 -dbid:0x1421fb78 name:brlock.tdb path:/var/ctdb/brlock.tdb.0 -dbid:0x17055d90 name:connections.tdb path:/var/ctdb/connections.tdb.0 -dbid:0xc0bdde6a name:sessionid.tdb path:/var/ctdb/sessionid.tdb.0 -dbid:0x122224da name:test.tdb path:/var/ctdb/test.tdb.0 -dbid:0x2672a57f name:idmap2.tdb path:/var/ctdb/persistent/idmap2.tdb.0 PERSISTENT -dbid:0xb775fff6 name:secrets.tdb path:/var/ctdb/persistent/secrets.tdb.0 PERSISTENT -dbid:0xe98e08b6 name:group_mapping.tdb path:/var/ctdb/persistent/group_mapping.tdb.0 PERSISTENT -dbid:0x7bbbd26c name:passdb.tdb path:/var/ctdb/persistent/passdb.tdb.0 PERSISTENT - </screen> + </refsect2> + + <refsect2> + <title>gratiousarp <parameter>IPADDR</parameter> <parameter>INTERFACE</parameter></title> <para> - Example output for an unhealthy database: + This command will send out a gratious arp for the specified interface + through the specified interface. This command is mainly used by the + ctdb eventscripts. </para> - <screen format="linespecific"> -Number of databases:1 -dbid:0xb775fff6 name:secrets.tdb path:/var/ctdb/persistent/secrets.tdb.0 PERSISTENT UNHEALTHY - </screen> + </refsect2> + <refsect2> + <title>tickle <parameter>SRC-IPADDR</parameter>:<parameter>SRC-PORT</parameter> <parameter>DST-IPADDR</parameter>:<parameter>DST-PORT</parameter></title> <para> - Example output for a healthy database as machinereadable output -Y: + This command will will send a TCP tickle to the source host for the + specified TCP connection. + A TCP tickle is a TCP ACK packet with an invalid sequence and + acknowledge number and will when received by the source host result + in it sending an immediate correct ACK back to the other end. + </para> + <para> + TCP tickles are useful to "tickle" clients after a IP failover has + occured since this will make the client immediately recognize the + TCP connection has been disrupted and that the client will need + to reestablish. This greatly speeds up the time it takes for a client + to detect and reestablish after an IP failover in the ctdb cluster. </para> - <screen format="linespecific"> -:ID:Name:Path:Persistent:Unhealthy: -:0x7bbbd26c:passdb.tdb:/var/ctdb/persistent/passdb.tdb.0:1:0: - </screen> </refsect2> - <refsect2><title>getdbstatus <dbname></title> + <refsect2> + <title>gettickles <parameter>IPADDR</parameter></title> <para> - This command displays more details about a database. + This command is used to show which TCP connections are registered with + CTDB to be "tickled" if there is a failover. </para> + </refsect2> + + <refsect2> + <title>version</title> <para> - Example: ctdb getdbstatus test.tdb.0 + Displays the CTDB version. </para> + </refsect2> + + <refsect2> + <title>setnatgwstate on|off</title> <para> - Example output: + Enable or disable the NAT gateway master capability on a node. </para> - <screen format="linespecific"> + </refsect2> + + </refsect1> + + <refsect1> + <title>Debugging Commands</title> + <para> + These commands are primarily used for CTDB development and testing and + should not be used for normal administration. + </para> + + + <refsect2> + <title>OPTIONS</title> + + <variablelist> + <varlistentry><term>--print-emptyrecords</term> + <listitem> + <para> + This enables printing of empty records when dumping databases + with the catdb, cattbd and dumpdbbackup commands. Records with + empty data segment are considered deleted by ctdb and cleaned + by the vacuuming mechanism, so this switch can come in handy for + debugging the vacuuming behaviour. + </para> + </listitem> + </varlistentry> + + <varlistentry><term>--print-datasize</term> + <listitem> + <para> + This lets database dumps (catdb, cattdb, dumpdbbackup) print the + size of the record data instead of dumping the data contents. + </para> + </listitem> + </varlistentry> + + <varlistentry><term>--print-lmaster</term> + <listitem> + <para> + This lets catdb print the lmaster for each record. + </para> + </listitem> + </varlistentry> + + <varlistentry><term>--print-hash</term> + <listitem> + <para> + This lets database dumps (catdb, cattdb, dumpdbbackup) print the + hash for each record. + </para> + </listitem> + </varlistentry> + + <varlistentry><term>--print-recordflags</term> + <listitem> + <para> + This lets catdb and dumpdbbackup print the + record flags for each record. Note that cattdb always + prints the flags. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect2> + + <refsect2> + <title>process-exists <parameter>PID</parameter></title> + <para> + This command checks if a specific process exists on the CTDB host. This is mainly used by Samba to check if remote instances of samba are still running or not. + </para> + </refsect2> + + <refsect2> + <title>getdbstatus <parameter>DBNAME</parameter></title> + <para> + This command displays more details about a database. + </para> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb getdbstatus test.tdb.0 dbid: 0x122224da name: test.tdb path: /var/ctdb/test.tdb.0 PERSISTENT: no HEALTH: OK - </screen> - <para> - Example: ctdb getdbstatus registry.tdb (with a corrupted TDB) - </para> - <para> - Example output: - </para> - <screen format="linespecific"> + +# ctdb getdbstatus registry.tdb # with a corrupted TDB dbid: 0xf2a58948 name: registry.tdb path: /var/ctdb/persistent/registry.tdb.0 PERSISTENT: yes HEALTH: NO-HEALTHY-NODES - ERROR - Backup of corrupted TDB in '/var/ctdb/persistent/registry.tdb.0.corrupted.20091208091949.0Z' - </screen> + </screen> + </refsect3> </refsect2> - <refsect2><title>catdb <dbname></title> + <refsect2> + <title>catdb <parameter>DBNAME</parameter></title> <para> - This command will dump a clustered TDB database to the screen. This is a debugging command. + This command will dump a clustered TDB database to the screen. This is a debugging command. </para> </refsect2> - <refsect2><title>cattdb <dbname></title> + <refsect2> + <title>cattdb <parameter>DBNAME</parameter></title> <para> - This command will dump the content of the local TDB database to the screen. This is a debugging command. + This command will dump the content of the local TDB database to the screen. This is a debugging command. </para> </refsect2> - <refsect2><title>dumpdbbackup <backup-file></title> + <refsect2> + <title>dumpdbbackup <parameter>BACKUP-FILE</parameter></title> <para> This command will dump the content of database backup to the screen (similar to ctdb catdb). This is a debugging command. </para> </refsect2> - <refsect2><title>getmonmode</title> + <refsect2> + <title>wipedb <parameter>DBNAME</parameter></title> <para> - This command returns the monutoring mode of a node. The monitoring mode is either ACTIVE or DISABLED. Normally a node will continuously monitor that all other nodes that are expected are in fact connected and that they respond to commands. + This command can be used to remove all content of a database. </para> + </refsect2> + + <refsect2> + <title>recover</title> <para> - ACTIVE - This is the normal mode. The node is actively monitoring all other nodes, both that the transport is connected and also that the node responds to commands. If a node becomes unavailable, it will be marked as DISCONNECTED and a recovery is initiated to restore the cluster. + This command will trigger the recovery daemon to do a cluster + recovery. </para> + </refsect2> + + <refsect2> + <title>ipreallocate, sync</title> <para> - DISABLED - This node is not monitoring that other nodes are available. In this mode a node failure will not be detected and no recovery will be performed. This mode is useful when for debugging purposes one wants to attach GDB to a ctdb process but wants to prevent the rest of the cluster from marking this node as DISCONNECTED and do a recovery. + This command will force the recovery master to perform a full ip reallocation process and redistribute all ip addresses. This is useful to "reset" the allocations back to its default state if they have been changed using the "moveip" command. While a "recover" will also perform this reallocation, a recovery is much more hevyweight since it will also rebuild all the databases. + </para> + </refsect2> + + <refsect2> + <title>getmonmode</title> + <para> + This command returns the monutoring mode of a node. The monitoring mode is either ACTIVE or DISABLED. Normally a node will continuously monitor that all other nodes that are expected are in fact connected and that they respond to commands. + </para> + <para> + ACTIVE - This is the normal mode. The node is actively monitoring all other nodes, both that the transport is connected and also that the node responds to commands. If a node becomes unavailable, it will be marked as DISCONNECTED and a recovery is initiated to restore the cluster. + </para> + <para> + DISABLED - This node is not monitoring that other nodes are available. In this mode a node failure will not be detected and no recovery will be performed. This mode is useful when for debugging purposes one wants to attach GDB to a ctdb process but wants to prevent the rest of the cluster from marking this node as DISCONNECTED and do a recovery. </para> </refsect2> - <refsect2><title>setmonmode <0|1></title> + <refsect2> + <title>setmonmode 0|1</title> <para> - This command can be used to explicitly disable/enable monitoring mode on a node. The main purpose is if one wants to attach GDB to a running ctdb daemon but wants to prevent the other nodes from marking it as DISCONNECTED and issuing a recovery. To do this, set monitoring mode to 0 on all nodes before attaching with GDB. Remember to set monitoring mode back to 1 afterwards. + This command can be used to explicitly disable/enable monitoring mode on a node. The main purpose is if one wants to attach GDB to a running ctdb daemon but wants to prevent the other nodes from marking it as DISCONNECTED and issuing a recovery. To do this, set monitoring mode to 0 on all nodes before attaching with GDB. Remember to set monitoring mode back to 1 afterwards. </para> </refsect2> - <refsect2><title>attach <dbname> [persistent]</title> + <refsect2> + <title>attach <parameter>DBNAME</parameter> [persistent]</title> <para> - This is a debugging command. This command will make the CTDB daemon create a new CTDB database and attach to it. + This is a debugging command. This command will make the CTDB daemon create a new CTDB database and attach to it. </para> </refsect2> - <refsect2><title>dumpmemory</title> + <refsect2> + <title>dumpmemory</title> <para> - This is a debugging command. This command will make the ctdb - daemon to write a fill memory allocation map to standard output. + This is a debugging command. This command will make the ctdb + daemon to write a fill memory allocation map to standard output. </para> </refsect2> - <refsect2><title>rddumpmemory</title> + <refsect2> + <title>rddumpmemory</title> <para> - This is a debugging command. This command will dump the talloc memory + This is a debugging command. This command will dump the talloc memory allocation tree for the recovery daemon to standard output. </para> </refsect2> - <refsect2><title>thaw</title> + <refsect2> + <title>thaw</title> <para> - Thaw a previously frozen node. + Thaw a previously frozen node. </para> </refsect2> - <refsect2><title>eventscript <arguments></title> + <refsect2> + <title>eventscript <parameter>ARGUMENTS</parameter></title> <para> - This is a debugging command. This command can be used to manually + This is a debugging command. This command can be used to manually invoke and run the eventscritps with arbitrary arguments. </para> </refsect2> - <refsect2><title>ban <bantime|0></title> + <refsect2> + <title>ban <parameter>BANTIME</parameter></title> <para> - Administratively ban a node for bantime seconds. A bantime of 0 means that the node should be permanently banned. + Administratively ban a node for BANTIME seconds. The node + will be unbanned after BANTIME seconds have elapsed. </para> <para> - A banned node does not participate in the cluster and does not host any records for the clustered TDB. Its ip address has been taken over by another node and no services are hosted. + A banned node does not participate in the cluster. It does + not host any records for the clustered TDB and does not host + any public IP addresses. </para> <para> - Nodes are automatically banned if they are the cause of too many - cluster recoveries. + Nodes are automatically banned if they misbehave. For + example, a node may be banned if it causes too many cluster + recoveries. </para> <para> - This is primarily a testing command. Note that the recovery daemon controls the overall ban state and it may automatically unban nodes at will. Meaning that a node that has been banned by the administrator can and ofter are unbanned before the admin specifid timeout triggers. If wanting to "drop" a node out from the cluster for mainentance or other reasons, use the "stop" / "continue" commands instad of "ban" / "unban". + To administratively exclude a node from a cluster use the + <command>stop</command> command. </para> </refsect2> - <refsect2><title>unban</title> + <refsect2> + <title>unban</title> <para> - This command is used to unban a node that has either been - administratively banned using the ban command or has been automatically - banned by the recovery daemon. + This command is used to unban a node that has either been + administratively banned using the ban command or has been + automatically banned. </para> </refsect2> - - <refsect2><title>check_srvids <srvid> ...</title> - <para> - This command checks whether a set of srvid message ports are registered on the - node or not. The command takes a list of values to check. - </para> + <refsect2> + <title> + rebalancenode + <optional><parameter>PNN-LIST</parameter></optional> + </title> <para> - Example: ctdb check_srvids 1 2 3 14765 + This command marks the given nodes as rebalance targets in the + LCP2 IP allocation algorithm. The + <command>reloadips</command> command will do this as necessary + so this command should not be needed. </para> + </refsect2> + + <refsect2> + <title>check_srvids <parameter>SRVID</parameter> ...</title> <para> - Example output: + This command checks whether a set of srvid message ports are + registered on the node or not. The command takes a list of + values to check. </para> - <screen format="linespecific"> + <refsect3> + <title>Example</title> + <screen format="linespecific"> +# ctdb check_srvids 1 2 3 14765 Server id 0:1 does not exist Server id 0:2 does not exist Server id 0:3 does not exist Server id 0:14765 exists - </screen> + </screen> + </refsect3> + </refsect2> + + <refsect2> + <title>vacuum [<parameter>max-records</parameter>]</title> + <para> + Over time CTDB databases will fill up with empty deleted + records which will lead to a progressive slow down of CTDB + database access. This command is used to prune all databases + and delete all empty records from the cluster. + </para> + + <para> + By default, vacuum will delete all empty records from all databases. + If [max_records] is specified, the command will only delete the first + [max_records] empty records for each database. + </para> + + <para> + Vacuum only deletes records where the local node is the + lmaster. To delete all records from the entire cluster you + need to run a vacuum from each node. + + This command is not disruptive. Samba is unaffected and will still be able to read/write records normally while the database is being vacuumed. + </para> + + <para> + Example: ctdb vacuum + </para> + + <para> + By default, this operation is issued from the 00.ctdb event script every 5 minutes. + </para> + </refsect2> + + <refsect2> + <title>repack [max_freelist]</title> + <para> + Over time, when records are created and deleted in a TDB, the TDB list of free space will become fragmented. This can lead to a slowdown in accessing TDB records. + This command is used to defragment a TDB database and pruning the freelist. + </para> + + <para> + If [max_freelist] is specified, then a database will only be repacked if it has more than this number of entries in the freelist. + </para> + <para> + During repacking of the database, the entire TDB database will be locked to prevent writes. If samba tries to write to a record in the database during a repack operation, samba will block until the repacking has completed. + </para> + + <para> + This command can be disruptive and can cause samba to block for the duration of the repack operation. In general, a repack operation will take less than one second to complete. + </para> + + <para> + A repack operation will only defragment the local TDB copy of the CTDB database. You need to run this command on all of the nodes to repack a CTDB database completely. + </para> + + <para> + Example: ctdb repack 1000 + </para> + + <para> + By default, this operation is issued from the 00.ctdb event script every 5 minutes. + </para> + </refsect2> </refsect1> - <refsect1><title>SEE ALSO</title> + <!-- UNDOCUMENTED: showban stats disablemonitor enablemonitor + isnotrecmaster addtickle deltickle regsrvid unregsrvid chksrvid + getsrvids rebalanceip setdbprio getdbprio msglisten msgsend + pfetch pstore pdelete tfetch tstore readkey writekey + checktcpport getdbseqnum ipiface + --> + + + <refsect1> + <title>SEE ALSO</title> <para> - ctdbd(1), onnode(1) + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>onnode</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb-tunables</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + <ulink url="http://ctdb.samba.org/"/> </para> </refsect1> - <refsect1><title>COPYRIGHT/LICENSE</title> -<literallayout> -Copyright (C) Andrew Tridgell 2007 -Copyright (C) Ronnie sahlberg 2007 - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 3 of the License, or (at -your option) any later version. - -This program is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, see http://www.gnu.org/licenses/. -</literallayout> - </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Ronnie Sahlberg, + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + </refentry> diff --git a/ctdb/doc/ctdb.7.xml b/ctdb/doc/ctdb.7.xml new file mode 100644 index 00000000000..989a28070c3 --- /dev/null +++ b/ctdb/doc/ctdb.7.xml @@ -0,0 +1,1001 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE refentry + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> +<refentry id="ctdb.7"> + +<refmeta> + <refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> +</refmeta> + + +<refnamediv> + <refname>ctdb</refname> + <refpurpose>Clustered TDB</refpurpose> +</refnamediv> + +<refsect1> + <title>DESCRIPTION</title> + + <para> + CTDB is a clustered database component in clustered Samba that + provides a high-availability load-sharing CIFS server cluster. + </para> + + <para> + The main functions of CTDB are: + </para> + + <itemizedlist> + <listitem> + <para> + Provide a clustered version of the TDB database with automatic + rebuild/recovery of the databases upon node failures. + </para> + </listitem> + + <listitem> + <para> + Monitor nodes in the cluster and services running on each node. + </para> + </listitem> + + <listitem> + <para> + Manage a pool of public IP addresses that are used to provide + services to clients. Alternatively, CTDB can be used with + LVS. + </para> + </listitem> + </itemizedlist> + + <para> + Combined with a cluster filesystem CTDB provides a full + high-availablity (HA) environment for services such as clustered + Samba, NFS and other services. + </para> +</refsect1> + +<refsect1> + <title>ANATOMY OF A CTDB CLUSTER</title> + + <para> + A CTDB cluster is a collection of nodes with 2 or more network + interfaces. All nodes provide network (usually file/NAS) services + to clients. Data served by file services is stored on shared + storage (usually a cluster filesystem) that is accessible by all + nodes. + </para> + <para> + CTDB provides an "all active" cluster, where services are load + balanced across all nodes. + </para> +</refsect1> + + <refsect1> + <title>Private vs Public addresses</title> + + <para> + Each node in a CTDB cluster has multiple IP addresses assigned + to it: + + <itemizedlist> + <listitem> + <para> + A single private IP address that is used for communication + between nodes. + </para> + </listitem> + <listitem> + <para> + One or more public IP addresses that are used to provide + NAS or other services. + </para> + </listitem> + </itemizedlist> + </para> + + <refsect2> + <title>Private address</title> + + <para> + Each node is configured with a unique, permanently assigned + private address. This address is configured by the operating + system. This address uniquely identifies a physical node in + the cluster and is the address that CTDB daemons will use to + communicate with the CTDB daemons on other nodes. + </para> + <para> + Private addresses are listed in the file specified by the + <varname>CTDB_NODES</varname> configuration variable (see + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>, default + <filename>/etc/ctdb/nodes</filename>). This file contains the + list of private addresses for all nodes in the cluster, one + per line. This file must be the same on all nodes in the + cluster. + </para> + <para> + Private addresses should not be used by clients to connect to + services provided by the cluster. + </para> + <para> + It is strongly recommended that the private addresses are + configured on a private network that is separate from client + networks. + </para> + + <para> + Example <filename>/etc/ctdb/nodes</filename> for a four node + cluster: + </para> + <screen format="linespecific"> +192.168.1.1 +192.168.1.2 +192.168.1.3 +192.168.1.4 + </screen> + </refsect2> + + <refsect2> + <title>Public addresses</title> + + <para> + Public addresses are used to provide services to clients. + Public addresses are not configured at the operating system + level and are not permanently associated with a particular + node. Instead, they are managed by CTDB and are assigned to + interfaces on physical nodes at runtime. + </para> + <para> + The CTDB cluster will assign/reassign these public addresses + across the available healthy nodes in the cluster. When one + node fails, its public addresses will be taken over by one or + more other nodes in the cluster. This ensures that services + provided by all public addresses are always available to + clients, as long as there are nodes available capable of + hosting this address. + </para> + <para> + The public address configuration is stored in a file on each + node specified by the <varname>CTDB_PUBLIC_ADDRESSES</varname> + configuration variable (see + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>, recommended + <filename>/etc/ctdb/public_addresses</filename>). This file + contains a list of the public addresses that the node is + capable of hosting, one per line. Each entry also contains + the netmask and the interface to which the address should be + assigned. + </para> + + <para> + Example <filename>/etc/ctdb/public_addresses</filename> for a + node that can host 4 public addresses, on 2 different + interfaces: + </para> + <screen format="linespecific"> +10.1.1.1/24 eth1 +10.1.1.2/24 eth1 +10.1.2.1/24 eth2 +10.1.2.2/24 eth2 + </screen> + + <para> + In many cases the public addresses file will be the same on + all nodes. However, it is possible to use different public + address configurations on different nodes. + </para> + + <para> + Example: 4 nodes partitioned into two subgroups: + </para> + <screen format="linespecific"> +Node 0:/etc/ctdb/public_addresses + 10.1.1.1/24 eth1 + 10.1.1.2/24 eth1 + +Node 1:/etc/ctdb/public_addresses + 10.1.1.1/24 eth1 + 10.1.1.2/24 eth1 + +Node 2:/etc/ctdb/public_addresses + 10.1.2.1/24 eth2 + 10.1.2.2/24 eth2 + +Node 3:/etc/ctdb/public_addresses + 10.1.2.1/24 eth2 + 10.1.2.2/24 eth2 + </screen> + <para> + In this example nodes 0 and 1 host two public addresses on the + 10.1.1.x network while nodes 2 and 3 host two public addresses + for the 10.1.2.x network. + </para> + <para> + Public address 10.1.1.1 can be hosted by either of nodes 0 or + 1 and will be available to clients as long as at least one of + these two nodes are available. + </para> + <para> + If both nodes 0 and 1 become unavailable then public address + 10.1.1.1 also becomes unavailable. 10.1.1.1 can not be failed + over to nodes 2 or 3 since these nodes do not have this public + address configured. + </para> + <para> + The <command>ctdb ip</command> command can be used to view the + current assignment of public addresses to physical nodes. + </para> + </refsect2> + </refsect1> + + + <refsect1> + <title>Node status</title> + + <para> + The current status of each node in the cluster can be viewed by the + <command>ctdb status</command> command. + </para> + + <para> + A node can be in one of the following states: + </para> + + <variablelist> + <varlistentry> + <term>OK</term> + <listitem> + <para> + This node is healthy and fully functional. It hosts public + addresses to provide services. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>DISCONNECTED</term> + <listitem> + <para> + This node is not reachable by other nodes via the private + network. It is not currently participating in the cluster. + It <emphasis>does not</emphasis> host public addresses to + provide services. It might be shut down. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>DISABLED</term> + <listitem> + <para> + This node has been administratively disabled. This node is + partially functional and participates in the cluster. + However, it <emphasis>does not</emphasis> host public + addresses to provide services. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>UNHEALTHY</term> + <listitem> + <para> + A service provided by this node has failed a health check + and should be investigated. This node is partially + functional and participates in the cluster. However, it + <emphasis>does not</emphasis> host public addresses to + provide services. Unhealthy nodes should be investigated + and may require an administrative action to rectify. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>BANNED</term> + <listitem> + <para> + CTDB is not behaving as designed on this node. For example, + it may have failed too many recovery attempts. Such nodes + are banned from participating in the cluster for a + configurable time period before they attempt to rejoin the + cluster. A banned node <emphasis>does not</emphasis> host + public addresses to provide services. All banned nodes + should be investigated and may require an administrative + action to rectify. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>STOPPED</term> + <listitem> + <para> + This node has been administratively exclude from the + cluster. A stopped node does no participate in the cluster + and <emphasis>does not</emphasis> host public addresses to + provide services. This state can be used while performing + maintenance on a node. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>PARTIALLYONLINE</term> + <listitem> + <para> + A node that is partially online participates in a cluster + like a healthy (OK) node. Some interfaces to serve public + addresses are down, but at least one interface is up. See + also <command>ctdb ifaces</command>. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title>CAPABILITIES</title> + + <para> + Cluster nodes can have several different capabilities enabled. + These are listed below. + </para> + + <variablelist> + + <varlistentry> + <term>RECMASTER</term> + <listitem> + <para> + Indicates that a node can become the CTDB cluster recovery + master. The current recovery master is decided via an + election held by all active nodes with this capability. + </para> + <para> + Default is YES. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>LMASTER</term> + <listitem> + <para> + Indicates that a node can be the location master (LMASTER) + for database records. The LMASTER always knows which node + has the latest copy of a record in a volatile database. + </para> + <para> + Default is YES. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>LVS</term> + <listitem> + <para> + Indicates that a node is configued in Linux Virtual Server + (LVS) mode. In this mode the entire CTDB cluster uses one + single public address for the entire cluster instead of + using multiple public addresses in failover mode. This is + an alternative to using a load-balancing layer-4 switch. + See the <citetitle>LVS</citetitle> section for more + details. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>NATGW</term> + <listitem> + <para> + Indicates that this node is configured to become the NAT + gateway master in a NAT gateway group. See the + <citetitle>NAT GATEWAY</citetitle> section for more + details. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para> + The RECMASTER and LMASTER capabilities can be disabled when CTDB + is used to create a cluster spanning across WAN links. In this + case CTDB acts as a WAN accelerator. + </para> + + </refsect1> + + <refsect1> + <title>LVS</title> + + <para> + LVS is a mode where CTDB presents one single IP address for the + entire cluster. This is an alternative to using public IP + addresses and round-robin DNS to loadbalance clients across the + cluster. + </para> + + <para> + This is similar to using a layer-4 loadbalancing switch but with + some restrictions. + </para> + + <para> + In this mode the cluster selects a set of nodes in the cluster + and loadbalance all client access to the LVS address across this + set of nodes. This set of nodes are all LVS capable nodes that + are HEALTHY, or if no HEALTHY nodes exists all LVS capable nodes + regardless of health status. LVS will however never loadbalance + traffic to nodes that are BANNED, STOPPED, DISABLED or + DISCONNECTED. The <command>ctdb lvs</command> command is used to + show which nodes are currently load-balanced across. + </para> + + <para> + One of the these nodes are elected as the LVSMASTER. This node + receives all traffic from clients coming in to the LVS address + and multiplexes it across the internal network to one of the + nodes that LVS is using. When responding to the client, that + node will send the data back directly to the client, bypassing + the LVSMASTER node. The command <command>ctdb + lvsmaster</command> will show which node is the current + LVSMASTER. + </para> + + <para> + The path used for a client I/O is: + <orderedlist> + <listitem> + <para> + Client sends request packet to LVSMASTER. + </para> + </listitem> + <listitem> + <para> + LVSMASTER passes the request on to one node across the + internal network. + </para> + </listitem> + <listitem> + <para> + Selected node processes the request. + </para> + </listitem> + <listitem> + <para> + Node responds back to client. + </para> + </listitem> + </orderedlist> + </para> + + <para> + This means that all incoming traffic to the cluster will pass + through one physical node, which limits scalability. You can + send more data to the LVS address that one physical node can + multiplex. This means that you should not use LVS if your I/O + pattern is write-intensive since you will be limited in the + available network bandwidth that node can handle. LVS does work + wery well for read-intensive workloads where only smallish READ + requests are going through the LVSMASTER bottleneck and the + majority of the traffic volume (the data in the read replies) + goes straight from the processing node back to the clients. For + read-intensive i/o patterns you can acheive very high throughput + rates in this mode. + </para> + + <para> + Note: you can use LVS and public addresses at the same time. + </para> + + <para> + If you use LVS, you must have a permanent address configured for + the public interface on each node. This address must be routable + and the cluster nodes must be configured so that all traffic + back to client hosts are routed through this interface. This is + also required in order to allow samba/winbind on the node to + talk to the domain controller. This LVS IP address can not be + used to initiate outgoing traffic. + </para> + <para> + Make sure that the domain controller and the clients are + reachable from a node <emphasis>before</emphasis> you enable + LVS. Also ensure that outgoing traffic to these hosts is routed + out through the configured public interface. + </para> + + <refsect2> + <title>Configuration</title> + + <para> + To activate LVS on a CTDB node you must specify the + <varname>CTDB_PUBLIC_INTERFACE</varname> and + <varname>CTDB_LVS_PUBLIC_IP</varname> configuration variables. + Setting the latter variable also enables the LVS capability on + the node at startup. + </para> + + <para> + Example: + <screen format="linespecific"> +CTDB_PUBLIC_INTERFACE=eth1 +CTDB_LVS_PUBLIC_IP=10.1.1.237 + </screen> + </para> + + </refsect2> + </refsect1> + + <refsect1> + <title>NAT GATEWAY</title> + + <para> + NAT gateway (NATGW) is an optional feature that is used to + configure fallback routing for nodes. This allows cluster nodes + to connect to external services (e.g. DNS, AD, NIS and LDAP) + when they do not host any public addresses (e.g. when they are + unhealthy). + </para> + <para> + This also applies to node startup because CTDB marks nodes as + UNHEALTHY until they have passed a "monitor" event. In this + context, NAT gateway helps to avoid a "chicken and egg" + situation where a node needs to access an external service to + become healthy. + </para> + <para> + Another way of solving this type of problem is to assign an + extra static IP address to a public interface on every node. + This is simpler but it uses an extra IP address per node, while + NAT gateway generally uses only one extra IP address. + </para> + + <refsect2> + <title>Operation</title> + + <para> + One extra NATGW public address is assigned on the public + network to each NATGW group. Each NATGW group is a set of + nodes in the cluster that shares the same NATGW address to + talk to the outside world. Normally there would only be one + NATGW group spanning an entire cluster, but in situations + where one CTDB cluster spans multiple physical sites it might + be useful to have one NATGW group for each site. + </para> + <para> + There can be multiple NATGW groups in a cluster but each node + can only be member of one NATGW group. + </para> + <para> + In each NATGW group, one of the nodes is selected by CTDB to + be the NATGW master and the other nodes are consider to be + NATGW slaves. NATGW slaves establish a fallback default route + to the NATGW master via the private network. When a NATGW + slave hosts no public IP addresses then it will use this route + for outbound connections. The NATGW master hosts the NATGW + public IP address and routes outgoing connections from + slave nodes via this IP address. It also establishes a + fallback default route. + </para> + </refsect2> + + <refsect2> + <title>Configuration</title> + + <para> + NATGW is usually configured similar to the following example configuration: + </para> + <screen format="linespecific"> +CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes +CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 +CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 +CTDB_NATGW_PUBLIC_IFACE=eth0 +CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1 + </screen> + + <para> + Normally any node in a NATGW group can act as the NATGW + master. Some configurations may have special nodes that lack + connectivity to a public network. In such cases, + <varname>CTDB_NATGW_SLAVE_ONLY</varname> can be used to limit the + NATGW functionality of thos nodes. + </para> + + <para> + See the <citetitle>NAT GATEWAY</citetitle> section in + <citerefentry><refentrytitle>ctdb.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry> for more details of + NATGW configuration. + </para> + </refsect2> + + + <refsect2> + <title>Implementation details</title> + + <para> + When the NATGW functionality is used, one of the nodes is + selected to act as a NAT gateway for all the other nodes in + the group when they need to communicate with the external + services. The NATGW master is selected to be a node that is + most likely to have usable networks. + </para> + + <para> + The NATGW master hosts the NATGW public IP address + <varname>CTDB_NATGW_PUBLIC_IP</varname> on the configured public + interfaces <varname>CTDB_NATGW_PUBLIC_IFACE</varname> and acts as + a router, masquerading outgoing connections from slave nodes + via this IP address. It also establishes a fallback default + route to the configured default gateway + <varname>CTDB_NATGW_DEFAULT_GATEWAY</varname> with a metric of 10. + A metric 10 route is used so it can co-exist with other + default routes that may be available. + </para> + + <para> + A NATGW slave establishes its fallback default route to the + NATGW master via the private network + <varname>CTDB_NATGW_PRIVATE_NETWORK</varname>with a metric of 10. + This route is used for outbound connections when no other + default route is available because the node hosts no public + addresses. A metric 10 routes is used so that it can co-exist + with other default routes that may be available when the node + is hosting public addresses. + </para> + + <para> + This is implemented in the <filename>11.natgw</filename> + eventscript. Please see the eventscript file for the finer + details. + </para> + + </refsect2> + </refsect1> + + <refsect1> + <title>POLICY ROUTING</title> + + <para> + Policy routing is an optional CTDB feature to support complex + network topologies. Public addresses may be spread across + several different networks (or VLANs) and it may not be possible + to route packets from these public addresses via the system's + default route. Therefore, CTDB has support for policy routing + via the <filename>13.per_ip_routing</filename> eventscript. + This allows routing to be specified for packets sourced from + each public address. The routes are added and removed as CTDB + moves public addresses between nodes. + </para> + + <refsect2> + <title>Configuration variables</title> + + <para> + There are 4 configuration variables related to policy routing: + <varname>CTDB_PER_IP_ROUTING_CONF</varname>, + <varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname>, + <varname>CTDB_PER_IP_ROUTING_TABLE_ID_LOW</varname>, + <varname>CTDB_PER_IP_ROUTING_TABLE_ID_HIGH</varname>. See the + <citetitle>POLICY ROUTING</citetitle> section in + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry> for more details. + </para> + </refsect2> + + <refsect2> + <title>Configuration</title> + + <para> + The format of each line of + <varname>CTDB_PER_IP_ROUTING_CONF</varname> is: + </para> + + <screen> +<public_address> <network> [ <gateway> ] + </screen> + + <para> + Leading whitespace is ignored and arbitrary whitespace may be + used as a separator. Lines that have a "public address" item + that doesn't match an actual public address are ignored. This + means that comment lines can be added using a leading + character such as '#', since this will never match an IP + address. + </para> + + <para> + A line without a gateway indicates a link local route. + </para> + + <para> + For example, consider the configuration line: + </para> + + <screen> + 192.168.1.99 192.168.1.1/24 + </screen> + + <para> + If the corresponding public_addresses line is: + </para> + + <screen> + 192.168.1.99/24 eth2,eth3 + </screen> + + <para> + <varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname> is 100, and + CTDB adds the address to eth2 then the following routing + information is added: + </para> + + <screen> + ip rule add from 192.168.1.99 pref 100 table ctdb.192.168.1.99 + ip route add 192.168.1.0/24 dev eth2 table ctdb.192.168.1.99 + </screen> + + <para> + This causes traffic from 192.168.1.1 to 192.168.1.0/24 go via + eth2. + </para> + + <para> + The <command>ip rule</command> command will show (something + like - depending on other public addresses and other routes on + the system): + </para> + + <screen> + 0: from all lookup local + 100: from 192.168.1.99 lookup ctdb.192.168.1.99 + 32766: from all lookup main + 32767: from all lookup default + </screen> + + <para> + <command>ip route show table ctdb.192.168.1.99</command> will show: + </para> + + <screen> + 192.168.1.0/24 dev eth2 scope link + </screen> + + <para> + The usual use for a line containing a gateway is to add a + default route corresponding to a particular source address. + Consider this line of configuration: + </para> + + <screen> + 192.168.1.99 0.0.0.0/0 192.168.1.1 + </screen> + + <para> + In the situation described above this will cause an extra + routing command to be executed: + </para> + + <screen> + ip route add 0.0.0.0/0 via 192.168.1.1 dev eth2 table ctdb.192.168.1.99 + </screen> + + <para> + With both configuration lines, <command>ip route show table + ctdb.192.168.1.99</command> will show: + </para> + + <screen> + 192.168.1.0/24 dev eth2 scope link + default via 192.168.1.1 dev eth2 + </screen> + </refsect2> + + <refsect2> + <title>Sample configuration</title> + + <para> + Here is a more complete example configuration. + </para> + + <screen> +/etc/ctdb/public_addresses: + + 192.168.1.98 eth2,eth3 + 192.168.1.99 eth2,eth3 + +/etc/ctdb/policy_routing: + + 192.168.1.98 192.168.1.0/24 + 192.168.1.98 192.168.200.0/24 192.168.1.254 + 192.168.1.98 0.0.0.0/0 192.168.1.1 + 192.168.1.99 192.168.1.0/24 + 192.168.1.99 192.168.200.0/24 192.168.1.254 + 192.168.1.99 0.0.0.0/0 192.168.1.1 + </screen> + + <para> + The routes local packets as expected, the default route is as + previously discussed, but packets to 192.168.200.0/24 are + routed via the alternate gateway 192.168.1.254. + </para> + + </refsect2> + </refsect1> + + <refsect1> + <title>NOTIFICATION SCRIPT</title> + + <para> + When certain state changes occur in CTDB, it can be configured + to perform arbitrary actions via a notification script. For + example, sending SNMP traps or emails when a node becomes + unhealthy or similar. + </para> + <para> + This is activated by setting the + <varname>CTDB_NOTIFY_SCRIPT</varname> configuration variable. + The specified script must be executable. + </para> + <para> + Use of the provided <filename>/etc/ctdb/notify.sh</filename> + script is recommended. It executes files in + <filename>/etc/ctdb/notify.d/</filename>. + </para> + <para> + CTDB currently generates notifications after CTDB changes to + these states: + </para> + + <simplelist> + <member>init</member> + <member>setup</member> + <member>startup</member> + <member>healthy</member> + <member>unhealthy</member> + </simplelist> + + </refsect1> + + <refsect1> + <title>DEBUG LEVELS</title> + + <para> + Valid values for DEBUGLEVEL are: + </para> + + <simplelist> + <member>EMERG (-3)</member> + <member>ALERT (-2)</member> + <member>CRIT (-1)</member> + <member>ERR (0)</member> + <member>WARNING (1)</member> + <member>NOTICE (2)</member> + <member>INFO (3)</member> + <member>DEBUG (4)</member> + </simplelist> + </refsect1> + + + <refsect1> + <title>REMOTE CLUSTER NODES</title> + <para> +It is possible to have a CTDB cluster that spans across a WAN link. +For example where you have a CTDB cluster in your datacentre but you also +want to have one additional CTDB node located at a remote branch site. +This is similar to how a WAN accelerator works but with the difference +that while a WAN-accelerator often acts as a Proxy or a MitM, in +the ctdb remote cluster node configuration the Samba instance at the remote site +IS the genuine server, not a proxy and not a MitM, and thus provides 100% +correct CIFS semantics to clients. + </para> + + <para> + See the cluster as one single multihomed samba server where one of + the NICs (the remote node) is very far away. + </para> + + <para> + NOTE: This does require that the cluster filesystem you use can cope + with WAN-link latencies. Not all cluster filesystems can handle + WAN-link latencies! Whether this will provide very good WAN-accelerator + performance or it will perform very poorly depends entirely + on how optimized your cluster filesystem is in handling high latency + for data and metadata operations. + </para> + + <para> + To activate a node as being a remote cluster node you need to set + the following two parameters in /etc/sysconfig/ctdb for the remote node: + <screen format="linespecific"> +CTDB_CAPABILITY_LMASTER=no +CTDB_CAPABILITY_RECMASTER=no + </screen> + </para> + + <para> + Verify with the command "ctdb getcapabilities" that that node no longer + has the recmaster or the lmaster capabilities. + </para> + + </refsect1> + + + <refsect1> + <title>SEE ALSO</title> + + <para> + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ltdbtool</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>onnode</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ping_pong</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb-tunables</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="http://ctdb.samba.org/"/> + </para> + </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Ronnie Sahlberg, + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + +</refentry> diff --git a/ctdb/doc/ctdbd.1.xml b/ctdb/doc/ctdbd.1.xml index 75974cf708d..402f083cc41 100644 --- a/ctdb/doc/ctdbd.1.xml +++ b/ctdb/doc/ctdbd.1.xml @@ -1,1804 +1,623 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry - PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" - "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> -<refentry id="ctdbd.1"> - -<refmeta> - <refentrytitle>ctdbd</refentrytitle> - <manvolnum>1</manvolnum> - <refmiscinfo class="source">ctdb</refmiscinfo> - <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> -</refmeta> - + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> -<refnamediv> - <refname>ctdbd</refname> - <refpurpose>The CTDB cluster daemon</refpurpose> -</refnamediv> +<refentry id="ctdbd.1"> -<refsynopsisdiv> - <cmdsynopsis> - <command>ctdbd</command> - </cmdsynopsis> - - <cmdsynopsis> - <command>ctdbd</command> - <arg choice="opt">-? --help</arg> - <arg choice="opt">-d --debug=<INTEGER></arg> - <arg choice="req">--dbdir=<directory></arg> - <arg choice="req">--dbdir-persistent=<directory></arg> - <arg choice="opt">--event-script-dir=<directory></arg> - <arg choice="opt">-i --interactive</arg> - <arg choice="opt">--listen=<address></arg> - <arg choice="opt">--logfile=<filename></arg> - <arg choice="opt">--lvs</arg> - <arg choice="req">--nlist=<filename></arg> - <arg choice="opt">--no-lmaster</arg> - <arg choice="opt">--no-recmaster</arg> - <arg choice="opt">--nosetsched</arg> - <arg choice="req">--notification-script=<filename></arg> - <arg choice="opt">--public-addresses=<filename></arg> - <arg choice="opt">--public-interface=<interface></arg> - <arg choice="req">--reclock=<filename></arg> - <arg choice="opt">--single-public-ip=<address></arg> - <arg choice="opt">--socket=<filename></arg> - <arg choice="opt">--start-as-disabled</arg> - <arg choice="opt">--start-as-stopped</arg> - <arg choice="opt">--syslog</arg> - <arg choice="opt">--log-ringbuf-size=<num-entries></arg> - <arg choice="opt">--torture</arg> - <arg choice="opt">--transport=<STRING></arg> - <arg choice="opt">--usage</arg> - </cmdsynopsis> - -</refsynopsisdiv> + <refmeta> + <refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ctdbd</refname> + <refpurpose>The CTDB cluster daemon</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>ctdbd</command> + <arg rep="repeat"><replaceable>OPTION</replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> - <refsect1><title>DESCRIPTION</title> - <para> - ctdbd is the main ctdb daemon. - </para> - <para> - ctdbd provides a clustered version of the TDB database with automatic rebuild/recovery of the databases upon nodefailures. - </para> + <refsect1> + <title>DESCRIPTION</title> <para> - Combined with a cluster filesystem ctdbd provides a full HA environment for services such as clustered Samba and NFS as well as other services. + ctdbd is the main CTDB daemon. </para> + <para> - ctdbd provides monitoring of all nodes in the cluster and automatically reconfigures the cluster and recovers upon node failures. + Note that ctdbd is not usually invoked directly. It is invoked + via <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> or via the initscript. </para> + <para> - ctdbd is the main component in clustered Samba that provides a high-availability load-sharing CIFS server cluster. + See <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for an overview of CTDB. </para> </refsect1> - <refsect1> - <title>OPTIONS</title> + <title>GENERAL OPTIONS</title> <variablelist> - <varlistentry><term>-? --help</term> - <listitem> - <para> - Print some help text to the screen. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>-d --debug=<DEBUGLEVEL></term> - <listitem> - <para> - This option sets the debuglevel on the ctdbd daemon which controls what will be written to the logfile. The default is 0 which will only log important events and errors. A larger number will provide additional logging. - </para> - </listitem> - </varlistentry> - - <varlistentry><term>--dbdir=<directory></term> - <listitem> - <para> - This is the directory on local storage where ctdbd keeps the local - copy of the TDB databases. This directory is local for each node and should not be stored on the shared cluster filesystem. - </para> - <para> - This directory would usually be /var/ctdb . - </para> - </listitem> + <varlistentry> + <term>-d, --debug=<parameter>DEBUGLEVEL</parameter></term> + <listitem> + <para> + This option sets the debug level to DEBUGLEVEL, which + controls what will be written to the logfile. The default is + 0 which will only log important events and errors. A larger + number will provide additional logging. + </para> + <para> + See the <citetitle>DEBUG LEVELS</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--dbdir-persistent=<directory></term> - <listitem> - <para> - This is the directory on local storage where ctdbd keeps the local - copy of the persistent TDB databases. This directory is local for each node and should not be stored on the shared cluster filesystem. - </para> - <para> - This directory would usually be /etc/ctdb/persistent . - </para> - </listitem> + <varlistentry> + <term>--dbdir=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY on local storage where ctdbd keeps a local copy of + TDB databases. This directory is local for each node and + should not be stored on the shared cluster filesystem. + </para> + <para> + This directory would usually be <filename>/var/ctdb</filename> + </para> + </listitem> </varlistentry> - <varlistentry><term>--event-script-dir=<directory></term> - <listitem> - <para> - This option is used to specify the directory where the CTDB event - scripts are stored. - </para> - <para> - This will normally be /etc/ctdb/events.d which is part of the ctdb distribution. - </para> - </listitem> + <varlistentry> + <term>--dbdir-persistent=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY on local storage where ctdbd keeps a local copy of + persistent TDB databases. This directory is local for each + node and should not be stored on the shared cluster + filesystem. + </para> + <para> + This directory would usually be + <filename>/etc/ctdb/persistent</filename> + </para> + </listitem> </varlistentry> - <varlistentry><term>-i --interactive</term> - <listitem> - <para> - By default ctdbd will detach itself from the shell and run in - the background as a daemon. This option makes ctdbd to start in interactive mode. - </para> - </listitem> + <varlistentry> + <term>--dbdir-state=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY on local storage where ctdbd keep internal state + TDB files. This directory is local for each node and + should not be stored on the shared cluster filesystem. + </para> + <para> + This directory would usually be + <filename>/var/ctdb/state</filename> + </para> + </listitem> </varlistentry> - <varlistentry><term>--listen=<address></term> - <listitem> - <para> - This specifies which ip address ctdb will bind to. By default ctdbd will bind to the first address it finds in the /etc/ctdb/nodes file and which is also present on the local system in which case you do not need to provide this option. - </para> - <para> - This option is only required when you want to run multiple ctdbd daemons/nodes on the same physical host in which case there would be multiple entries in /etc/ctdb/nodes what would match a local interface. - </para> - </listitem> + <varlistentry> + <term>--event-script-dir=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY where the CTDB event scripts are stored. See the + <citetitle>EVENT SCRIPTS</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more information. + </para> + <para> + Default is <envar>CTDB_BASE</envar>/events.d, so usually + <filename>/etc/ctdb/events.d</filename>, which is part of + the CTDB installation. + </para> + </listitem> </varlistentry> - <varlistentry><term>--logfile=<filename></term> - <listitem> - <para> - This is the file where ctdbd will write its log. This is usually /var/log/log.ctdb . - </para> - </listitem> + <varlistentry> + <term>--logfile=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME where ctdbd will write its log. This is usually + <filename>/var/log/log.ctdb</filename>. + </para> + </listitem> </varlistentry> - <varlistentry><term>--lvs</term> - <listitem> - <para> - This option is used to activate the LVS capability on a CTDB node. - Please see the LVS section. - </para> - </listitem> + <varlistentry> + <term>--log-ringbuf-size=<parameter>NUM</parameter></term> + <listitem> + <para> + Set the size of the log ringbuffer to NUM entries. + </para> + <para> + CTDB uses an in-memory ringbuffer containing NUM most + recent log entries for all log levels (except DEBUG). The + ringbugger can be useful for extracting detailed logs even + if some entries are not logged to the regular logs. + </para> + <para> + Use the <command>ctdb getlog</command> command to retrieve + log entries from the ringbuffer. + </para> + </listitem> </varlistentry> - <varlistentry><term>--nlist=<filename></term> - <listitem> - <para> - This file contains a list of the private ip addresses of every node in the cluster. There is one line/ip address for each node. This file must be the same for all nodes in the cluster. - </para> - <para> - This file is usually /etc/ctdb/nodes . - </para> - </listitem> + <varlistentry> + <term>--lvs</term> + <listitem> + <para> + This option is used to activate the LVS capability on a CTDB + node. Please see the <citetitle>LVS</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--no-lmaster</term> - <listitem> - <para> - This argument specifies that this node can NOT become an lmaster - for records in the database. This means that it will never show up - in the vnnmap. This feature is primarily used for making a cluster - span across a WAN link and use CTDB as a WAN-accelerator. - </para> - <para> - Please see the "remote cluster nodes" section for more information. - </para> - </listitem> + <varlistentry> + <term>--max-persistent-check-errors=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM specifies the maximum number of health check failures + allowed for persistent databases during startup. + </para> + <para> + The default value is 0. Setting this to non-zero allows a + node with unhealthy persistent databases to startup and + join the cluster as long as there is another node with + healthy persistent databases. + </para> + </listitem> </varlistentry> - <varlistentry><term>--no-recmaster</term> - <listitem> - <para> - This argument specifies that this node can NOT become a recmaster - for the database. This feature is primarily used for making a cluster - span across a WAN link and use CTDB as a WAN-accelerator. - </para> - <para> - Please see the "remote cluster nodes" section for more information. - </para> - </listitem> + <varlistentry> + <term>--nlist=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME containing a list of the private IP addresses, one + per line, for each node in the cluster. This file + <emphasis>must be the same on each node</emphasis> in the + cluster. + </para> + <para> + Default is <envar>CTDB_BASE</envar>/nodes, so usually + <filename>/etc/ctdb/nodes</filename>. + </para> + </listitem> </varlistentry> - <varlistentry><term>--nosetsched</term> - <listitem> - <para> - This is a ctdbd debugging option. this option is only used when - debugging ctdbd. + <varlistentry> + <term>--no-lmaster</term> + <listitem> + <para> + This argument specifies that this node can NOT become an lmaster + for records in the database. This means that it will never show up + in the vnnmap. This feature is primarily used for making a cluster + span across a WAN link and use CTDB as a WAN-accelerator. </para> <para> - Normally ctdb will change its scheduler to run as a real-time - process. This is the default mode for a normal ctdbd operation - to gurarantee that ctdbd always gets the cpu cycles that it needs. - </para> - <para> - This option is used to tell ctdbd to NOT run as a real-time process - and instead run ctdbd as a normal userspace process. - This is useful for debugging and when you want to run ctdbd under - valgrind or gdb. (You don't want to attach valgrind or gdb to a - real-time process.) - </para> - </listitem> + Please see the <citetitle>REMOTE CLUSTER NODES</citetitle> + section in <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--notification-script=<filename></term> - <listitem> - <para> - This specifies a script which will be invoked by ctdb when certain - state changes occur in ctdbd and when you may want to trigger this - to run certain scripts. - </para> - <para> - This file is usually /etc/ctdb/notify.sh . - </para> - <para> - See the NOTIFICATION SCRIPT section below for more information. - </para> - </listitem> + <varlistentry> + <term>--no-recmaster</term> + <listitem> + <para> + This argument specifies that this node can NOT become a recmaster + for the database. This feature is primarily used for making a cluster + span across a WAN link and use CTDB as a WAN-accelerator. + </para> + <para> + Please see the <citetitle>REMOTE CLUSTER NODES</citetitle> + section in <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--public_addresses=<filename></term> - <listitem> - <para> - When used with IP takeover this specifies a file containing the public ip addresses to use on the cluster. This file contains a list of ip addresses netmasks and interfaces. When ctdb is operational it will distribute these public ip addresses evenly across the available nodes. - </para> - <para> - This is usually the file /etc/ctdb/public_addresses - </para> - </listitem> + <varlistentry> + <term>--notification-script=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME specifying a script to be invoked by ctdbd when + certain state changes occur. + </para> + <para> + This file is usually + <filename>/etc/ctdb/notify.sh</filename>. + </para> + <para> + Please see the <citetitle>NOTIFICATION SCRIPT</citetitle> + section in <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--public-interface=<interface></term> - <listitem> - <para> - This option tells ctdb which interface to attach public-addresses - to and also where to attach the single-public-ip when used. - </para> - <para> - This is only required when using public ip addresses and only when - you don't specify the interface explicitly in /etc/ctdb/public_addresses or when you are using --single-public-ip. - </para> - <para> - If you omit this argument when using public addresses or single public ip, ctdb will not be able to send out Gratious ARPs correctly or be able to kill tcp connections correctly which will lead to application failures. - </para> - </listitem> + <varlistentry> + <term>--pidfile=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME for file containing process ID of main CTDB + daemon. This file is automatically created and removed by + CTDB. + </para> + <para> + The default is to not create a PID file. + </para> + </listitem> </varlistentry> - <varlistentry><term>--reclock=<filename></term> - <listitem> - <para> - This is the name of the lock file stored of the shared cluster filesystem that ctdbd uses to prevent split brains from occuring. - This file must be stored on shared storage. - </para> - <para> - It is possible to run CTDB without a reclock file, but then there - will be no protection against split brain if the network becomes - partitioned. Using CTDB without a reclock file is strongly - discouraged. - </para> - </listitem> + <varlistentry> + <term>--public_addresses=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME specifying a file containing the public IP + addresses to use on the cluster when CTDB should use IP + takeover. This file contains a list of IP addresses, + netmasks and interfaces. CTDB will distribute these public + IP addresses appropriately across the available nodes. + </para> + <para> + The IP addresses specified in this file can differ across + nodes. + </para> + <para> + This is usually the file + <filename>/etc/ctdb/public_addresses</filename> + </para> + </listitem> </varlistentry> - <varlistentry><term>--socket=<filename></term> - <listitem> - <para> - This specifies the name of the domain socket that ctdbd will create. This socket is used for local clients to attach to and communicate with the ctdbd daemon. - </para> - <para> - The default is /tmp/ctdb.socket . You only need to use this option if you plan to run multiple ctdbd daemons on the same physical host. - </para> - </listitem> + <varlistentry> + <term>--public-interface=<parameter>INTERFACE</parameter></term> + <listitem> + <para> + INTERFACE on which to attach public IP addresses or on which + to attach the single-public-ip when used. + </para> + <para> + When using public IP addresses, this is only required if + interfaces are not explicitly specified in the public + addresses file. + </para> + </listitem> </varlistentry> - <varlistentry><term>--start-as-disabled</term> - <listitem> - <para> - This makes the ctdb daemon to be DISABLED when it starts up. - </para> - <para> - As it is DISABLED it will not get any of the public ip addresses - allocated to it, and thus this allow you to start ctdb on a node - without causing any ip address to failover from other nodes onto - the new node. - </para> - <para> - When used, the administrator must keep track of when nodes start and - manually enable them again using the "ctdb enable" command, or else - the node will not host any services. - </para> - <para> - A node that is DISABLED will not host any services and will not be - reachable/used by any clients. - </para> - </listitem> + <varlistentry> + <term>--reclock=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is the name of the recovery lock file stored in + <emphasis>shared storage</emphasis> that ctdbd uses to + prevent split brains from occuring. + </para> + <para> + It is possible to run CTDB without a recovery lock file, but + then there will be no protection against split brain if the + cluster/network becomes partitioned. Using CTDB without a + reclock file is strongly discouraged. + </para> + </listitem> </varlistentry> - <varlistentry><term>--start-as-stopped</term> - <listitem> - <para> - This makes the ctdb daemon to be STOPPED when it starts up. - </para> - <para> - A node that is STOPPED does not host any public addresses. It is not part of the VNNMAP so it does act as an LMASTER. It also has all databases locked in recovery mode until restarted. - </para> - <para> - To restart and activate a STOPPED node, the command "ctdb continue" is used. - </para> - <para> - A node that is STOPPED will not host any services and will not be - reachable/used by any clients. - </para> - </listitem> + <varlistentry> + <term>--single-public-ip=<parameter>IPADDR</parameter></term> + <listitem> + <para> + IPADDR specifies the single IP that CTDB will use in + conjuction with LVS. + </para> + <para> + Please see the <citetitle>LVS</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> </varlistentry> - <varlistentry><term>--syslog</term> - <listitem> - <para> - Send all log messages to syslog instead of to the ctdb logfile. - </para> - </listitem> + <varlistentry> + <term>--start-as-disabled</term> + <listitem> + <para> + This makes ctdbd start in the DISABLED state. + </para> + <para> + To allow the node to host public IP addresses and + services, it must be manually enabled using the + <command>ctdb enable</command> command. + </para> + <para> + Please see the <citetitle>NODE STATES</citetitle> section + in <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information about the DISABLED state. + </para> + </listitem> </varlistentry> - <varlistentry><term>--log-ringbuf-size=<num-entries></term> - <listitem> - <para> - In addition to the normal loggign to a log file, - CTDBD also keeps a in-memory ringbuffer containing the most recent - log entries for all log levels (except DEBUG). - </para> - <para> - This is useful since it allows for keeping continuous logs to a file - at a reasonable non-verbose level, but shortly after an incident has - occured, a much more detailed log can be pulled from memory. This - can allow you to avoid having to reproduce an issue due to the - on-disk logs being of insufficient detail. - </para> - <para> - This in-memory ringbuffer contains a fixed number of the most recent - entries. This is settable at startup either through the - --log-ringbuf-size argument, or preferably by using - CTDB_LOG_RINGBUF_SIZE in the sysconfig file. - </para> - <para> - Use the "ctdb getlog" command to access this log. - </para> - </listitem> + <varlistentry> + <term>--start-as-stopped</term> + <listitem> + <para> + This makes ctdbd start in the STOPPED state. + </para> + <para> + To allow the node to take part in the cluster it must be + manually continued with the the <command>ctdb + enable</command> command. + </para> + <para> + Please see the <citetitle>NODE STATES</citetitle> section + in <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information about the STOPPED state. + </para> + </listitem> </varlistentry> - - <varlistentry><term>--torture</term> - <listitem> - <para> - This option is only used for development and testing of ctdbd. It adds artificial errors and failures to the common codepaths in ctdbd to verify that ctdbd can recover correctly for failures. - </para> - <para> - You do NOT want to use this option unless you are developing and testing new functionality in ctdbd. - </para> - </listitem> + <varlistentry> + <term>--syslog</term> + <listitem> + <para> + Send log messages to syslog instead of the CTDB logfile. + This option overrides --logfile. The default is to log to + a file. + </para> + </listitem> </varlistentry> - <varlistentry><term>--transport=<STRING></term> - <listitem> - <para> - This option specifies which transport to use for ctdbd internode communications. The default is "tcp". - </para> - <para> - Currently only "tcp" is supported but "infiniband" might be - implemented in the future. - </para> - </listitem> + <varlistentry> + <term>--transport=tcp|infiniband</term> + <listitem> + <para> + This option specifies which transport to use for ctdbd + internode communications. The default is "tcp". + </para> + <para> + The "infiniband" support is not regularly tested. + </para> + </listitem> </varlistentry> - <varlistentry><term>--usage</term> - <listitem> - <para> - Print useage information to the screen. - </para> - </listitem> + <varlistentry> + <term>-?, --help</term> + <listitem> + <para> + Display a summary of options. + </para> + </listitem> </varlistentry> </variablelist> </refsect1> - - <refsect1><title>Private vs Public addresses</title> - <para> - When used for ip takeover in a HA environment, each node in a ctdb - cluster has multiple ip addresses assigned to it. One private and one or more public. - </para> - - <refsect2><title>Private address</title> - <para> - This is the physical ip address of the node which is configured in - linux and attached to a physical interface. This address uniquely - identifies a physical node in the cluster and is the ip addresses - that ctdbd will use to communicate with the ctdbd daemons on the - other nodes in the cluster. - </para> - <para> - The private addresses are configured in /etc/ctdb/nodes - (unless the --nlist option is used) and contain one line for each - node in the cluster. Each line contains the private ip address for one - node in the cluster. This file must be the same on all nodes in the - cluster. - </para> - <para> - Since the private addresses are only available to the network when the - corresponding node is up and running you should not use these addresses - for clients to connect to services provided by the cluster. Instead - client applications should only attach to the public addresses since - these are guaranteed to always be available. - </para> - <para> - When using ip takeover, it is strongly recommended that the private - addresses are configured on a private network physically separated - from the rest of the network and that this private network is dedicated - to CTDB traffic. - </para> - Example /etc/ctdb/nodes for a four node cluster: - <screen format="linespecific"> - 10.1.1.1 - 10.1.1.2 - 10.1.1.3 - 10.1.1.4 - </screen> - </refsect2> - <refsect2><title>Public address</title> - <para> - A public address on the other hand is not attached to an interface. - This address is managed by ctdbd itself and is attached/detached to - a physical node at runtime. - </para> - <para> - The ctdb cluster will assign/reassign these public addresses across the - available healthy nodes in the cluster. When one node fails, its public address - will be migrated to and taken over by a different node in the cluster - to ensure that all public addresses are always available to clients as - long as there are still nodes available capable of hosting this address. - </para> - <para> - These addresses are not physically attached to a specific node. - The 'ctdb ip' command can be used to view the current assignment of - public addresses and which physical node is currently serving it. - </para> - <para> - On each node this file contains a list of the public addresses that - this node is capable of hosting. - The list also contain the netmask and the - interface where this address should be attached for the case where you - may want to serve data out through multiple different interfaces. - </para> - Example /etc/ctdb/public_addresses for a node that can host 4 public addresses: - <screen format="linespecific"> - 11.1.1.1/24 eth0 - 11.1.1.2/24 eth0 - 11.1.2.1/24 eth1 - 11.1.2.2/24 eth1 - </screen> - - <para> - In most cases this file would be the same on all nodes in a cluster but - there are exceptions when one may want to use different files - on different nodes. - </para> - Example: 4 nodes partitioned into two subgroups : - <screen format="linespecific"> - Node 0:/etc/ctdb/public_addresses - 10.1.1.1/24 eth0 - 10.1.1.2/24 eth0 - - Node 1:/etc/ctdb/public_addresses - 10.1.1.1/24 eth0 - 10.1.1.2/24 eth0 - - Node 2:/etc/ctdb/public_addresses - 10.2.1.1/24 eth0 - 10.2.1.2/24 eth0 - - Node 3:/etc/ctdb/public_addresses - 10.2.1.1/24 eth0 - 10.2.1.2/24 eth0 - </screen> - <para> - In this example nodes 0 and 1 host two public addresses on the - 10.1.1.x network while nodes 2 and 3 host two public addresses for the - 10.2.1.x network. - </para> - <para> - Ip address 10.1.1.1 can be hosted by either of nodes 0 or 1 and will be - available to clients as long as at least one of these two nodes are - available. - If both nodes 0 and node 1 become unavailable 10.1.1.1 also becomes - unavailable. 10.1.1.1 can not be failed over to node 2 or node 3 since - these nodes do not have this ip address listed in their public - addresses file. - </para> - </refsect2> - </refsect1> - - - <refsect1><title>Node status</title> - <para> - The current status of each node in the cluster can be viewed by the - 'ctdb status' command. - </para> - <para> - There are five possible states for a node. - </para> - - <para> - OK - This node is fully functional. - </para> - - <para> - DISCONNECTED - This node could not be connected through the network - and is currently not particpating in the cluster. If there is a - public IP address associated with this node it should have been taken - over by a different node. No services are running on this node. - </para> - - <para> - DISABLED - This node has been administratively disabled. This node is - still functional and participates in the CTDB cluster but its IP - addresses have been taken over by a different node and no services are - currently being hosted. - </para> - - <para> - UNHEALTHY - A service provided by this node is malfunctioning and should - be investigated. The CTDB daemon itself is operational and participates - in the cluster. Its public IP address has been taken over by a different - node and no services are currently being hosted. All unhealthy nodes - should be investigated and require an administrative action to rectify. - </para> - - <para> - BANNED - This node failed too many recovery attempts and has been banned - from participating in the cluster for a period of RecoveryBanPeriod - seconds. Any public IP address has been taken over by other nodes. This - node does not provide any services. All banned nodes should be - investigated and require an administrative action to rectify. This node - does not perticipate in the CTDB cluster but can still be communicated - with. I.e. ctdb commands can be sent to it. - </para> - - <para> - STOPPED - A node that is stopped does not host any public ip addresses, - nor is it part of the VNNMAP. A stopped node can not become LVSMASTER, - RECMASTER or NATGW. - This node does not perticipate in the CTDB cluster but can still be - communicated with. I.e. ctdb commands can be sent to it. - </para> - </refsect1> - - <refsect1> - <title>PUBLIC TUNABLES</title> - <para> - These are the public tuneables that can be used to control how ctdb behaves. - </para> - - <refsect2><title>MaxRedirectCount</title> - <para>Default: 3</para> - <para> - If we are not the DMASTER and need to fetch a record across the network - we first send the request to the LMASTER after which the record - is passed onto the current DMASTER. If the DMASTER changes before - the request has reached that node, the request will be passed onto the - "next" DMASTER. For very hot records that migrate rapidly across the - cluster this can cause a request to "chase" the record for many hops - before it catches up with the record. - - this is how many hops we allow trying to chase the DMASTER before we - switch back to the LMASTER again to ask for new directions. - </para> - <para> - When chasing a record, this is how many hops we will chase the record - for before going back to the LMASTER to ask for new guidance. - </para> - </refsect2> - - <refsect2><title>SeqnumInterval</title> - <para>Default: 1000</para> - <para> - Some databases have seqnum tracking enabled, so that samba will be able - to detect asynchronously when there has been updates to the database. - Everytime a database is updated its sequence number is increased. - </para> - <para> - This tunable is used to specify in 'ms' how frequently ctdb will - send out updates to remote nodes to inform them that the sequence - number is increased. - </para> - </refsect2> - - <refsect2><title>ControlTimeout</title> - <para>Default: 60</para> - <para> - This is the default - setting for timeout for when sending a control message to either the - local or a remote ctdb daemon. - </para> - </refsect2> - - <refsect2><title>TraverseTimeout</title> - <para>Default: 20</para> - <para> - This setting controls how long we allow a traverse process to run. - After this timeout triggers, the main ctdb daemon will abort the - traverse if it has not yet finished. - </para> - </refsect2> - - <refsect2><title>KeepaliveInterval</title> - <para>Default: 5</para> - <para> - How often in seconds should the nodes send keepalives to eachother. - </para> - </refsect2> - - <refsect2><title>KeepaliveLimit</title> - <para>Default: 5</para> - <para> - After how many keepalive intervals without any traffic should a node - wait until marking the peer as DISCONNECTED. - </para> - <para> - If a node has hung, it can thus take KeepaliveInterval*(KeepaliveLimit+1) - seconds before we determine that the node is DISCONNECTED and that we - require a recovery. This limitshould not be set too high since we want - a hung node to be detectec, and expunged from the cluster well before - common CIFS timeouts (45-90 seconds) kick in. - </para> - </refsect2> - - <refsect2><title>RecoverTimeout</title> - <para>Default: 20</para> - <para> - This is the default setting for timeouts for controls when sent from the - recovery daemon. We allow longer control timeouts from the recovery daemon - than from normal use since the recovery dameon often use controls that - can take a lot longer than normal controls. - </para> - </refsect2> - - <refsect2><title>RecoverInterval</title> - <para>Default: 1</para> - <para> - How frequently in seconds should the recovery daemon perform the - consistency checks that determine if we need to perform a recovery or not. - </para> - </refsect2> - - <refsect2><title>ElectionTimeout</title> - <para>Default: 3</para> - <para> - When electing a new recovery master, this is how many seconds we allow - the election to take before we either deem the election finished - or we fail the election and start a new one. - </para> - </refsect2> - - <refsect2><title>TakeoverTimeout</title> - <para>Default: 9</para> - <para> - This is how many seconds we allow controls to take for IP failover events. - </para> - </refsect2> - - <refsect2><title>MonitorInterval</title> - <para>Default: 15</para> - <para> - How often should ctdb run the event scripts to check for a nodes health. - </para> - </refsect2> - - <refsect2><title>TickleUpdateInterval</title> - <para>Default: 20</para> - <para> - How often will ctdb record and store the "tickle" information used to - kickstart stalled tcp connections after a recovery. - </para> - </refsect2> - - <refsect2><title>EventScriptTimeout</title> - <para>Default: 20</para> - <para> - How long should ctdb let an event script run before aborting it and - marking the node unhealthy. - </para> - </refsect2> - - <refsect2><title>EventScriptTimeoutCount</title> - <para>Default: 1</para> - <para> - How many events in a row needs to timeout before we flag the node UNHEALTHY. - This setting is useful if your scripts can not be written so that they - do not hang for benign reasons. - </para> - </refsect2> - - <refsect2><title>EventScriptUnhealthyOnTimeout</title> - <para>Default: 0</para> - <para> - This setting can be be used to make ctdb never become UNHEALTHY if your - eventscripts keep hanging/timing out. - </para> - </refsect2> - - <refsect2><title>RecoveryGracePeriod</title> - <para>Default: 120</para> - <para> - During recoveries, if a node has not caused recovery failures during the - last grace period, any records of transgressions that the node has caused - recovery failures will be forgiven. This resets the ban-counter back to - zero for that node. - </para> - </refsect2> - - <refsect2><title>RecoveryBanPeriod</title> - <para>Default: 300</para> - <para> - If a node becomes banned causing repetitive recovery failures. The node will - eventually become banned from the cluster. - This controls how long the culprit node will be banned from the cluster - before it is allowed to try to join the cluster again. - Don't set to small. A node gets banned for a reason and it is usually due - to real problems with the node. - </para> - </refsect2> - - <refsect2><title>DatabaseHashSize</title> - <para>Default: 100001</para> - <para> - Size of the hash chains for the local store of the tdbs that ctdb manages. - </para> - </refsect2> - - <refsect2><title>DatabaseMaxDead</title> - <para>Default: 5</para> - <para> - How many dead records per hashchain in the TDB database do we allow before - the freelist needs to be processed. - </para> - </refsect2> - - <refsect2><title>RerecoveryTimeout</title> - <para>Default: 10</para> - <para> - Once a recovery has completed, no additional recoveries are permitted - until this timeout has expired. - </para> - </refsect2> - - <refsect2><title>EnableBans</title> - <para>Default: 1</para> - <para> - When set to 0, this disables BANNING completely in the cluster and thus - nodes can not get banned, even it they break. Don't set to 0 unless you - know what you are doing. You should set this to the same value on - all nodes to avoid unexpected behaviour. - </para> - </refsect2> - - <refsect2><title>DeterministicIPs</title> - <para>Default: 0</para> - <para> - When enabled, this tunable makes ctdb try to keep public IP addresses - locked to specific nodes as far as possible. This makes it easier for - debugging since you can know that as long as all nodes are healthy - public IP X will always be hosted by node Y. - </para> - <para> - The cost of using deterministic IP address assignment is that it - disables part of the logic where ctdb tries to reduce the number of - public IP assignment changes in the cluster. This tunable may increase - the number of IP failover/failbacks that are performed on the cluster - by a small margin. - </para> - - </refsect2> - <refsect2><title>LCP2PublicIPs</title> - <para>Default: 1</para> - <para> - When enabled this switches ctdb to use the LCP2 ip allocation - algorithm. - </para> - </refsect2> - - <refsect2><title>ReclockPingPeriod</title> - <para>Default: x</para> - <para> - Obsolete - </para> - </refsect2> - - <refsect2><title>NoIPFailback</title> - <para>Default: 0</para> - <para> - When set to 1, ctdb will not perform failback of IP addresses when a node - becomes healthy. Ctdb WILL perform failover of public IP addresses when a - node becomes UNHEALTHY, but when the node becomes HEALTHY again, ctdb - will not fail the addresses back. - </para> - <para> - Use with caution! Normally when a node becomes available to the cluster - ctdb will try to reassign public IP addresses onto the new node as a way - to distribute the workload evenly across the clusternode. Ctdb tries to - make sure that all running nodes have approximately the same number of - public addresses it hosts. - </para> - <para> - When you enable this tunable, CTDB will no longer attempt to rebalance - the cluster by failing IP addresses back to the new nodes. An unbalanced - cluster will therefore remain unbalanced until there is manual - intervention from the administrator. When this parameter is set, you can - manually fail public IP addresses over to the new node(s) using the - 'ctdb moveip' command. - </para> - </refsect2> - - <refsect2><title>DisableIPFailover</title> - <para>Default: 0</para> - <para> - When enabled, ctdb will not perform failover or failback. Even if a - node fails while holding public IPs, ctdb will not recover the IPs or - assign them to another node. - </para> - <para> - When you enable this tunable, CTDB will no longer attempt to recover - the cluster by failing IP addresses over to other nodes. This leads to - a service outage until the administrator has manually performed failover - to replacement nodes using the 'ctdb moveip' command. - </para> - </refsect2> - - <refsect2><title>NoIPTakeover</title> - <para>Default: 0</para> - <para> - When set to 1, ctdb will not allow IP addresses to be failed over - onto this node. Any IP addresses that the node currently hosts - will remain on the node but no new IP addresses can be failed over - to the node. - </para> - </refsect2> - - <refsect2><title>NoIPHostOnAllDisabled</title> - <para>Default: 0</para> - <para> - If no nodes are healthy then by default ctdb will happily host - public IPs on disabled (unhealthy or administratively disabled) - nodes. This can cause problems, for example if the underlying - cluster filesystem is not mounted. When set to 1 on a node and - that node is disabled it, any IPs hosted by this node will be - released and the node will not takeover any IPs until it is no - longer disabled. - </para> - </refsect2> - - <refsect2><title>DBRecordCountWarn</title> - <para>Default: 100000</para> - <para> - When set to non-zero, ctdb will log a warning when we try to recover a - database with more than this many records. This will produce a warning - if a database grows uncontrollably with orphaned records. - </para> - </refsect2> - - <refsect2><title>DBRecordSizeWarn</title> - <para>Default: 10000000</para> - <para> - When set to non-zero, ctdb will log a warning when we try to recover a - database where a single record is bigger than this. This will produce - a warning if a database record grows uncontrollably with orphaned - sub-records. - </para> - </refsect2> - - <refsect2><title>DBSizeWarn</title> - <para>Default: 1000000000</para> - <para> - When set to non-zero, ctdb will log a warning when we try to recover a - database bigger than this. This will produce - a warning if a database grows uncontrollably. - </para> - </refsect2> - - <refsect2><title>VerboseMemoryNames</title> - <para>Default: 0</para> - <para> - This feature consumes additional memory. when used the talloc library - will create more verbose names for all talloc allocated objects. - </para> - </refsect2> - - <refsect2><title>RecdPingTimeout</title> - <para>Default: 60</para> - <para> - If the main dameon has not heard a "ping" from the recovery dameon for - this many seconds, the main dameon will log a message that the recovery - daemon is potentially hung. - </para> - </refsect2> - - <refsect2><title>RecdFailCount</title> - <para>Default: 10</para> - <para> - If the recovery daemon has failed to ping the main dameon for this many - consecutive intervals, the main daemon will consider the recovery daemon - as hung and will try to restart it to recover. - </para> - </refsect2> - - <refsect2><title>LogLatencyMs</title> - <para>Default: 0</para> - <para> - When set to non-zero, this will make the main daemon log any operation that - took longer than this value, in 'ms', to complete. - These include "how long time a lockwait child process needed", - "how long time to write to a persistent database" but also - "how long did it take to get a response to a CALL from a remote node". - </para> - </refsect2> - - <refsect2><title>RecLockLatencyMs</title> - <para>Default: 1000</para> - <para> - When using a reclock file for split brain prevention, if set to non-zero - this tunable will make the recovery dameon log a message if the fcntl() - call to lock/testlock the recovery file takes longer than this number of - ms. - </para> - </refsect2> - - <refsect2><title>RecoveryDropAllIPs</title> - <para>Default: 120</para> - <para> - If we have been stuck in recovery, or stopped, or banned, mode for - this many seconds we will force drop all held public addresses. - </para> - </refsect2> - - <refsect2><title>verifyRecoveryLock</title> - <para>Default: 1</para> - <para> - Should we take a fcntl() lock on the reclock file to verify that we are the - sole recovery master node on the cluster or not. - </para> - </refsect2> - - <refsect2><title>DeferredAttachTO</title> - <para>Default: 120</para> - <para> - When databases are frozen we do not allow clients to attach to the - databases. Instead of returning an error immediately to the application - the attach request from the client is deferred until the database - becomes available again at which stage we respond to the client. - </para> - <para> - This timeout controls how long we will defer the request from the client - before timing it out and returning an error to the client. - </para> - </refsect2> - - <refsect2><title>HopcountMakeSticky</title> - <para>Default: 50</para> - <para> - If the database is set to 'STICKY' mode, using the 'ctdb setdbsticky' - command, any record that is seen as very hot and migrating so fast that - hopcount surpasses 50 is set to become a STICKY record for StickyDuration - seconds. This means that after each migration the record will be kept on - the node and prevented from being migrated off the node. - </para> - <para> - This setting allows one to try to identify such records and stop them from - migrating across the cluster so fast. This will improve performance for - certain workloads, such as locking.tdb if many clients are opening/closing - the same file concurrently. - </para> - </refsect2> - - <refsect2><title>StickyDuration</title> - <para>Default: 600</para> - <para> - Once a record has been found to be fetch-lock hot and has been flagged to - become STICKY, this is for how long, in seconds, the record will be - flagged as a STICKY record. - </para> - </refsect2> - - <refsect2><title>StickyPindown</title> - <para>Default: 200</para> - <para> - Once a STICKY record has been migrated onto a node, it will be pinned down - on that node for this number of ms. Any request from other nodes to migrate - the record off the node will be deferred until the pindown timer expires. - </para> - </refsect2> - - <refsect2><title>MaxLACount</title> - <para>Default: 20</para> - <para> - When record content is fetched from a remote node, if it is only for - reading the record, pass back the content of the record but do not yet - migrate the record. Once MaxLACount identical requests from the - same remote node have been seen will the record be forcefully migrated - onto the requesting node. This reduces the amount of migration for a - database read-mostly workload at the expense of more frequent network - roundtrips. - </para> - </refsect2> - - <refsect2><title>StatHistoryInterval</title> - <para>Default: 1</para> - <para> - Granularity of the statistics collected in the statistics history. - </para> - </refsect2> - - <refsect2><title>AllowClientDBAttach</title> - <para>Default: 1</para> - <para> - When set to 0, clients are not allowed to attach to any databases. - This can be used to temporarily block any new processes from attaching - to and accessing the databases. - </para> - </refsect2> - - <refsect2><title>RecoverPDBBySeqNum</title> - <para>Default: 1</para> - <para> - When set to zero, database recovery for persistent databases is - record-by-record and recovery process simply collects the most recent - version of every individual record. - </para> - <para> - When set to non-zero, persistent databases will instead be recovered as - a whole db and not by individual records. The node that contains the - highest value stored in the record "__db_sequence_number__" is selected - and the copy of that nodes database is used as the recovered database. - </para> - <para> - By default, recovery of persistent databses is done using - __db_sequence_number__ record. - </para> - - </refsect2> - - <refsect2><title>FetchCollapse</title> - <para>Default: 1</para> - <para> - When many clients across many nodes try to access the same record at the - same time this can lead to a fetch storm where the record becomes very - active and bounces between nodes very fast. This leads to high CPU - utilization of the ctdbd daemon, trying to bounce that record around - very fast, and poor performance. - </para> - <para> - This parameter is used to activate a fetch-collapse. A fetch-collapse - is when we track which records we have requests in flight so that we only - keep one request in flight from a certain node, even if multiple smbd - processes are attemtping to fetch the record at the same time. This - can improve performance and reduce CPU utilization for certain - workloads. - </para> - <para> - This timeout controls if we should collapse multiple fetch operations - of the same record into a single request and defer all duplicates or not. - </para> - </refsect2> - - <refsect2><title>Samba3AvoidDeadlocks</title> - <para>Default: 0</para> - <para> - Enable code that prevents deadlocks with Samba (only for Samba 3.x). - </para> - <para> - This should be set to 1 when using Samba version 3.x to enable special - code in CTDB to avoid deadlock with Samba version 3.x. This code - is not required for Samba version 4.x and must not be enabled for - Samba 4.x. - </para> - </refsect2> - </refsect1> - - <refsect1><title>LVS</title> - <para> - LVS is a mode where CTDB presents one single IP address for the entire - cluster. This is an alternative to using public IP addresses and round-robin - DNS to loadbalance clients across the cluster. - </para> - - <para> - This is similar to using a layer-4 loadbalancing switch but with some restrictions. - </para> - - <para> - In this mode the cluster select a set of nodes in the cluster and loadbalance - all client access to the LVS address across this set of nodes. This set of nodes are all LVS capable nodes that are HEALTHY, or if no HEALTHY nodes exists - all LVS capable nodes regardless of health status. - LVS will however never loadbalance traffic to nodes that are BANNED, - STOPPED, DISABLED or DISCONNECTED. The "ctdb lvs" command is used to show - which nodes are currently load-balanced across. - </para> - - <para> - One of the these nodes are elected as the LVSMASTER. This node receives all - traffic from clients coming in to the LVS address and multiplexes it - across the internal network to one of the nodes that LVS is using. - When responding to the client, that node will send the data back - directly to the client, bypassing the LVSMASTER node. - The command "ctdb lvsmaster" will show which node is the current - LVSMASTER. - </para> - - <para> - The path used for a client i/o is thus : - <screen format="linespecific"> - (1) Client sends request packet to LVSMASTER - (2) LVSMASTER passes the request on to one node across the internal network. - (3) Selected node processes the request. - (4) Node responds back to client. - </screen> - </para> - - <para> - This means that all incoming traffic to the cluster will pass through - one physical node, which limits scalability. You can send more data to the - LVS address that one physical node can multiplex. This means that you - should not use LVS if your I/O pattern is write-intensive since you will be - limited in the available network bandwidth that node can handle. - LVS does work wery well for read-intensive workloads where only smallish - READ requests are going through the LVSMASTER bottleneck and the majority - of the traffic volume (the data in the read replies) goes straight from - the processing node back to the clients. For read-intensive i/o patterns you can acheive very high throughput rates in this mode. - </para> - - <para> - Note: you can use LVS and public addresses at the same time. - </para> - - <refsect2><title>Configuration</title> - <para> - To activate LVS on a CTDB node you must specify CTDB_PUBLIC_INTERFACE and - CTDB_LVS_PUBLIC_ADDRESS in /etc/sysconfig/ctdb. - </para> - - <para> -You must also specify the "--lvs" command line argument to ctdbd to activate LVS as a capability of the node. This should be done automatically for you by the /etc/init.d/ctdb script. - </para> - - <para> - Example: - <screen format="linespecific"> - CTDB_PUBLIC_INTERFACE=eth0 - CTDB_LVS_PUBLIC_IP=10.0.0.237 - </screen> - </para> - - </refsect2> - - <para> - If you use LVS, you must still have a real/permanent address configured - for the public interface on each node. This address must be routable - and the cluster nodes must be configured so that all traffic back to client - hosts are routed through this interface. This is also required in order - to allow samba/winbind on the node to talk to the domain controller. - (we can not use the lvs IP address to initiate outgoing traffic) - </para> - <para> - I.e. make sure that you can "ping" both the domain controller and also - all of the clients from the node BEFORE you enable LVS. Also make sure - that when you ping these hosts that the traffic is routed out through the - eth0 interface. - </para> - </refsect1> - - - <refsect1><title>REMOTE CLUSTER NODES</title> - <para> -It is possible to have a CTDB cluster that spans across a WAN link. -For example where you have a CTDB cluster in your datacentre but you also -want to have one additional CTDB node located at a remote branch site. -This is similar to how a WAN accelerator works but with the difference -that while a WAN-accelerator often acts as a Proxy or a MitM, in -the ctdb remote cluster node configuration the Samba instance at the remote site -IS the genuine server, not a proxy and not a MitM, and thus provides 100% -correct CIFS semantics to clients. - </para> - - <para> - See the cluster as one single multihomed samba server where one of - the NICs (the remote node) is very far away. - </para> - - <para> - NOTE: This does require that the cluster filesystem you use can cope - with WAN-link latencies. Not all cluster filesystems can handle - WAN-link latencies! Whether this will provide very good WAN-accelerator - performance or it will perform very poorly depends entirely - on how optimized your cluster filesystem is in handling high latency - for data and metadata operations. - </para> - - <para> - To activate a node as being a remote cluster node you need to set - the following two parameters in /etc/sysconfig/ctdb for the remote node: - <screen format="linespecific"> -CTDB_CAPABILITY_LMASTER=no -CTDB_CAPABILITY_RECMASTER=no - </screen> - </para> - - <para> - Verify with the command "ctdb getcapabilities" that that node no longer - has the recmaster or the lmaster capabilities. - </para> - - </refsect1> - - - <refsect1><title>NAT-GW</title> - <para> - Sometimes it is desireable to run services on the CTDB node which will - need to originate outgoing traffic to external servers. This might - be contacting NIS servers, LDAP servers etc. etc. - </para> - <para> - This can sometimes be problematic since there are situations when a - node does not have any public ip addresses assigned. This could - be due to the nobe just being started up and no addresses have been - assigned yet or it could be that the node is UNHEALTHY in which - case all public addresses have been migrated off. - </para> - <para> - If then the service status of CTDB depends on such services being - able to always being able to originate traffic to external resources - this becomes extra troublesome. The node might be UNHEALTHY because - the service can not be reached, and the service can not be reached - because the node is UNHEALTHY. - </para> - <para> - There are two ways to solve this problem. The first is by assigning a - static ip address for one public interface on every node which will allow - every node to be able to route traffic to the public network even - if there are no public addresses assigned to the node. - This is the simplest way but it uses up a lot of ip addresses since you - have to assign both static and also public addresses to each node. - </para> - <refsect2><title>NAT-GW</title> - <para> - A second way is to use the built in NAT-GW feature in CTDB. - With NAT-GW you assign one public NATGW address for each natgw group. - Each NATGW group is a set of nodes in the cluster that shares the same - NATGW address to talk to the outside world. Normally there would only be - one NATGW group spanning the entire cluster, but in situations where one - ctdb cluster spans multiple physical sites it is useful to have one - NATGW group for each of the two sites. - </para> - <para> - There can be multiple NATGW groups in one cluster but each node can only - be member of one NATGW group. - </para> - <para> - In each NATGW group, one of the nodes is designated the NAT Gateway - through which all traffic that is originated by nodes in this group - will be routed through if a public addresses are not available. - </para> - </refsect2> - - <refsect2><title>Configuration</title> - <para> - NAT-GW is configured in /etc/sysconfig/ctdb by setting the following - variables: - </para> - <screen format="linespecific"> -# NAT-GW configuration -# Some services running on nthe CTDB node may need to originate traffic to -# remote servers before the node is assigned any IP addresses, -# This is problematic since before the node has public addresses the node might -# not be able to route traffic to the public networks. -# One solution is to have static public addresses assigned with routing -# in addition to the public address interfaces, thus guaranteeing that -# a node always can route traffic to the external network. -# This is the most simple solution but it uses up a large number of -# additional ip addresses. -# -# A more complex solution is NAT-GW. -# In this mode we only need one additional ip address for the cluster from -# the exsternal public network. -# One of the nodes in the cluster is elected to be hosting this ip address -# so it can reach the external services. This node is also configured -# to use NAT MASQUERADING for all traffic from the internal private network -# to the external network. This node is the NAT-GW node. -# -# All other nodes are set up with a default rote with a metric of 10 to point -# to the nat-gw node. -# -# The effect of this is that only when a node does not have a public address -# and thus no proper routes to the external world it will instead -# route all packets through the nat-gw node. -# -# CTDB_NATGW_NODES is the list of nodes that belong to this natgw group. -# You can have multiple natgw groups in one cluster but each node -# can only belong to one single natgw group. -# -# CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 -# CTDB_NATGW_PUBLIC_IFACE=eth0 -# CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1 -# CTDB_NATGW_PRIVATE_NETWORK=10.1.1.0/24 -# CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes -# -# Normally any node in the natgw group can act as the natgw master. -# In some configurations you may have special nodes that is a part of the -# cluster/natgw group, but where the node lacks connectivity to the -# public network. -# For these cases, set this variable to make these nodes not able to -# become natgw master. -# -# CTDB_NATGW_SLAVE_ONLY=yes - </screen> - </refsect2> - - <refsect2><title>CTDB_NATGW_PUBLIC_IP</title> - <para> - This is an ip address in the public network that is used for all outgoing - traffic when the public addresses are not assigned. - This address will be assigned to one of the nodes in the cluster which - will masquerade all traffic for the other nodes. - </para> - <para> - Format of this parameter is IPADDRESS/NETMASK - </para> - </refsect2> - - <refsect2><title>CTDB_NATGW_PUBLIC_IFACE</title> - <para> - This is the physical interface where the CTDB_NATGW_PUBLIC_IP will be - assigned to. This should be an interface connected to the public network. - </para> - <para> - Format of this parameter is INTERFACE - </para> - </refsect2> - - <refsect2><title>CTDB_NATGW_DEFAULT_GATEWAY</title> - <para> - This is the default gateway to use on the node that is elected to host - the CTDB_NATGW_PUBLIC_IP. This is the default gateway on the public network. - </para> - <para> - Format of this parameter is IPADDRESS - </para> - </refsect2> - - <refsect2><title>CTDB_NATGW_PRIVATE_NETWORK</title> - <para> - This is the network/netmask used for the interal private network. - </para> - <para> - Format of this parameter is IPADDRESS/NETMASK - </para> - </refsect2> - - <refsect2><title>CTDB_NATGW_NODES</title> - <para> - This is the list of all nodes that belong to the same NATGW group - as this node. The default is /etc/ctdb/natgw_nodes. - </para> - </refsect2> - - <refsect2><title>Operation</title> - <para> - When the NAT-GW functionality is used, one of the nodes is elected - to act as a NAT router for all the other nodes in the group when - they need to originate traffic to the external public network. - </para> - <para> - The NAT-GW node is assigned the CTDB_NATGW_PUBLIC_IP to the - specified interface and the provided default route. Given that - the NAT-GW mechanism acts as a last resort, its default route is - added with a metric of 10 so that it can coexist with other - configured static routes. The NAT-GW is configured to act as a - router and to masquerade all traffic it receives from the - internal private network and which is destined to the external - network(s). - </para> - <para> - All other nodes in the group are configured with a default route of - metric 10 pointing to the designated NAT GW node. - </para> - <para> - This is implemented in the 11.natgw eventscript. Please see the - eventscript for further information. - </para> - - </refsect2> - - <refsect2><title>Removing/Changing NATGW at runtime</title> - <para> - The following are the procedures to change/remove a NATGW configuration - at runtime, without having to restart ctdbd. - </para> - - <para> - If you want to remove NATGW completely from a node, use these steps: - </para> - <screen format="linespecific"> -1, Run 'CTDB_BASE=/etc/ctdb /etc/ctdb/events.d/11.natgw removenatgw' -2, Then remove the configuration from /etc/sysconfig/ctdb - </screen> - - <para> - If you want to change the NATGW configuration on a node : - </para> - <screen format="linespecific"> -1, Run 'CTDB_BASE=/etc/ctdb /etc/ctdb/events.d/11.natgw removenatgw' -2, Then change the configuration in /etc/sysconfig/ctdb -3, Run 'CTDB_BASE=/etc/ctdb /etc/ctdb/events.d/11.natgw updatenatgw' - </screen> - - </refsect2> - - </refsect1> - <refsect1> - <title>POLICY ROUTING</title> - - <para> - A node running CTDB may be a component of a complex network - topology. In particular, public addresses may be spread across - several different networks (or VLANs) and it may not be possible - to route packets from these public addresses via the system's - default route. Therefore, CTDB has support for policy routing - via the 13.per_ip_routing eventscript. This allows routing to - be specified for packets sourced from each public address. The - routes are added and removed as CTDB moves public addresses - between nodes. - </para> - - <refsect2> - <title>Configuration variables</title> + <title>DEBUGGING OPTIONS</title> - <para> - There are 4 configuration variables related to policy routing: - </para> + <variablelist> - <variablelist> - <varlistentry><term><varname>CTDB_PER_IP_ROUTING_CONF</varname></term> + <varlistentry> + <term>-i, --interactive</term> <listitem> <para> - The name of a configuration file that specifies the - desired routes for each source address. The configuration - file format is discussed below. A recommended value is - <filename>/etc/ctdb/policy_routing</filename>. + Enable interactive mode. This will make ctdbd run in the + foreground and not detach from the terminal. By default + ctdbd will detach itself and run in the background as a + daemon. </para> + </listitem> + </varlistentry> + <varlistentry> + <term>--listen=<parameter>IPADDR</parameter></term> + <listitem> + <para> + This specifies which IP address that ctdbd will bind to. + </para> + <para> + By default ctdbd will bind to the first address it finds in + the <filename>/etc/ctdb/nodes</filename> file that is also + present on the local system. + </para> <para> - The special value <constant>__auto_link_local__</constant> - indicates that no configuration file is provided and that - CTDB should generate reasonable link-local routes for each - public address. + This option is only required when you want to run multiple + ctdbd daemons/nodes on the same physical host in which case + there would be multiple entries in + <filename>/etc/ctdb/nodes</filename> that would match a + local interface. </para> </listitem> - </varlistentry> + </varlistentry> - <varlistentry><term><varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname></term> + <varlistentry> + <term>--nopublicipcheck</term> <listitem> <para> - This sets the priority (or preference) for the routing - rules that are added by CTDB. + This option is used when testing with multiple local + daemons on a single machine. It disables checks related + to public IP addresses. </para> + </listitem> + </varlistentry> + <varlistentry> + <term>--nosetsched</term> + <listitem> + <para> + This is a debugging option. This option is only used when + debugging ctdbd. + </para> <para> - This should be (strictly) greater than 0 and (strictly) - less than 32766. A priority of 100 is recommended, unless - this conflicts with a priority already in use on the - system. See ip(8) for more details. + Normally ctdbd will change its scheduler to run as a + real-time process. This is the default mode for a normal + ctdbd operation to gurarantee that ctdbd always gets the CPU + cycles that it needs. + </para> + <para> + This option is used to tell ctdbd to + <emphasis>not</emphasis> run as a real-time process and + instead run ctdbd as a normal userspace process. This is + useful for debugging and when you want to run ctdbd under + valgrind or gdb. (You don't want to attach valgrind or gdb + to a real-time process.) </para> </listitem> - </varlistentry> - - <varlistentry> - <term> - <varname>CTDB_PER_IP_ROUTING_TABLE_ID_LOW</varname>, - <varname>CTDB_PER_IP_ROUTING_TABLE_ID_HIGH</varname> - </term> - <listitem> - <para> - CTDB determines a unique routing table number to use for - the routing related to each public address. These - variables indicate the minimum and maximum routing table - numbers that are used. - </para> - - <para> - The ip command uses some reserved routing table numbers - below 255. Therefore, CTDB_PER_IP_ROUTING_TABLE_ID_LOW - should be (strictly) greater than 255. 1000 and 9000 - are recommended values, unless this range conflicts with - routing tables numbers already in use on the system. - </para> - - <para> - CTDB uses the standard file - <filename>/etc/iproute2/rt_tables</filename> to maintain - a mapping between the routing table numbers and labels. - The label for a public address <addr;gt; will look - like ctdb.<addr>. This means that the associated - rules and routes are easy to read (and manipulate). - </para> - </listitem> - </varlistentry> - </variablelist> - </refsect2> - - <refsect2> - <title>Configuration file</title> - - <para> - The format of each line is: - </para> - - <screen> - <public_address> <network> [ <gateway> ] - </screen> - - <para> - Leading whitespace is ignored and arbitrary whitespace may be - used as a separator. Lines that have a "public address" item - that doesn't match an actual public address are ignored. This - means that comment lines can be added using a leading - character such as '#', since this will never match an IP - address. - </para> - - <para> - A line without a gateway indicates a link local route. - </para> - - <para> - For example, consider the configuration line: - </para> - - <screen> - 192.168.1.99 192.168.1.1/24 - </screen> - - <para> - If the corresponding public_addresses line is: - </para> + </varlistentry> - <screen> - 192.168.1.99/24 eth2,eth3 - </screen> + <varlistentry> + <term>--socket=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME specifies the name of the Unix domain socket that + ctdbd will create. This socket is used by local clients to + communicate with ctdbd. + </para> + <para> + The default is <filename>/tmp/ctdb.socket</filename> . You + only need to use this option if you plan to run multiple + ctdbd daemons on the same physical host, usually for + testing. + </para> + </listitem> + </varlistentry> - <para> - <varname>CTDB_PER_IP_ROUTING_RULE_PREF</varname> is 100, and - CTDB adds the address to eth2 then the following routing - information is added: - </para> + <varlistentry> + <term>--script-log-level=<parameter>DEBUGLEVEL</parameter></term> + <listitem> + <para> + This option sets the debug level of event script output to + DEBUGLEVEL. The default is ERR (0). + </para> + <para> + See the <citetitle>DEBUG LEVELS</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for more + information. + </para> + </listitem> + </varlistentry> - <screen> - ip rule add from 192.168.1.99 pref 100 table ctdb.192.168.1.99 - ip route add 192.168.1.0/24 dev eth2 table ctdb.192.168.1.99 - </screen> + <varlistentry> + <term>--sloppy-start</term> + <listitem> + <para> + This is debugging option. This speeds up the initial + recovery during startup at the expense of some consistency + checking. <emphasis>Don't use this option in + production</emphasis>. + </para> + </listitem> + </varlistentry> - <para> - This causes traffic from 192.168.1.1 to 192.168.1.0/24 go via - eth2. - </para> + <varlistentry> + <term>--torture</term> + <listitem> + <para> + This option is only used for development and testing of + CTDB. It adds artificial errors and failures to the + common codepaths in ctdbd to verify that ctdbd can recover + correctly from failures. + </para> + <para> + <emphasis>Do not use this option</emphasis> unless you are + developing and testing new functionality in CTDB. + </para> + </listitem> + </varlistentry> - <para> - The <command>ip rule</command> command will show (something - like - depending on other public addresses and other routes on - the system): - </para> + <varlistentry> + <term>--valgrinding</term> + <listitem> + <para> + This is a debugging option. This option is only used when + debugging ctdbd. This enables additional debugging + capabilities and implies --nosetsched. + </para> + </listitem> + </varlistentry> - <screen> - 0: from all lookup local - 100: from 192.168.1.99 lookup ctdb.192.168.1.99 - 32766: from all lookup main - 32767: from all lookup default - </screen> + </variablelist> + </refsect1> - <para> - <command>ip route show table ctdb.192.168.1.99</command> will show: - </para> + <refsect1> + <title>SEE ALSO</title> + <para> + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, - <screen> - 192.168.1.0/24 dev eth2 scope link - </screen> + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, - <para> - The usual use for a line containing a gateway is to add a - default route corresponding to a particular source address. - Consider this line of configuration: - </para> + <citerefentry><refentrytitle>onnode</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, - <screen> - 192.168.1.99 0.0.0.0/0 192.168.1.1 - </screen> + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, - <para> - In the situation described above this will cause an extra - routing command to be executed: - </para> + <citerefentry><refentrytitle>ctdb-tunables</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, - <screen> - ip route add 0.0.0.0/0 via 192.168.1.1 dev eth2 table ctdb.192.168.1.99 - </screen> + <ulink url="http://ctdb.samba.org/"/> + </para> + </refsect1> + <refentryinfo> + <author> + <contrib> + This documentation was written by + Ronnie Sahlberg, + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> <para> - With both configuration lines, <command>ip route show table - ctdb.192.168.1.99</command> will show: + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. </para> - - <screen> - 192.168.1.0/24 dev eth2 scope link - default via 192.168.1.1 dev eth2 - </screen> - </refsect2> - - <refsect2> - <title>Example configuration</title> - <para> - Here is a more complete example configuration. + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. </para> - - <screen> -/etc/ctdb/public_addresses: - - 192.168.1.98 eth2,eth3 - 192.168.1.99 eth2,eth3 - -/etc/ctdb/policy_routing: - - 192.168.1.98 192.168.1.0/24 - 192.168.1.98 192.168.200.0/24 192.168.1.254 - 192.168.1.98 0.0.0.0/0 192.168.1.1 - 192.168.1.99 192.168.1.0/24 - 192.168.1.99 192.168.200.0/24 192.168.1.254 - 192.168.1.99 0.0.0.0/0 192.168.1.1 - </screen> - <para> - The routes local packets as expected, the default route is as - previously discussed, but packets to 192.168.200.0/24 are - routed via the alternate gateway 192.168.1.254. + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. </para> + </legalnotice> + </refentryinfo> - </refsect2> - </refsect1> - - <refsect1><title>NOTIFICATION SCRIPT</title> - <para> - Notification scripts are used with ctdb to have a call-out from ctdb - to a user-specified script when certain state changes occur in ctdb. - This is commonly to set up either sending SNMP traps or emails - when a node becomes unhealthy and similar. - </para> - <para> - This is activated by setting CTDB_NOTIFY_SCRIPT=<your script> in the - sysconfig file, or by adding --notification-script=<your script>. - </para> - <para> - See /etc/ctdb/notify.sh for an example script. This script - executes files in <filename>/etc/ctdb/notify.d/</filename>, so - it is recommended that you handle notifications using the - example script and by place executable scripts in - <filename>/etc/ctdb/notify.d/</filename> to handle the desired - notifications. - </para> - <para> - CTDB currently generates notifications on these state changes: - </para> - - <refsect2><title>unhealthy</title> - <para> - This call-out is triggered when the node changes to UNHEALTHY state. - </para> - </refsect2> - - <refsect2><title>healthy</title> - <para> - This call-out is triggered when the node changes to HEALTHY state. - </para> - </refsect2> - - <refsect2><title>startup</title> - <para> - This call-out is triggered when ctdb has started up and all managed services are up and running. - </para> - </refsect2> - - </refsect1> - - -<refsect1><title>ClamAV Daemon</title> -<para> -CTDB has support to manage the popular anti-virus daemon ClamAV. -This support is implemented through the -eventscript : /etc/ctdb/events.d/31.clamd. -</para> - -<refsect2><title>Configuration</title> -<para> -Start by configuring CLAMAV normally and test that it works. Once this is -done, copy the configuration files over to all the nodes so that all nodes -share identical CLAMAV configurations. -Once this is done you can proceed with the intructions below to activate -CTDB support for CLAMAV. -</para> - -<para> -First, to activate CLAMAV support in CTDB, edit /etc/sysconfig/ctdb and add the two lines : -</para> -<screen format="linespecific"> -CTDB_MANAGES_CLAMD=yes -CTDB_CLAMD_SOCKET="/path/to/clamd.socket" -</screen> - -<para> -Second, activate the eventscript -</para> -<screen format="linespecific"> -ctdb enablescript 31.clamd -</screen> - -<para> -Third, CTDB will now be starting and stopping this service accordingly, -so make sure that the system is not configured to start/stop this service -automatically. -On RedHat systems you can disable the system starting/stopping CLAMAV automatically by running : -<screen format="linespecific"> -chkconfig clamd off -</screen> -</para> - - -<para> -Once you have restarted CTDBD, use -<screen format="linespecific"> -ctdb scriptstatus -</screen> -and verify that the 31.clamd eventscript is listed and that it was executed successfully. -</para> - -</refsect2> -</refsect1> - - - - - <refsect1><title>SEE ALSO</title> - <para> - ctdb(1), onnode(1) - <ulink url="http://ctdb.samba.org/"/> - </para> - </refsect1> - - <refsect1><title>COPYRIGHT/LICENSE</title> -<literallayout> -Copyright (C) Andrew Tridgell 2007 -Copyright (C) Ronnie sahlberg 2007 - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 3 of the License, or (at -your option) any later version. - -This program is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, see http://www.gnu.org/licenses/. -</literallayout> - </refsect1> </refentry> diff --git a/ctdb/doc/ctdbd.conf.5.xml b/ctdb/doc/ctdbd.conf.5.xml new file mode 100644 index 00000000000..a1f6db5ef61 --- /dev/null +++ b/ctdb/doc/ctdbd.conf.5.xml @@ -0,0 +1,1598 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE refentry + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> + +<refentry id="ctdbd.conf.5"> + + <refmeta> + <refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ctdbd.conf</refname> + <refpurpose>CTDB daemon configuration file</refpurpose> + </refnamediv> + + <refsect1> + <title>DESCRIPTION</title> + + <para> + This file contains CTDB configuration variables that are affect + the operation of CTDB. The default location of this file is + <filename>/etc/ctdb/ctdbd.conf</filename>. + </para> + + <para> + This file is a shell script (see + <citerefentry><refentrytitle>sh</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>) but is usually limited + to simple variable assignments and shell-style comments. + </para> + + <para> + CTDB configuration variables are grouped into several categories below. + </para> + + <para> + Variables defined in this document can also be set in a + distribution-specific configuration file such as + <filename>/etc/sysconfig/ctdb</filename> (Red Hat) or + <filename>/etc/default/ctdb</filename> (Debian). However, these + files should be reserved for variables used by the initscript. + A historical alternative is + <filename>/etc/ctdb/sysconfig/ctdb</filename> - this is + deprecated. + </para> + + </refsect1> + + <refsect1> + <title> + INITSCRIPT CONFIGURATION + </title> + + <para> + Some options must be available to the initscript so they need to + be set in the distribution-specific initscript configuration, + such as <filename>/etc/sysconfig/ctdb</filename> or + <filename>/etc/default/ctdb</filename>. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_PIDFILE=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is the name of the file used to contain the + process ID (PID) of the main CTDB daemon when it is + running. This is passed from the initscript to + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>. + </para> + + <para> + Default is <filename>/var/run/ctdb/ctdbd.pid</filename>. + Corresponds to <option>--pidfile</option>. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title> + GLOBAL CONFIGURATION + </title> + + <para> + These options may be used in the initscripts, daemon and + scripts. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_BASE=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY containing CTDB scripts and configuration files. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_VARDIR=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY containing CTDB files that are modified at + runtime. + </para> + <para> + Defaults to <filename>/var/ctdb</filename>, unless + <filename>/var/lib/ctdb</filename> already exists in which + case it is used. + </para> + </listitem> + </varlistentry> + + + </variablelist> + </refsect1> + + <refsect1> + <title> + DAEMON CONFIGURATION + </title> + + <para> + Variables in this section are processed by + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> and are converted into + command-line arguments to + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>. Correspondence with + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> options is shown for + each variable. The the documentation for the relevant options + for more details. + </para> + + <para> + Many of these variables are also used by event scripts. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_CAPABILITY_LMASTER=yes|no</term> + <listitem> + <para> + Defaults to yes. Corresponds to <option>--no-lmaster</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_CAPABILITY_RECMASTER=yes|no</term> + <listitem> + <para> + Defaults to yes. Corresponds to + <option>--no-recmaster</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_DBDIR=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + Defaults to <varname>CTDB_VARDIR</varname>. Corresponds to + <option>--dbdir</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_DBDIR_PERSISTENT=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + Defaults to <varname>CTDB_VARDIR</varname>/persistent. + Corresponds to <option>--dbdir-persistent</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_DBDIR_STATE=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + Defaults to <varname>CTDB_VARDIR</varname>/state. + Corresponds to <option>--dbdir-state</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_DEBUGLEVEL=<parameter>DEBUGLEVEL</parameter></term> + <listitem> + <para> + Default is ERR (0). Corresponds to <option>-d</option> or + <option>--debug</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_EVENT_SCRIPT_DIR=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + Default is <varname>CTDB_BASE</varname>/events.d, so usually + <filename>/etc/ctdb/events.d</filename>. Corresponds to + <option>--event-script-dir</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_LOGFILE=<parameter>FILENAME</parameter></term> + <listitem> + <para> + Defaults to <filename>/var/log/log.ctdb</filename>. + Corresponds to <option>--logfile</option>. See also + <citetitle>CTDB_SYSLOG</citetitle>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_LOG_RINGBUF_SIZE=<parameter>NUM</parameter></term> + <listitem> + <para> + Default is 0. Corresponds to + <option>--log-ringbuf-size</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_LVS_PUBLIC_IP=<parameter>IPADDR</parameter></term> + <listitem> + <para> + No default. Corresponds to "<option>--lvs</option> + <option>--single-public-ip IPADDR"</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NODES=<parameter>FILENAME</parameter></term> + <listitem> + <para> + Default is <varname>CTDB_BASE</varname>/nodes, so usually + <filename>/etc/ctdb/nodes</filename>. Corresponds to + <option>--nlist</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NOTIFY_SCRIPT=<parameter>FILENAME</parameter></term> + <listitem> + <para> + No default, usually + <filename>/etc/ctdb/notify.sh</filename>. Corresponds to + <option>--notification-script</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MAX_PERSISTENT_CHECK_ERRORS=<parameter>NUM</parameter></term> + <listitem> + <para> + Default 0. Corresponds to + <option>--max-persistent-check-errors</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_PUBLIC_ADDRESSES=<parameter>FILENAME</parameter></term> + <listitem> + <para> + No default, usually + <filename>/etc/ctdb/public_addresses</filename>. + Corresponds to <option>--public-addresses</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_PUBLIC_INTERFACE=<parameter>INTERFACE</parameter></term> + <listitem> + <para> + No default. Corresponds to + <option>--public-interface</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_RECOVERY_LOCK=<parameter>FILENAME</parameter></term> + <listitem> + <para> + Defaults to + <filename>/some/place/on/shared/storage</filename>, which + should be change to a useful value. Corresponds to + <option>--reclock</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SCRIPT_LOG_LEVEL=<parameter>DEBUGLEVEL</parameter></term> + <listitem> + <para> + Defaults to ERR (0). Corresponds to + <option>--script-log-level</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SOCKET=<parameter>FILENAME</parameter></term> + <listitem> + <para> + Defaults to <filename>/tmp/ctdb.socket</filename>. + Corresponds to <option>--socket</option>. + </para> + <para> + If you change this then you probably want to set this in + root's enviroment (perhaps in a file in + <filename>/etc/profile.d</filename>) so that you can use + the <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> command in a + straightforward manner. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_START_AS_DISABLED=yes|no</term> + <listitem> + <para> + Default is no. Corresponds to + <option>--start-as-disabled</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_START_AS_STOPPED=yes|no</term> + <listitem> + <para> + Default is no. Corresponds to + <option>--start-as-stopped</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SYSLOG=yes|no</term> + <listitem> + <para> + Default is no. Corresponds to <option>--syslog</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_TRANSPORT=tcp|infiniband</term> + <listitem> + <para> + Defaults to tcp. Corresponds to + <option>--transport</option>. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para> + While the following variables do not translate into daemon + options they are used by + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> when starting and + stopping <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_SHUTDOWN_TIMEOUT=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is the number of seconds to wait for + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> to shut down + gracefully before giving up and killing it. + </para> + + <para> + Defaults is 30. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_STARTUP_TIMEOUT=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is the number of seconds to wait for + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> complete early + initialisation up to a point where it is unlikely to + abort. If <command>ctdbd</command> doesn't complete the + "setup" event before this timeout then it is killed. + </para> + + <para> + Defaults is 10. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect1> + + <refsect1> + <title>NETWORK CONFIGURATION</title> + + <refsect2> + <title>NAT GATEWAY</title> + + <para> + NAT gateway is used to configure fallback routing for nodes + when they do not host any public IP addresses. For example, + it allows unhealthy nodes to reliably communicate with + external infrastructure. One node in a NAT gateway group will + be designated as the NAT gateway master node and other (slave) + nodes will be configured with fallback routes via the NAT + gateway master node. For more information, see the + <citetitle>NAT GATEWAY</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_NATGW_DEFAULT_GATEWAY=<parameter>IPADDR</parameter></term> + <listitem> + <para> + IPADDR is an alternate network gateway to use on the NAT + gateway master node. A fallback default route is added + via this network gateway. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NATGW_NODES=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME contains the list of nodes that belong to the + same NAT gateway group. + </para> + <para> + File format: + <screen> +<parameter>IPADDR</parameter> + </screen> + </para> + <para> + No default, usually + <filename>/etc/ctdb/natgw_nodes</filename> when enabled. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NATGW_PRIVATE_NETWORK=<parameter>IPADDR/MASK</parameter></term> + <listitem> + <para> + IPADDR/MASK is the private sub-network that is + internally routed via the NAT gateway master node. This + is usually the private network that is used for node + addresses. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NATGW_PUBLIC_IFACE=<parameter>IFACE</parameter></term> + <listitem> + <para> + IFACE is the network interface on which the + CTDB_NATGW_PUBLIC_IP will be configured. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NATGW_PUBLIC_IP=<parameter>IPADDR/MASK</parameter></term> + <listitem> + <para> + IPADDR/MASK indicates the IP address that is used for + outgoing traffic (originating from + CTDB_NATGW_PRIVATE_NETWORK) on the NAT gateway master + node. This <emphasis>must not</emphasis> be a + configured public IP address. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NATGW_SLAVE_ONLY=yes|no</term> + <listitem> + <para> + When set to "yes" a node can not be a NAT gateway master node. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + </variablelist> + + <refsect3> + <title>Example</title> + <screen> +CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes +CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 +CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1 +CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 +CTDB_NATGW_PUBLIC_IFACE=eth0 + </screen> + </refsect3> + + </refsect2> + + <refsect2> + <title>POLICY ROUTING</title> + + <para> + A node running CTDB may be a component of a complex network + topology. In particular, public addresses may be spread + across several different networks (or VLANs) and it may not be + possible to route packets from these public addresses via the + system's default route. Therefore, CTDB has support for + policy routing via the <filename>13.per_ip_routing</filename> + eventscript. This allows routing to be specified for packets + sourced from each public address. The routes are added and + removed as CTDB moves public addresses between nodes. + </para> + + <para> + For more information, see the <citetitle>POLICY + ROUTING</citetitle> section in + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>. + </para> + + <variablelist> + <varlistentry> + <term>CTDB_PER_IP_ROUTING_CONF=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME contains elements for constructing the desired + routes for each source address. + </para> + + <para> + The special FILENAME value + <constant>__auto_link_local__</constant> indicates that no + configuration file is provided and that CTDB should + generate reasonable link-local routes for each public IP + address. + </para> + + <para> + File format: + <screen> +<parameter>IPADDR</parameter> <parameter>DEST-IPADDR/MASK</parameter> <optional><parameter>GATEWAY-IPADDR</parameter></optional> + </screen> + </para> + + <para> + No default, usually + <filename>/etc/ctdb/policy_routing</filename> when enabled. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_PER_IP_ROUTING_RULE_PREF=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM sets the priority (or preference) for the routing + rules that are added by CTDB. + </para> + + <para> + This should be (strictly) greater than 0 and (strictly) + less than 32766. A priority of 100 is recommended, unless + this conflicts with a priority already in use on the + system. See + <citerefentry><refentrytitle>ip</refentrytitle> + <manvolnum>8</manvolnum></citerefentry>, for more details. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + CTDB_PER_IP_ROUTING_TABLE_ID_LOW=<parameter>LOW-NUM</parameter>, + CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=<parameter>HIGH-NUM</parameter> + </term> + <listitem> + <para> + CTDB determines a unique routing table number to use for + the routing related to each public address. LOW-NUM and + HIGH-NUM indicate the minimum and maximum routing table + numbers that are used. + </para> + + <para> + <citerefentry><refentrytitle>ip</refentrytitle> + <manvolnum>8</manvolnum></citerefentry> uses some + reserved routing table numbers below 255. Therefore, + CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) + greater than 255. + </para> + + <para> + CTDB uses the standard file + <filename>/etc/iproute2/rt_tables</filename> to maintain + a mapping between the routing table numbers and labels. + The label for a public address + <replaceable>ADDR</replaceable> will look like + ctdb.<replaceable>addr</replaceable>. This means that + the associated rules and routes are easy to read (and + manipulate). + </para> + + <para> + No default, usually 1000 and 9000. + </para> + </listitem> + </varlistentry> + </variablelist> + + <refsect3> + <title>Example</title> + <screen> +CTDB_PER_IP_ROUTING_CONF=/etc/ctdb/policy_routing +CTDB_PER_IP_ROUTING_RULE_PREF=100 +CTDB_PER_IP_ROUTING_TABLE_ID_LOW=1000 +CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=9000 + </screen> + </refsect3> + + </refsect2> + + <refsect2> + <title>MISCELLANEOUS NETWORK CONFIGURATION</title> + + <variablelist> + + <varlistentry> + <term>CTDB_PARTIALLY_ONLINE_INTERFACES=yes|no</term> + <listitem> + <para> + Whether one or more offline interfaces should cause a + monitor event to fail if there are other interfaces that + are up. If this is "yes" and a node has some interfaces + that are down then <command>ctdb status</command> will + display the node as "PARTIALLYONLINE". + </para> + + <para> + Default is "no". + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect2> + + </refsect1> + + <refsect1> + <title>SERVICE CONFIGURATION</title> + + <para> + CTDB can be configured to manage and/or monitor various NAS (and + other) services via its eventscripts. + </para> + + <para> + In the simplest case CTDB will manage a service. This means the + service will be started and stopped along with CTDB, CTDB will + monitor the service and CTDB will do any required + reconfiguration of the service when public IP addresses are + failed over. + </para> + + <refsect2> + <title>SAMBA</title> + + <refsect3> + <title>Eventscripts</title> + + <simplelist> + <member><filename>49.winbind</filename></member> + <member><filename>50.samba</filename></member> + </simplelist> + </refsect3> + + <variablelist> + + <varlistentry> + <term>CTDB_MANAGES_SAMBA=yes|no</term> + <listitem> + <para> + Should CTDB manage Samba? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MANAGES_WINBIND=yes|no</term> + <listitem> + <para> + Should CTDB manage Winbind? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SAMBA_CHECK_PORTS=<parameter>PORT-LIST</parameter></term> + <listitem> + <para> + When monitoring Samba, check TCP ports in + space-separated PORT-LIST. + </para> + <para> + Default is to monitor ports that Samba is configured to listen on. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SAMBA_SKIP_SHARE_CHECK=yes|no</term> + <listitem> + <para> + As part of monitoring, should CTDB skip the check for + the existence of each directory configured as share in + Samba. This may be desirable if there is a large number + of shares. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SERVICE_NMB=<parameter>SERVICE</parameter></term> + <listitem> + <para> + Distribution specific SERVICE for managing nmbd. + </para> + <para> + Default is distribution-dependant. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term>CTDB_SERVICE_SMB=<parameter>SERVICE</parameter></term> + <listitem> + <para> + Distribution specific SERVICE for managing smbd. + </para> + <para> + Default is distribution-dependant. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SERVICE_WINBIND=<parameter>SERVICE</parameter></term> + <listitem> + <para> + Distribution specific SERVICE for managing winbindd. + </para> + <para> + Default is "winbind". + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect2> + + <refsect2> + <title>NFS</title> + + <para> + This includes parameters for the kernel NFS server and the + user-space + <ulink url="https://github.com/nfs-ganesha/nfs-ganesha/wiki">NFS-Ganesha</ulink> + server. + </para> + + <refsect3> + <title>Eventscripts</title> + + <simplelist> + <member><filename>60.nfs</filename></member> + <member><filename>60.ganesha</filename></member> + </simplelist> + </refsect3> + + <variablelist> + + <varlistentry> + <term>CTDB_CLUSTER_FILESYSTEM_TYPE=gpfs</term> + <listitem> + <para> + The type of cluster filesystem to use with NFS-ganesha. + Currently only "gpfs" is supported. + </para> + <para> + Default is "gpfs". + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MANAGES_NFS=yes|no</term> + <listitem> + <para> + Should CTDB manage NFS? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MONITOR_NFS_THREAD_COUNT=yes|no</term> + <listitem> + <para> + Whether to monitor the NFS kernel server thread count. + </para> + <para> + This works around a limitation in some NFS initscripts + where some threads can be stuck in host filesystem calls + (perhaps due to slow storage), a restart occurs, some + threads don't exit, the start only adds the missing + number of threads, the stuck threads exit, and the + result is a lower than expected thread count. Note that + if you must also set <varname>RPCNFSDCOUNT</varname> + (RedHat/Debian) or <varname>USE_KERNEL_NFSD_NUMBER</varname> + (SUSE) in your NFS configuration so the monitoring code + knows how many threads there should be - if neither of + these are set then this option will be ignored. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NFS_DUMP_STUCK_THREADS=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is the number of NFS kernel server threads to dump + stack traces for if some are still alive after stopping + NFS during a restart. + </para> + <para> + Default is 0. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NFS_SERVER_MODE=kernel|ganesha</term> + <listitem> + <para> + Selects which NFS server to be managed. + </para> + <para> + This replaces the deprecated variable + <varname>NFS_SERVER_MODE</varname>. + </para> + <para> + Default is "kernel". + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NFS_SKIP_KNFSD_ALIVE_CHECK=yes|no</term> + <listitem> + <para> + During monitoring, should CTDB skip the + <command>rpcinfo</command> check that is used to see if + the NFS kernel server is functional. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_NFS_SKIP_SHARE_CHECK=yes|no</term> + <listitem> + <para> + As part of monitoring, should CTDB skip the check for + the existence of each directory exported via NFS. This + may be desirable if there is a large number of exports. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_RPCINFO_LOCALHOST=<parameter>IPADDR</parameter>|<parameter>HOSTNAME</parameter></term> + <listitem> + <para> + IPADDR or HOSTNAME indicates the address that + <command>rpcinfo</command> should connect to when doing + <command>rpcinfo</command> check on RPC service during + monitoring. Optimally this would be "localhost". + However, this can add some performance overheads. + </para> + <para> + Default is "127.0.0.1". + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SKIP_GANESHA_NFSD_CHECK=yes|no</term> + <listitem> + <para> + As part of monitoring, should CTDB skip the check for + the existence of each directory exported via + NFS-Ganesha. This may be desirable if there is a large + number of exports. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect2> + + <refsect2> + <title>APACHE HTTPD</title> + + <para> + CTDB can manage the Apache web server. + </para> + + <refsect3> + <title>Eventscript</title> + + <simplelist> + <member><filename>41.httpd</filename></member> + </simplelist> + </refsect3> + + <variablelist> + <varlistentry> + <term>CTDB_MANAGES_HTTPD=yes|no</term> + <listitem> + <para> + Should CTDB manage the Apache web server? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2> + <title>CLAMAV</title> + + <para> + CTDB has support to manage the popular anti-virus daemon + ClamAV. + </para> + + <refsect3> + <title>Eventscript</title> + + <simplelist> + <member><filename>31.clamd</filename></member> + </simplelist> + + <para> + This eventscript is not enabled by default. Use + <command>ctdb enablescript</command> to enable it. + </para> + + </refsect3> + + <variablelist> + + <varlistentry> + <term>CTDB_MANAGES_CLAMD=yes|no</term> + <listitem> + <para> + Should CTDB manage ClamAV? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_CLAMD_SOCKET=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is the socket to monitor ClamAV. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect2> + + <refsect2> + <title>ISCSI</title> + + <para> + CTDB has support for managing the Linux iSCSI tgtd service. + </para> + + <refsect3> + <title>Eventscript</title> + + <simplelist> + <member><filename>70.iscsi</filename></member> + </simplelist> + </refsect3> + + <variablelist> + + <varlistentry> + <term>CTDB_MANAGES_ISCSI=yes|no</term> + <listitem> + <para> + Should CTDB manage iSCSI tgtd? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_START_ISCSI_SCRIPTS=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY on shared storage containing scripts to start + tgtd for each public IP address. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2> + <title>MULTIPATHD</title> + + <para> + CTDB can monitor multipath devices to ensure that active paths + are available. + </para> + + <refsect3> + <title>Eventscript</title> + + <simplelist> + <member><filename>20.multipathd</filename></member> + </simplelist> + + <para> + This eventscript is not enabled by default. Use + <command>ctdb enablescript</command> to enable it. + </para> + </refsect3> + + <variablelist> + <varlistentry> + <term>CTDB_MONITOR_MPDEVICES=<parameter>MP-DEVICE-LIST</parameter></term> + <listitem> + <para> + MP-DEVICE-LIST is a list of multipath devices for CTDB to monitor? + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2> + <title>VSFTPD</title> + + <para> + CTDB can manage the vsftpd FTP server. + </para> + + <refsect3> + <title>Eventscript</title> + + <simplelist> + <member><filename>40.vsftpd</filename></member> + </simplelist> + </refsect3> + + <variablelist> + <varlistentry> + <term>CTDB_MANAGES_VSFTPD=yes|no</term> + <listitem> + <para> + Should CTDB manage the vsftpd FTP server? + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2> + <title> + SYSTEM RESOURCE MONITORING CONFIGURATION + </title> + + <para> + CTDB can experience seemingly random (performance and other) + issues if system resources become too contrained. Options in + this section can be enabled to allow certain system resources to + be checked. + </para> + + <refsect3> + <title>Eventscripts</title> + + <simplelist> + <member><filename>00.ctdb</filename></member> + <member><filename>40.fs_use</filename></member> + </simplelist> + + <para> + Filesystem usage monitoring is in + <filename>40.fs_use</filename>. This eventscript is not + enabled by default. Use <command>ctdb + enablescript</command> to enable it. + </para> + </refsect3> + + <variablelist> + + <varlistentry> + <term>CTDB_CHECK_FS_USE=<parameter>FS-LIMIT-LIST</parameter></term> + <listitem> + <para> + FS-LIMIT-LIST is a space-separated list of + <parameter>FILESYSTEM</parameter>:<parameter>LIMIT</parameter> + pairs indicating that a node should be flagged unhealthy + if the space used on FILESYSTEM reaches LIMIT%. + </para> + + <para> + No default. + </para> + + <para> + Note that this feature uses the + <filename>40.fs_use</filename> eventscript, which is not + enabled by default. Use <command>ctdb + enablescript</command> to enable it. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_CHECK_SWAP_IS_NOT_USED=yes|no</term> + <listitem> + <para> + Should a warning be logged if swap space is in use. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MONITOR_FREE_MEMORY=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is a lower limit on available system memory, expressed + in megabytes. If this is set and the amount of available + memory falls below this limit then some debug information + will be logged, the node will be disabled and then CTDB + will be shut down. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MONITOR_FREE_MEMORY_WARN=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is a lower limit on available system memory, expressed + in megabytes. If this is set and the amount of available + memory falls below this limit then a warning will be + logged. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + </variablelist> + </refsect2> + + <refsect2> + <title>MISCELLANEOUS SERVICE-RELATED CONFIGURATION</title> + + <variablelist> + + <varlistentry> + <term>CTDB_MANAGED_SERVICES=<parameter>SERVICE-LIST</parameter></term> + <listitem> + <para> + SERVICE-LIST is a space-separated list of SERVICEs that + CTDB should manage. This can be used as an alternative + to the + <varname>CTDB_MANAGES_<replaceable>SERVICE</replaceable></varname> + variables. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SERVICE_AUTOSTARTSTOP=yes|no</term> + <listitem> + <para> + When CTDB should start and stop services if they become + managed or unmanaged. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect2> + + </refsect1> + + <refsect1> + <title> + TUNABLES CONFIGURATION + </title> + + <para> + CTDB tunables (see + <citerefentry><refentrytitle>ctdbd-tunables</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>) can be set from the + configuration file. They are set as follows: + + <literallayout> +CTDB_SET_<replaceable>TUNABLE</replaceable>=<replaceable>VALUE</replaceable> + </literallayout> + </para> + + <para> + For example: + + <screen format="linespecific"> +CTDB_SET_MonitorInterval=20 + </screen> + </para> + </refsect1> + + <refsect1> + <title> + DEBUG AND TEST + </title> + + <para> + Variable in this section are for debugging and testing CTDB. + They should not generally be needed. + </para> + + <variablelist> + + <varlistentry> + <term>CTDB_DEBUG_HUNG_SCRIPT=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is a script to run to log debug information when + an event script times out. + </para> + <para> + Default is <filename><varname>CTDB_BASE</varname>/debug-hung-script.sh</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_DEBUG_LOCKS=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is a script to run to log debug information when + an CTDB fails to freeze databases during recovery. + </para> + <para> + No default, usually + <filename><varname>CTDB_BASE</varname>/debug_locks.sh</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_ETCDIR=<parameter>DIRECTORY</parameter></term> + <listitem> + <para> + DIRECTORY containing system configuration files. This is + used to provide alternate configuration when testing and + should not need to be changed from the default. + </para> + <para> + Default is <filename>/etc</filename>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_INIT_STYLE=debian|redhat|suse</term> + <listitem> + <para> + This is the init style used by the Linux distribution (or + other operating system) being used. This is usually + determined dynamically by checking the system. This + variable is used by the initscript to determine which init + system primitives to use. It is also used by some + eventscripts to choose the name of initscripts for certain + services, since these can vary between distributions. + </para> + <para> + No fixed default. + </para> + <para> + If this option needs to be changed from the calculated + default for the initscript to function properly, then it + must be set in the distribution-specific initscript + configuration, such as + <filename>/etc/sysconfig/ctdb</filename> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_MAX_CORRUPT_DB_BACKUPS=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is the maximum number of volatile TDB database backups + to be kept (for each database) when a corrupt database is + found during startup. Volatile TDBs are zeroed during + startup so backups are needed to debug any corruption that + occurs before a restart. + </para> + <para> + Default is 10. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_RC_LOCAL=<parameter>FILENAME</parameter></term> + <listitem> + <para> + FILENAME is a script fragment to be sourced by the + <filename>functions</filename> that is sourced by scripts. + On example use would be to override function definitions + in unit tests. As a sanity check, this file must be + executable for it to be used. + </para> + <para> + No default. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_RUN_TIMEOUT_MONITOR=yes|no</term> + <listitem> + <para> + Whether CTDB should simulate timing out monitor events. + This uses the <filename>99.timeout</filename> eventscript. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SCRIPT_DEBUGLEVEL=<parameter>NUM</parameter></term> + <listitem> + <para> + NUM is the level debugging messages printed by CTDB + scripts. Setting this to a higher number (e.g. 4) will + cause some scripts to log more messages. + </para> + <para> + Default is 2. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_SUPPRESS_COREFILE=yes|no</term> + <listitem> + <para> + Whether CTDB core files should be suppressed. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>CTDB_VALGRIND=yes|no|<parameter>COMMAND</parameter></term> + <listitem> + <para> + If "yes", this causes + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> to be run under + <citerefentry><refentrytitle>valgrind</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> with logs going to + <filename>/var/log/ctdb_valgrind</filename>. If neither + "yes" nor "no" then the value is assumed to be a COMMAND + (e.g. a <command>valgrind</command> variation, a + <citerefentry><refentrytitle>gdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry> command) that is + used in place of the default <command>valgrind</command> + command. In either case, the <option>--valgrind</option> + option is passed to <command>ctdbd</command>. + </para> + <para> + Default is no. + </para> + </listitem> + </varlistentry> + + </variablelist> + + </refsect1> + + + <refsect1> + <title>FILES</title> + + <simplelist> + <member><filename>/etc/ctdb/ctdbd.conf</filename></member> + <member><filename>/etc/sysconfig/ctdb</filename></member> + <member><filename>/etc/default/ctdb</filename></member> + <member><filename>/etc/ctdb/sysconfig/ctdb</filename></member> + </simplelist> + </refsect1> + + <refsect1> + <title>SEE ALSO</title> + <para> + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>onnode</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb-tunables</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="http://ctdb.samba.org/"/> + </para> + </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + +</refentry> diff --git a/ctdb/doc/ctdbd_wrapper.1.xml b/ctdb/doc/ctdbd_wrapper.1.xml new file mode 100644 index 00000000000..b119681a93a --- /dev/null +++ b/ctdb/doc/ctdbd_wrapper.1.xml @@ -0,0 +1,106 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE refentry + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> + +<refentry id="ctdbd_wrapper.1"> + + <refmeta> + <refentrytitle>ctdbd_wrapper</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ctdbd_wrapper</refname> + <refpurpose>Wrapper for ctdbd</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>ctdbd_wrapper</command> + <arg choice="req"><replaceable>PIDFILE</replaceable></arg> + <group choice="req"> + <arg choice="plain">start</arg> + <arg choice="plain">stop</arg> + </group> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1> + <title>DESCRIPTION</title> + <para> + ctdbd_wrapper is used to start or stop the main CTDB daemon. + </para> + + <para> + <replaceable>PIDFILE</replaceable> specifies the location of the + file containing the PID of the main CTDB daemon. + </para> + + <para> + ctdbd_wrapper constructs command-line options for ctdbd from + configuration variables specified in + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>. + </para> + + <para> + See <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> for an overview of CTDB. + </para> + </refsect1> + + <refsect1> + <title>SEE ALSO</title> + <para> + <citerefentry><refentrytitle>ctdbd</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdbd.conf</refentrytitle> + <manvolnum>5</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="http://ctdb.samba.org/"/> + </para> + </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Amitay Isaacs, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + +</refentry> diff --git a/ctdb/doc/ltdbtool.1.xml b/ctdb/doc/ltdbtool.1.xml index fe9e3e880d6..790db0ce2fe 100644 --- a/ctdb/doc/ltdbtool.1.xml +++ b/ctdb/doc/ltdbtool.1.xml @@ -1,88 +1,75 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry - PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" - "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> <refentry id="ltdbtool.1"> -<refmeta> - <refentrytitle>ltdbtool</refentrytitle> - <manvolnum>1</manvolnum> - <refmiscinfo class="source"> </refmiscinfo> - <refmiscinfo class="manual"> </refmiscinfo> -</refmeta> + <refmeta> + <refentrytitle>ltdbtool</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + <refnamediv> + <refname>ltdbtool</refname> + <refpurpose>manipulate CTDB's local TDB files</refpurpose> + </refnamediv> -<refnamediv> - <refname>ltdbtool</refname> - <refpurpose>handle ctdb's local tdb copies </refpurpose> -</refnamediv> - -<refsynopsisdiv> - <cmdsynopsis> - <command>ltdbtool [OPTIONS] COMMAND</command> - </cmdsynopsis> + <refsynopsisdiv> + <cmdsynopsis> + <command>ltdbtool</command> + <arg rep="repeat"><replaceable>OPTION</replaceable></arg> + <arg choice="req"><replaceable>COMMAND</replaceable></arg> + <arg><replaceable>COMMAND-ARGS</replaceable></arg> + </cmdsynopsis> </refsynopsisdiv> - <refsect1><title>DESCRIPTION</title> + <refsect1> + <title>DESCRIPTION</title> + + <para> + ltdbtool is a utility to manipulate CTDB's local TDB databases + (LTDBs) without connecting to a CTDB daemon. + </para> + <para> - ltdbtool is a utility to cope with ctdb's local tdb copies (LTDBs) - without connecting to a ctdb daemon. + It can be used to: </para> - <para>It can be used to - <itemizedlist spacing='compact'> <!-- mark='opencircle' --> + + <itemizedlist> <listitem> <para> - dump the contents of a LTDB, optionally printing the ctdb - record header information, + dump the contents of a LTDB, optionally printing the CTDB + record header information, </para> </listitem> <listitem> <para> convert between an LTDB and a non-clustered tdb - by adding or removing ctdb headers and + by adding or removing CTDB headers and </para> </listitem> <listitem> - <para>convert between 64 and 32 bit LTDBs where the ctdb record - headers differ by 4 bytes of padding. - </para> + <para>convert between 64 and 32 bit LTDBs where the CTDB record + headers differ by 4 bytes of padding. + </para> </listitem> </itemizedlist> - </para> </refsect1> - <refsect1><title>COMMANDS</title> + <refsect1> + <title>OPTIONS</title> <variablelist> - <varlistentry><term>help</term> - <listitem> - <para> - Print a help text. - </para> - </listitem> - </varlistentry> - <varlistentry><term>dump <IDB></term> + <varlistentry><term>-e</term> <listitem> <para> - Dump the contents of a LTDB file to standard output in a - human-readable format. + Dump empty records. These are normally excluded. </para> </listitem> </varlistentry> - <varlistentry><term>convert <IDB> <ODB></term> - <listitem> - <para> - Make a copy of a LTDB optionally adding or removing ctdb headers. - </para> - </listitem> - </varlistentry> - </variablelist> - </refsect1> - - <refsect1><title>OPTIONS</title> - - <variablelist> <varlistentry><term>-p</term> <listitem> <para> @@ -91,25 +78,33 @@ </listitem> </varlistentry> - <varlistentry><term>-s {0|32|64}</term> + <varlistentry> + <term> + -s + <group choice="req"> + <arg choice="plain">0</arg> + <arg choice="plain">32</arg> + <arg choice="plain">64</arg> + </group> + </term> <listitem> <para> - Specify how to determine the ctdb record header size + Specify how to determine the CTDB record header size for the input database: <variablelist spacing="normal"> <varlistentry><term>0</term> <listitem> - <para>no ctdb header</para> + <para>no CTDB header</para> </listitem> </varlistentry> <varlistentry><term>32</term> <listitem> - <para>ctdb header size of a 32 bit system (20 bytes)</para> + <para>CTDB header size of a 32 bit system (20 bytes)</para> </listitem> </varlistentry> <varlistentry><term>64</term> <listitem> - <para>ctdb header size of a 64 bit system (24 bytes)</para> + <para>CTDB header size of a 64 bit system (24 bytes)</para> </listitem> </varlistentry> </variablelist> @@ -118,27 +113,37 @@ </listitem> </varlistentry> - <varlistentry><term>-o {0|32|64}</term> + <varlistentry> + <term> + -o + <group choice="req"> + <arg choice="plain">0</arg> + <arg choice="plain">32</arg> + <arg choice="plain">64</arg> + </group> + </term> <listitem> <para> - Specify how to determine the ctdb record header size - for the output database, see -s + Specify how to determine the CTDB record header size + for the output database, see -s. </para> </listitem> </varlistentry> - <varlistentry><term>-S <SIZE></term> + <varlistentry><term>-S <parameter>SIZE</parameter></term> <listitem> <para> - Explicitly specify the ctdb record header size of the input database in bytes. + Explicitly specify the CTDB record header SIZE of the + input database in bytes. </para> </listitem> </varlistentry> - <varlistentry><term>-O <SIZE></term> + <varlistentry><term>-O <parameter>SIZE</parameter></term> <listitem> <para> - Explicitly specify the ctdb record header size for the output database in bytes. + Explicitly specify the CTDB record header SIZE for the + output database in bytes. </para> </listitem> </varlistentry> @@ -146,14 +151,51 @@ <varlistentry><term>-h</term> <listitem> <para> - Print a help text. + Print help text. </para> </listitem> </varlistentry> </variablelist> </refsect1> - <refsect1><title>EXAMPLES</title> + <refsect1> + <title>COMMANDS</title> + + <variablelist> + <varlistentry><term>help</term> + <listitem> + <para> + Print help text. + </para> + </listitem> + </varlistentry> + + <varlistentry><term>dump <parameter>IDB</parameter></term> + <listitem> + <para> + Dump the contents of an LTDB input file IDB to standard + output in a human-readable format. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + convert <parameter>IDB</parameter> <parameter>ODB</parameter> + </term> + <listitem> + <para> + Copy an LTDB input file IDB to output file ODB, optionally + adding or removing CTDB headers. + </para> + </listitem> + </varlistentry> + </variablelist> + </refsect1> + + <refsect1> + <title>EXAMPLES</title> + <para> Print a local tdb in "tdbdump" style: </para> @@ -169,21 +211,21 @@ </screen> <para> - Strip the ctdb headers from records: + Strip the CTDB headers from records: </para> <screen format="linespecific"> ltdbtool convert -o0 idmap2.tdb.0 idmap.tdb </screen> <para> - Strip 64 bit ctdb headers from records, running on i386: + Strip 64 bit CTDB headers from records, running on i386: </para> <screen format="linespecific"> ltdbtool convert -s64 -o0 idmap2.tdb.0 idmap.tdb </screen> <para> - Strip the ctdb headers from records by piping through tdbrestore: + Strip the CTDB headers from records by piping through tdbrestore: </para> <screen format="linespecific"> ltdbtool dump idmap2.tdb.0 | tdbrestore idmap.tdb @@ -206,27 +248,53 @@ <refsect1><title>SEE ALSO</title> <para> - ctdbd(1), ctdb(1), tdbdump(1), tdbrestore(1), + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>tdbdump</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>tdbrestore</refentrytitle> + <manvolnum>1</manvolnum></citerefentry>, + + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + <ulink url="http://ctdb.samba.org/"/> </para> </refsect1> - <refsect1><title>COPYRIGHT/LICENSE</title> -<literallayout> -Copyright (C) Gregor Beck 2011 -Copyright (C) Michael Adam 2011 - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 3 of the License, or (at -your option) any later version. - -This program is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, see http://www.gnu.org/licenses/. -</literallayout> - </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by Gregor Beck + </contrib> + </author> + + <copyright> + <year>2011</year> + <holder>Gregor Beck</holder> + <holder>Michael Adam</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + </refentry> diff --git a/ctdb/doc/onnode.1.xml b/ctdb/doc/onnode.1.xml index 65b17924cec..561764c1f0e 100644 --- a/ctdb/doc/onnode.1.xml +++ b/ctdb/doc/onnode.1.xml @@ -1,210 +1,221 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry - PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" - "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> <refentry id="onnode.1"> -<refmeta> - <refentrytitle>onnode</refentrytitle> - <manvolnum>1</manvolnum> - <refmiscinfo class="source"> </refmiscinfo> - <refmiscinfo class="manual"> </refmiscinfo> -</refmeta> - - -<refnamediv> - <refname>onnode</refname> - <refpurpose>run commands on ctdb nodes</refpurpose> -</refnamediv> - -<refsynopsisdiv> - <cmdsynopsis> - <command>onnode [OPTION] ... NODES COMMAND ...</command> - </cmdsynopsis> -</refsynopsisdiv> - - <refsect1><title>DESCRIPTION</title> + <refmeta> + <refentrytitle>onnode</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>onnode</refname> + <refpurpose>run commands on CTDB cluster nodes</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>onnode</command> + <arg rep="repeat"><replaceable>OPTION</replaceable></arg> + <arg choice="req"><replaceable>NODES</replaceable></arg> + <arg choice="req"><replaceable>COMMAND</replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1> + <title>DESCRIPTION</title> <para> onnode is a utility to run commands on a specific node of a CTDB cluster, or on all nodes. </para> <para> - The NODES option specifies which node to run a command on. You - can specify a numeric node number (from 0 to N-1) or a - descriptive node specification (see DESCRIPTIVE NODE - SPECIFICATIONS below). You can also specify lists of nodes, - separated by commas, and ranges of numeric node numbers, - separated by dashes. If nodes are specified multiple times then - the command will be executed multiple times on those nodes. The - order of nodes is significant. + <replaceable>NODES</replaceable> specifies which node(s) to run + a command on. See section <citetitle>NODES + SPECIFICATION</citetitle> for details. </para> <para> - The COMMAND can be any shell command. The onnode utility uses - ssh or rsh to connect to the remote nodes and run the command. + <replaceable>COMMAND</replaceable> can be any shell command. The + onnode utility uses ssh or rsh to connect to the remote nodes + and run the command. </para> </refsect1> - <refsect1><title>DESCRIPTIVE NODE SPECIFICATIONS</title> - - <para> - The following descriptive node specification can be used in - place of numeric node numbers: - </para> + <refsect1> + <title>OPTIONS</title> <variablelist> - <varlistentry><term>all</term> + <varlistentry><term>-c</term> <listitem> <para> - All nodes. + Execute COMMAND in the current working directory on the + specified nodes. </para> </listitem> </varlistentry> - <varlistentry><term>any</term> + + <varlistentry><term>-f <parameter>FILENAME</parameter></term> <listitem> <para> - A node where ctdbd is running. This semi-random but - there is a bias towards choosing a low numbered node. + Specify an alternative nodes FILENAME to use instead of + the default. This option overrides the CTDB_NODES_FILE + environment variable. See the discussion of + <filename>/etc/ctdb/nodes</filename> in the FILES section + for more details. </para> </listitem> </varlistentry> - <varlistentry><term>ok | healthy</term> + + <varlistentry><term>-n</term> <listitem> <para> - All nodes that are not disconnected, banned, disabled or - unhealthy. + Allow nodes to be specified by name rather than node + numbers. These nodes don't need to be listed in the nodes + file. You can avoid the nodes file entirely by combining + this with <code>-f /dev/null</code>. </para> </listitem> </varlistentry> - <varlistentry><term>con | connected</term> + + <varlistentry><term>-o <parameter>PREFIX</parameter></term> <listitem> <para> - All nodes that are not disconnected. + Causes standard output from each node to be saved into a + file with name PREFIX.<replaceable>IP</replaceable>. </para> </listitem> </varlistentry> - <varlistentry><term>lvs | lvsmaster</term> + + <varlistentry><term>-p</term> <listitem> <para> - The current LVS master. + Run COMMAND in parallel on the specified nodes. The + default is to run COMMAND sequentially on each node. </para> </listitem> </varlistentry> - <varlistentry><term>natgw | natgwlist</term> + + <varlistentry><term>-P</term> <listitem> <para> - The current NAT gateway. + Push files to nodes. Names of files to push are specified + rather than the usual command. Quoting is fragile/broken + - filenames with whitespace in them are not supported. </para> </listitem> </varlistentry> - <varlistentry><term>rm | recmaster</term> + + <varlistentry><term>-q</term> <listitem> <para> - The current recovery master. + Do not print node addresses. Normally, onnode prints + informational node addresses if more than one node is + specified. This overrides -v. </para> </listitem> </varlistentry> - </variablelist> - </refsect1> - <refsect1><title>OPTIONS</title> - - <variablelist> - <varlistentry><term>-c</term> + <varlistentry><term>-v</term> <listitem> <para> - Execute COMMAND in the current working directory on the - specified nodes. + Print node addresses even if only one node is specified. + Normally, onnode prints informational node addresses when + more than one node is specified. </para> </listitem> </varlistentry> - <varlistentry><term>-o <prefix></term> + <varlistentry><term>-h, --help</term> <listitem> <para> - Causes standard output from each node to be saved into a - file with name <prefix>.<ip>. + Show a short usage guide. </para> </listitem> </varlistentry> + </variablelist> + </refsect1> - <varlistentry><term>-p</term> + <refsect1> + <title>NODES SPECIFICATION</title> + + <para> + Nodes can be specified via numeric node numbers (from 0 to N-1) + or mnemonics. Multiple nodes are specified using lists of + nodes, separated by commas, and ranges of numeric node numbers, + separated by dashes. If nodes are specified multiple times then + the command will be executed multiple times on those nodes. The + order of nodes is significant. + </para> + + <para> + The following mnemonics are available: + </para> + + <variablelist> + <varlistentry><term>all</term> <listitem> <para> - Run COMMAND in parallel on the specified nodes. The - default is to run COMMAND sequentially on each node. + All nodes. </para> </listitem> </varlistentry> - - <varlistentry><term>-q</term> + <varlistentry><term>any</term> <listitem> <para> - Do not print node addresses. Normally, onnode prints - informational node addresses if more than one node is - specified. This overrides -v. + A node where ctdbd is running. This semi-random but + there is a bias towards choosing a low numbered node. </para> </listitem> </varlistentry> - - <varlistentry><term>-n</term> + <varlistentry><term>ok | healthy</term> <listitem> <para> - Allow nodes to be specified by name rather than node - numbers. These nodes don't need to be listed in the nodes - file. You can avoid the nodes file entirely by combining - this with <code>-f /dev/null</code>. + All nodes that are not disconnected, banned, disabled or + unhealthy. </para> </listitem> </varlistentry> - - <varlistentry><term>-f <file></term> + <varlistentry><term>con | connected</term> <listitem> <para> - Specify an alternative nodes file to use instead of the - default. This option overrides the CTDB_NODES_FILE - environment variable. See the discussion of - <filename>/etc/ctdb/nodes</filename> in the FILES section - for more details. + All nodes that are not disconnected. </para> </listitem> </varlistentry> - - <varlistentry><term>-v</term> + <varlistentry><term>lvs | lvsmaster</term> <listitem> <para> - Print a node addresses even if only one node is specified. - Normally, onnode prints informational node addresses when - more than one node is specified. + The current LVS master. </para> </listitem> </varlistentry> - - <varlistentry><term>-P</term> + <varlistentry><term>natgw | natgwlist</term> <listitem> <para> - Push files to nodes. Names of files to push are specified - rather than the usual command. Quoting is fragile/broken - - filenames with whitespace in them are not supported. + The current NAT gateway. </para> </listitem> </varlistentry> - - <varlistentry><term>-h, --help</term> + <varlistentry><term>rm | recmaster</term> <listitem> <para> - Show a short usage guide. + The current recovery master. </para> </listitem> </varlistentry> </variablelist> </refsect1> - <refsect1><title>EXAMPLES</title> + <refsect1> + <title>EXAMPLES</title> + <para> - The following command would show the process ID of ctdb on all nodes + The following command would show the process ID of ctdbd on all nodes </para> <screen format="linespecific"> - onnode all pidof ctdbd + onnode all ctdb getpid </screen> <para> @@ -216,10 +227,11 @@ </screen> <para> - The following command would restart the ctdb service on all nodes. + The following command would restart the ctdb service on all + nodes, in parallel. </para> <screen format="linespecific"> - onnode all service ctdb restart + onnode -p all service ctdb restart </screen> <para> @@ -231,16 +243,25 @@ </screen> </refsect1> - <refsect1><title>ENVIRONMENT</title> + <refsect1> + <title>ENVIRONMENT</title> <variablelist> + <varlistentry><term><envar>CTDB_BASE</envar></term> + <listitem> + <para> + Directory containing CTDB configuration files. The + default is <filename>/etc/ctdb</filename>. + </para> + </listitem> + </varlistentry> + <varlistentry><term><envar>CTDB_NODES_FILE</envar></term> <listitem> <para> Name of alternative nodes file to use instead of the - default. See the discussion of - <filename>/etc/ctdb/nodes</filename> in the FILES section - for more details. + default. See the <citetitle>FILES</citetitle> section for + more details. </para> </listitem> </varlistentry> @@ -248,7 +269,8 @@ </variablelist> </refsect1> - <refsect1><title>FILES</title> + <refsect1> + <title>FILES</title> <variablelist> <varlistentry><term><filename>/etc/ctdb/nodes</filename></term> @@ -259,13 +281,13 @@ </para> <para> Actually, the default is - <filename>$<envar>CTDB_BASE</envar>/nodes</filename>, - where <envar>$CTDB_BASE</envar> defaults to - <filename>/etc/ctdb</filename>. If a relative path is - given (via the -f option or <envar>$CTDB_BASE</envar>) and - no corresponding file exists relative to the current - directory then the file is also searched for in the - <filename>$<envar>CTDB_BASE</envar></filename> directory. + <filename>$CTDB_BASE/nodes</filename>, where + <envar>CTDB_BASE</envar> defaults to + <filename>/etc/ctdb</filename>. If a relative path is + given (via the -f option or <envar>CTDB_BASE</envar>) and + no corresponding file exists relative to the current + directory then the file is also searched for in the + <filename>$CTDB_BASE</filename> directory. </para> </listitem> </varlistentry> @@ -274,39 +296,64 @@ <listitem> <para> If this file exists it is sourced by onnode. The main - purpose is to allow the administrator to set $SSH to - something other than "ssh". In this case the -t option is - ignored. For example, the administrator may choose to use - use rsh instead of ssh. + purpose is to allow the administrator to set + <envar>SSH</envar> to something other than "ssh". In this + case the -t option is ignored. For example, the + administrator may choose to use use rsh instead of ssh. </para> </listitem> </varlistentry> </variablelist> </refsect1> - <refsect1><title>SEE ALSO</title> + <refsect1> + <title>SEE ALSO</title> + <para> - ctdbd(1), ctdb(1), <ulink url="http://ctdb.samba.org/"/> + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="http://ctdb.samba.org/"/> </para> </refsect1> - <refsect1><title>COPYRIGHT/LICENSE</title> -<literallayout> -Copyright (C) Andrew Tridgell 2007 -Copyright (C) Ronnie sahlberg 2007 -Copyright (C) Martin Schwenke 2008 - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 3 of the License, or (at -your option) any later version. - -This program is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, see http://www.gnu.org/licenses/. -</literallayout> - </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by + Andrew Tridgell, + Martin Schwenke + </contrib> + </author> + + <copyright> + <year>2007</year> + <holder>Andrew Tridgell</holder> + <holder>Ronnie Sahlberg</holder> + </copyright> + <copyright> + <year>2008</year> + <holder>Martin Schwenke</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + </refentry> diff --git a/ctdb/doc/ping_pong.1.xml b/ctdb/doc/ping_pong.1.xml index 2e4b016bbc2..47e90e896a7 100644 --- a/ctdb/doc/ping_pong.1.xml +++ b/ctdb/doc/ping_pong.1.xml @@ -1,67 +1,80 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry - PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" - "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> <refentry id="ping_pong.1"> -<refmeta> - <refentrytitle>ping_pong</refentrytitle> - <manvolnum>1</manvolnum> - <refmiscinfo class="source"> </refmiscinfo> - <refmiscinfo class="manual"> </refmiscinfo> -</refmeta> - - -<refnamediv> - <refname>ping_pong</refname> - <refpurpose>measures the ping-pong byte range lock latency</refpurpose> -</refnamediv> - -<refsynopsisdiv> - <cmdsynopsis> - <command>ping_pong [options] <file> <num_locks></command> - </cmdsynopsis> - - <cmdsynopsis> - <command>ping_pong</command> - <arg choice="opt">-r</arg> - <arg choice="opt">-w</arg> - <arg choice="opt">-m</arg> - <arg choice="opt">-c</arg> - </cmdsynopsis> - -</refsynopsisdiv> - - <refsect1><title>DESCRIPTION</title> + <refmeta> + <refentrytitle>ping_pong</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">ctdb</refmiscinfo> + <refmiscinfo class="manual">CTDB - clustered TDB database</refmiscinfo> + </refmeta> + + <refnamediv> + <refname>ping_pong</refname> + <refpurpose>measures the ping-pong byte range lock latency</refpurpose> + </refnamediv> + + <refsynopsisdiv> + <cmdsynopsis> + <command>ping_pong</command> + <group choice="req"> + <arg choice="plain">-r</arg> + <arg choice="plain">-w</arg> + <arg choice="plain">-rw</arg> + </group> + <arg>-m</arg> + <arg>-c</arg> + <arg choice="req"><replaceable>FILENAME</replaceable></arg> + <arg choice="req"><replaceable>NUM-LOCKS</replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + + <refsect1> + <title>DESCRIPTION</title> <para> - This measures the ping-pong byte range lock latency. It is - especially useful on a cluster of nodes sharing a common lock - manager as it will give some indication of the lock managers - performance under stress. + ping_pong measures the byte range lock latency. It is especially + useful on a cluster of nodes sharing a common lock manager as it + will give some indication of the lock manager's performance + under stress. </para> + <para> + FILENAME is a file on shared storage to use for byte range + locking tests. + </para> + + <para> + NUM-LOCKS is the number of byte range locks, so needs to be + (strictly) greater than the number of nodes in the cluster. + </para> </refsect1> - <refsect1><title>OPTIONS</title> + <refsect1> + <title>OPTIONS</title> <variablelist> - <varlistentry><term>-r</term> - <listitem> - <para> - do reads + <varlistentry> + <term>-r</term> + <listitem> + <para> + test read performance </para> - </listitem> + </listitem> </varlistentry> - <varlistentry><term>-w</term> + <varlistentry> + <term>-w</term> <listitem> <para> - do writes + test write performance </para> </listitem> </varlistentry> - <varlistentry><term>-m</term> + <varlistentry> + <term>-m</term> <listitem> <para> use mmap @@ -69,8 +82,9 @@ </listitem> </varlistentry> - <varlistentry><term>-c</term> - <listitem> + <varlistentry> + <term>-c</term> + <listitem> <para> validate the locks </para> @@ -80,7 +94,8 @@ </variablelist> </refsect1> - <refsect1><title>EXAMPLES</title> + <refsect1> + <title>EXAMPLES</title> <para> Testing lock coherence </para> @@ -103,27 +118,47 @@ </screen> </refsect1> - <refsect1><title>SEE ALSO</title> + <refsect1> + <title>SEE ALSO</title> <para> - <ulink url="https://wiki.samba.org/index.php/Ping_pong"/>, ctdb(1), ctdbd(1) + <citerefentry><refentrytitle>ctdb</refentrytitle> + <manvolnum>7</manvolnum></citerefentry>, + + <ulink url="https://wiki.samba.org/index.php/Ping_pong"/> + </para> </refsect1> - <refsect1><title>COPYRIGHT/LICENSE</title> -<literallayout> -Copyright (C) Andrew Tridgell 2002 - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 3 of the License, or (at -your option) any later version. - -This program is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, see http://www.gnu.org/licenses/. -</literallayout> - </refsect1> + + <refentryinfo> + <author> + <contrib> + This documentation was written by Mathieu Parent + </contrib> + </author> + + <copyright> + <year>2002</year> + <holder>Andrew Tridgell</holder> + </copyright> + <legalnotice> + <para> + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 3 of + the License, or (at your option) any later version. + </para> + <para> + This program is distributed in the hope that it will be + useful, but WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. See the GNU General Public License for more details. + </para> + <para> + You should have received a copy of the GNU General Public + License along with this program; if not, see + <ulink url="http://www.gnu.org/licenses"/>. + </para> + </legalnotice> + </refentryinfo> + </refentry> |