summaryrefslogtreecommitdiff
path: root/zookeeper-client
Commit message (Collapse)AuthorAgeFilesLines
* [ZOOKEEPER-4674] TestReadOnlyClient.cc: Stop/start "normal" server in test ↵Enrico Olivelli2023-02-131-0/+18
| | | | | setUp/tearDown (#1983) Co-authored-by: Damien Diederen <dd@crosstwine.com>
* ZOOKEEPER-4654: Fix C client test compilation error in Util.cc.Chris Nauroth2022-12-181-0/+2
| | | | | | | | Author: Chris Nauroth <cnauroth@apache.org> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1967 from cnauroth/ZOOKEEPER-4654
* ZOOKEEPER-4641: GH CI fails with error: implicit declaration of function ↵Enrico Olivelli2022-12-151-5/+0
| | | | | | | | | | | | | | | | FIPS_mode Modifications: - remove the block, as it is used only for debug More details here: https://issues.apache.org/jira/browse/ZOOKEEPER-4641 Author: Enrico Olivelli <enrico.olivelli@datastax.com> Reviewers: Andor Molnar <andor@apache.org>, Chris Nauroth <cnauroth@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1951 from eolivelli/fix/ZOOKEEPER-4641
* ZOOKEEPER-4479: C tests: Avoid some jitter which results in flaky testsDamien Diederen2022-03-013-15/+20
| | | | | | | | | | | | With these commits, the various tests in `TestOperations.cc` set the `last_sent` and `last_recv` fields of the "force-connected" ZooKeeper handle to deterministic values (which match the initial value of the time mock). When that value is not specified, these fields get set to the "real" time at the end of the `forceConnected` function, which can be sufficiently different from the value held in the mock for some of the tests to fail. Author: Damien Diederen <ddiederen@apache.org> Reviewers: Enrico Olivelli <eolivelli@apache.org> Closes #1827 from ztzg/ZOOKEEPER-4479-flaky-c-test-operations
* [maven-release-plugin] prepare for next development iterationEnrico Olivelli2022-02-032-2/+2
|
* ZOOKEEPER-4461: Use logback in zookeeper-contrib projectAndor Molnar2022-02-011-1/+1
| | | | | | | | | | | | I've only found Ant references in the `-recipes` projects which I don't necesseraly want to migrate, so this patch is focusing on `-contrib` projects. Please review. Author: Andor Molnar <andor@apache.org> Reviewers: Enrico Olivelli <eolivelli@apache.org> Closes #1807 from anmolnar/ZOOKEEPER-4461
* fix a typo in zookeeper.hfengpan04032021-11-211-1/+1
| | | | | | | | | | Typo fix Author: fengpan0403 <fengpan0403@163.com> Reviewers: maoling <maoling@apache.org> Closes #1778 from fengpan0403/master
* ZOOKEEPER-4342: Fix: Robustify C client against errors during SASL negotiationDamien Diederen2021-08-251-0/+3
| | | | | | | | | | | | | | Before this, the client was ignoring the error field of the response header, and only considering SASL-level errors. This commit makes it consider `hdr.err`. It also zeroes the `res` data structure, to avoid a crash in `deallocate` if `deserialize` is skipped, and sets `input_buffer` to `NULL` to avoid a double-free. (I looked into adding a non-regression test, but doing so requires adding quite a bit of infrastructure to the mocks so that the SASL library can be correctly initialized. Punting for now.) Author: Damien Diederen <ddiederen@apache.org> Reviewers: Enrico Olivelli<eolivelli@apache.org> Closes #1733 from ztzg/ZOOKEEPER-4342-robustify-c-client-sasl-errors
* ZOOKEEPER-4210: Preserve return code from nonblocking sendSam Mikes2021-03-091-44/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | Async API calls attempt to flush the send buffer, which calls flush_send_queue(); and can report ZOPERATIONTIMEOUT ZSYSTEMERROR ZCONNECTIONLOSS Specifically: send_buffer() calls send(2) with MSG_NOSIGNAL, which can return EPIPE; then send_buffer return -1, causing ZCONNECTIONLOSS from flush_send_queue(). Current async API calls drop the return value from flush_send_queue(), as below: adaptor_send_queue(zh, 0); return (rc < 0)?ZMARSHALLINGERROR:ZOK; The async API then returns ZOK instead of ZCONNECTIONLOSS. Author: Sam Mikes <smikes@apple.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Damien Diederen <ddiederen@apache.org> Closes #1602 from smikes/asyncsend-returncode-3.6
* ZOOKEEPER-4217: add new arg 'func' to handle_socket_error_msg()Sam Mikes2021-03-071-14/+16
| | | | | | | | | | Add argument `const char *func` typically initialized with __func__ to include current function name in log message. Author: Sam Mikes <smikes@apple.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>, Damien Diederen <ddiederen@apache.org> Closes #1609 from smikes/handle-msg-report-func
* ZOOKEEPER-4201: C client: Disable SASL deprecation warnings on macOSDamien Diederen2021-02-174-0/+46
| | | | | | | | | | | | | | | | This patch works around the numerous deprecation notices added to the CyrusSASL library on macOS. It is a direct "port" of the solution to MESOS-3030, which hit exactly the same problem: https://issues.apache.org/jira/browse/MESOS-3030 https://reviews.apache.org/r/39230/diff/3/ The PR also includes a fix for the the `clockid_t` compilation issue mentioned in the ticket description, but the test suite as a whole remains broken on macOS as its linker does not support the `--wrap` option. Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1593 from ztzg/ZOOKEEPER-4201-catalina-c-client-fixes
* Change version to 3.8.0 in C clientDamien Diederen2021-01-143-3/+3
|
* [maven-release-plugin] prepare for next development iterationDamien Diederen2021-01-142-2/+2
|
* ZOOKEEPER-3973: Add GitHub Actions for CIChristopher Tubbs2021-01-131-30/+49
| | | | | | | | | | | | | | | | | | | | | * Add GitHub Actions workflows * Add a CI workflow for PRs and branches * Add a manual workflow for custom builds * Remove redundant builds from Travis config (leave only s390x) * Update build status badges on README * Move inline links to bottom of the Markdown page for readability * Other: normalize URLs by using https and removing trailing slashes and unnecessary trailing .html, and fix syntax for section header * Remove extraneous unactivated configuration from pom.xml files (and update an XML comment to remove an extra confusing backslash) * Add ability to disable cppunit tests independently of surefire tests * Build only selected tests in CI workflow for now Author: Christopher Tubbs <ctubbsii@apache.org> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Damien Diederen <ddiederen@apache.org> Closes #1508 from ctubbsii/ZOOKEEPER-3973-github-actions
* ZOOKEEPER-3426: C client: Consider encoded length before completing handshakeDamien Diederen2021-01-134-4/+148
| | | | | | | | | | | | | | Suhas Dantkale noticed that the C client, which uses non-blocking sockets, could end up reading only the first 40 bytes of a 41-byte connection response and concluding that its handshake was complete, leaving one byte in the pipe. Similarly (though even less likely), the library could be left hanging waiting for a 41th byte after having received a fragmented 40-byte response from an old server. This patch makes the logic resistant to fragmentation by considering the server-provided length encoded in the response packet. Author: Damien Diederen <dd@crosstwine.com> Reviewers: Mate Szalay-Beko <symat@apache.org>, Enrico Olivelli <eolivelli@apache.org> Closes #1576 from ztzg/ZOOKEEPER-3426-c-client-short-handshake
* ZOOKEEPER-3983: Avoid mocking unix domain socketsChristopher Tubbs2020-10-221-0/+5
| | | | | | | | | | | | | | | This fixes issues with the C client tests, causing them to hang if certain nameservices are in use (such as SSS), which use UNIX domain sockets for getting the `getlogin()` and `getpwuid_r()` This change makes the mocking of sockets less aggressive, since the tests do not need to mock `AF_UNIX` socket types. Author: Christopher Tubbs <ctubbsii@apache.org> Reviewers: Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1514 from ctubbsii/ZOOKEEPER-3983-no-mock-unix-sockets
* ZOOKEEPER-3561: Generalize target authentication scheme for ZooKeeper ↵Mohammad Arshad2020-10-202-2/+2
| | | | | | | | | | | | authentication enforcement. Added enforce.auth.enabled and enforce.auth.scheme to enforce any authentication scheme. Author: Mohammad Arshad <arshad@apache.org> Reviewers: Michael Han <hanm@apache.org>, Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@gmail.com>, Andor Molnár <andor@apache.org> Closes #1500 from arshadmohammad/ZOOKEEPER-3561-master
* ZOOKEEPER-3951: C Client: Fix compilation with SASL disabledDamien Diederen2020-10-121-2/+14
| | | | | | | | | | | | The `sasl_client` field doesn't exist when configured without SASL. Use an accessor outside of the blocks already guarded by `HAVE_CYRUS_SASL_H`. (Reported by [Parag](https://issues.apache.org/jira/browse/ZOOKEEPER-3951).) Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1477 from ztzg/ZOOKEEPER-3951-no-sasl-compilation
* ZOOKEEPER-3954: C client: GCC 10 compilation fixesDamien Diederen2020-10-121-14/+14
| | | | | | | | | | | | | | | | | | The in this PR avoids a confusing and scary compilation issue [encountered by Michael Hudson-Doyle](https://issues.apache.org/jira/browse/ZOOKEEPER-3954) when building the C client with GCC 10 and "aggressive" optimization settings: > `free_auth_completions` is being inlined into `free_completions`, and this lets gcc see that members of `a_list` are being accessed without initialization This is (fortunately!) a red herring: what GCC doesn't see is that, in practice, `zoo_lock_auth` always returns zero, and that `a_list` is always initialized in the conditional block which follows it. That issue is easily "fixed" by removing the `if`. The rest of the client code doesn't check `zoo_lock_auth`'s return value--and we have bigger issues if unconditional locks start failing anyway. See also https://github.com/apache/zookeeper/pull/1481 and https://github.com/apache/zookeeper/pull/1486. Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1487 from ztzg/ZOOKEEPER-3954-gcc10-compilation-fixes-master
* ZOOKEEPER-3944: Fix sasl memory leakXiaotian2020-09-281-0/+1
| | | | | | | | | | Within ```zoo_sasl_client_destroy```, it carefully free all members of the sasl client, while here it doesn't free the struct itself before set it to NULL. This error is detected by leak sanitizer. Author: Xiaotian <523950475@qq.com> Reviewers: Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1466 from xiaotian-qi/ZOOKEEPER-3944
* ZOOKEEPER-3937: C client: avoid out-of-order packets during SASL negotiationDamien Diederen2020-09-282-1/+91
| | | | | | | | | | | | | | This patch prevents SASL-enabled C clients from sending request packets for as long as the SASL negociation is not complete. "Ideally," applications should wait for `ZOO_CONNECTED_STATE` or `ZOO_READONLY_STATE` events before issuing requests, but this is not what happens in the wild. Without this patch, it was easy to cause a desynchronization by creating a handle and blasting requests right away. (The added test simply avoids doing a `ctx.waitForConnected()`.) Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1457 from ztzg/ZOOKEEPER-3937-c-client-out-of-order-packets
* ZOOKEEPER-3863: Fix c client unit test (local session must be enabled for ↵Michael Han2020-09-201-0/+1
| | | | | | | | | | | | | | | | | | | | | read only observers to work). Problem ZOOKEEPER-3863 requires local session to be enabled for read only server to work. When merge ZOOKEEPER-3863 (in c47ef905e077184bc5b7f555a3e2dfeb6dc046e1), we only updated Java tests, and missed C tests. As a result, Jenkins build is broken. Solution Enable local session for C client tests. Notes This was not caught by Jenkins when we merge ZOOKEEPER-3863 because previous Jenkins was testing master branch (as discussed in https://github.com/apache/zookeeper/pull/1460). Author: Michael Han <lhan@twitter.com> Reviewers: Enrico Olivelli <eolivelli@gmail.com> Closes #1462 from hanm/fixme
* ZOOKEEPER-3885: add locking for watchers hashtablesTudor Bosman2020-07-314-2/+49
| | | | | | | | | | See the comments in the JIRA issue. Author: Tudor Bosman <tudor@rockset.com> Reviewers: Mate Szalay-Beko <symat@apache.org>, Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@apache.org> Closes #1403 from tudor/htlocking1
* ZOOKEEPER-3101: add reminder to ZOO_ERRORSKent R. Spillner2020-07-172-2/+6
| | | | | | | | | | | | | | Add comment at bottom of ZOO_ERRORS reminding developers to also update zerror(int) when adding or changing values. Suggested by phunt in https://github.com/apache/zookeeper/pull/575#issuecomment-406356144. Signed-off-by: Kent R. Spillner <kspillneracm.org> Author: Kent R. Spillner <kspillner@acm.org> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>, Benjamin Reed <breed@apache.org> Closes #581 from sl4mmy/zookeeper-3101
* ZOOKEEPER-1998: Allow C client to throttle host name resolutionsDamien Diederen2020-05-174-16/+171
| | | | | | | | | | | | | | | | Some environments experience high DNS load because of the name resolutions introduced by [ZOOKEEPER-1355](https://issues.apache.org/jira/browse/ZOOKEEPER-1355). This patch allows clients to set a minimum delay to observe between "routine" resolutions using a `zoo_set_servers_resolution_delay` API function. An application can influence the rate of polling via its `delay_ms` parameter: when set to a value greater than zero, the client skips most "routine" resolutions which would have happened in a window of that many milliseconds since the last successful one. Setting `delay_ms` to `0` disables the new logic, reverting to the default behavior. Setting it to `-1` disables network resolutions during normal operation (but not, e.g., on connection loss). Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Mate Szalay-Beko <symat@apache.org>, Suhas Dantkale Closes #1068 from ztzg/ZOOKEEPER-1998-throttle-dns-resolutions
* ZOOKEEPER-3791: Miscellaneous Maven fixesChristopher Tubbs2020-05-042-23/+12
| | | | | | | | | | | | | | | | | | * Fix fatjar and full-build profile problems with redundant module activation * Remove "activeByDefault" profiles that don't do anything useful * Make javadoc a bit quieter * Fix assembly plugin warning about `/` * Fix antrun warnings about tasks config renamed to target * Fix docs module packaging (doesn't need to be jar) * Fix zookeeper-it module by moving classes into standarad Maven directory layout and configuring surefire plugin to find the tests Author: Christopher Tubbs <ctubbsii@apache.org> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org> Closes #1314 from ctubbsii/misc-maven-fixes
* ZOOKEEPER-3726: invalid ipv6 address comparison in C clientVladislav Tyulbashev2020-04-112-2/+103
| | | | | | | | | | | | | | Fix for https://issues.apache.org/jira/browse/ZOOKEEPER-3726 sockaddr_storage can contain ipv4 or ipv6 address. If address is ipv6, then we need to compare more bytes. In this PR correct comparison of sockaddr_storage was added. Author: Vladislav Tyulbashev <vtyulb@vtyulb.ru> Reviewers: Norbert Kalmar <nkalmar@apache.org>, Mate Szalay-Beko <symat@apache.org> Closes #1252 from vtyulb/ZOOKEEPER-3726
* ZOOKEEPER-3654: Incorrect *_CFLAGS handling in AutomakeDamien Diederen2020-03-257-50/+53
| | | | | | | | | | | | | | | | | | | | | | | | The `Makefile.am` distributed with the C client defines some per-target `*_CFLAGS` and `*_CXXFLAGS` variables. These however, do not reference `AM_CFLAGS` (resp. AM_CXXFLAGS`, which means that some options (notably `-Wall`) are missing when building subsets of the code. Dixit the [Automake docs](https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html): > In compilations with per-target flags, the ordinary ‘AM_’ form of > the flags variable is _not_ automatically included in the > compilation (however, the user form of the variable _is_ included). > So for instance, if you want the hypothetical ‘maude’ compilations > to also use the value of ‘AM_CFLAGS’, you would need to write: > > maude_CFLAGS = ... your flags ... $(AM_CFLAGS) Restoring the flags, however, causes compilation failures (in the library) and a slew of new warnings (in the tests) which had not been noticed because of the missing options. This series of patches (all "tagged" ZOOKEEPER-3654) fix these warnings and errors before re-enabling `-Wall` and friends for all targets. Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Andor Molnar <andor@apache.org> Closes #1186 from ztzg/ZOOKEEPER-3654-incorrect-automake-flags
* ZOOKEEPER-3683: Discard requests that are delayed longer than a confi…Jie Huang2020-03-242-1/+4
| | | | | | | | | | | …gured threshold Author: Jie Huang <jiehuang@fb.com> Author: Ivailo Nedelchev <nedelchev@fb.com> Reviewers: Michael Han <hanm@apache.org>, Allan Lyu <fangmin@apache.org>, Damien Diederen <dd@crosstwine.com> Closes #1211 from jhuan31/ZOOKEEPER-3683
* ZOOKEEPER-3741: fix buffer length in C client causing warning with new gccMate Szalay-Beko2020-02-271-1/+1
| | | | | | | | | | | | We get a warning that we are trying to call like `sprintf(buf,"%s:%d",addrstr,port);`, and both `buf` and `addrstr` are 128 long char arrays. So in theory, we can overflow. The fix is to increase the length of the destination string array (`buf`). Actually this problem only causing compile time warning / failure only on the 3.5.5 and 3.5.6. On 3.5.7 this was fixed and on 3.6+ branches the compiler can not detect the problem due to some code refactoring made by ZOOKEEPER-3068, but the issue is still present on the master branch. Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org> Closes #1273 from symat/ZOOKEEPER-3741
* ZOOKEEPER-1105: wait for server response in C client zookeeper_closeMate Szalay-Beko2020-01-281-13/+57
| | | | | | | | | | | | | | | | | | | | | | **Thanks for Lincoln Lee for the original fix!** In the current implementation, we always get a WARN in server side ("EndOfStreamException: Unable to read additional data from client") whenever we close a zookeeper handler from the C client. This also happens in the end of every execution of the command line C client. The reason is that currently we don't wait for the response from the server when we initiate the closing of the client connection, and we terminate the socket on the client side too early. I tested the patch both on linux and windows. I also tested it both with NIO and Netty server side socket implementations. Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com> Reviewers: Andor Molnar <andor@apache.org>, Norbert Kalmar <nkalmar@apache.org> Closes #1176 from symat/ZOOKEEPER-1105
* ZOOKEEPER-1112: Add (Cyrus) SASL authentication support to C client libraryDamien Diederen2020-01-2216-151/+1446
| | | | | | | | | | | | | | | | | | | | | | | | | | This is a "respin" of https://github.com/apache/zookeeper/pull/1054, which I withdrew due to some annoying shortcomings. This changeset allows C clients to use SASL to authenticate with the ZooKeeper server. It is loosely based on patches #1 and #2 by Tom Klonikowski, at https://reviews.apache.org/r/2252/, but the result has been extensively reworked to follow the semantics of the Java client: * No SASL operations are exposed through the API; * The configuration is provided, and stored, at "handle init time"; * SASL authentication is automatically performed after each (re)connect. It introduces an optional dependency on the Cyrus SASL library, which can either be autodetected (default) or configured using the `--without-sasl`/`--with-sasl[=DIR]` flags, or -DWITH_CYRUS_SASL for CMake/Windows. `TestServerRequireClientSASLAuth.cc` has been renamed to `TestSASLAuth.cc`, and a test has been added which successfully (re)authenticates using the `DIGEST-MD5` mechanism. The code has also been used to successfully authenticate clients via `GSSAPI`/Kerberos. This commit also adds SASL support to the `cli.c` client. Co-authored-by: Tom Klonikowski <klonik_tinformatik.haw-hamburg.de> Author: Damien Diederen <dd@crosstwine.com> Reviewers: Mate Szalay-Beko <szalay.beko.mate@gmail.com>, Norbert Kalmar <nkalmar@apache.org> Closes #1134 from ztzg/ZOOKEEPER-1112-c-client-sasl-support-v2
* ZOOKEEPER-3678: Remove Redundant GroupID from Maven POMsDavid Mollitor2020-01-132-4/+0
| | | | | | | | Author: David Mollitor <dmollitor@apache.org> Reviewers: Norbert Kalmar <nkalmar@apache.org> Closes #1206 from belugabehr/ZOOKEEPER-3678
* ZOOKEEPER-3640: Implement "batch mode" in cli_mtDamien Diederen2020-01-061-2/+23
| | | | | | | | | | | | | | | Batch mode never was implemented in `cli_mt`. This patch seems to work, but: 1. There may be a cleaner way of waiting for the completion; 2. ~~`nanosleep` is POSIX; the Windows path should probably use `Sleep`~~ (DONE). symat: Comments welcome. Author: Damien Diederen <dd@crosstwine.com> Reviewers: andor@apache.org Closes #1173 from ztzg/ZOOKEEPER-3640-implement-batch-mode-in-cli-mt
* Change version to 3.7.0 in C clientEnrico Olivelli2019-12-153-3/+3
|
* [maven-release-plugin] prepare for next development iterationEnrico Olivelli2019-12-152-2/+2
|
* ZOOKEEPER-3630: Autodetection of openssl during ZooKeeper C client buildMate Szalay-Beko2019-12-148-31/+183
| | | | | | | | | | | | | | | | | | | | | | **Thanks for ztzg for raising the issue and suggesting the solution!** In this patch we enhance the way how the openssl library is found during C client build. I introduced and documented a new build parameter for `configure` (`--with-openssl=...`), `cmake` (`-D WITH_OPENSSL=...`) and `mvn` (`-Dc-client-openssl=...`), so independent of the build tool, the build will work the same way: - By default, the OpenSSL library will be autodetected. If the library is found, then the C-client will be compiled with SSL support, otherwise we get a warning message, but the build will continue without SSL support. The SSL related unit tests will be skipped as well. - you can explicitly disable the SSL support (e.g. `-Dc-client-openssl=no`) - or you can specify an alternative path to look for the openssl library (e.g. `-Dc-client-openssl=/path/to/openssl`) I tested the patch - using mvn on linux - using make on linux - using cmake on linux and on windows In addition, I also added the openssl dependencies to the dev docker image and copied the OpenSSL license to the C client LICENSE file. Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@cloudera.com>, Damien Diederen Closes #1159 from symat/ZOOKEEPER-3630
* ZOOKEEPER-3599: cli.c: Resuscitate "old-style" argument parsingDamien Diederen2019-12-111-31/+50
| | | | | | | | | | | | | | ZOOKEEPER-2122 added SSL support to the C client and to the `cli_st`/`mt` tools. It introduced (much-welcome!) GNU-style `getopt_long` argument parsing, but did not try to preserve backwards compatibility. An earlier version of this https://github.com/apache/zookeeper/pull/1131 introduced (optional) POSIX-style `getopt` argument parsing, and was consequently largely obsoleted by the SSL update. It did, however, support "old-style" arguments to avoid [breaking workflows](https://xkcd.com/1172/). This patch salvages that feature while preserving the `getopt_long` goodness. It is "legacy-only"; adding new positional arguments is not supported—as discussed in this thread: https://github.com/apache/zookeeper/pull/1131#pullrequestreview-320870245. Author: Damien Diederen <dd@crosstwine.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org> Closes #1131 from ztzg/ZOOKEEPER-3599-use-getopt-in-cli-c
* ZOOKEEPER-3635: Use Docker and Maven Release Plugin to prepare ZooKeeper ↵Enrico Olivelli2019-12-043-7/+26
| | | | | | | | | | | | | | | | | | releases - configure maven-release-plugin - configure maven-scm-plugin - use maven-antrun-plugin in order to alter C client files during release procedure - update Apache Parent pom to 21 - use openjdk8 for building releases Author: Enrico Olivelli <eolivelli@apache.org> Author: Enrico Olivelli <eolivelli@gmail.com> Author: Enrico Olivelli <enrico.olivelli@diennea.com> Reviewers: Norbert Kalmar <nkalmar@apache.org> Closes #1162 from eolivelli/fix/ZOOKEEPER-3635-new-release-proc
* ZOOKEEPER-2122: add SSL support for C-clientMate Szalay-Beko2019-11-2126-225/+1536
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR is based on the works of Asnish Amarnath and Suhas Dantkale. Most of the kudos should go to them and those who were reviewing all the previous PRs. **The PR includes the following changes from PR#639:** - OPENSSL 1.1.1 version support in C-client **The PR includes the following changes from PR#990:** - also supporting OPENSSL 1.0.2 - SSL connection on non-blocking socket is handled correctly - Support of Certificate Chains - Fix Memory leaks - Dynamically generated test certificates **The following new changes were added into the PR:** - fix CMake + VisualStudio2019 build with windows - fix C CLI to compile / work both with windows and linux (I tested them manually) - fix (and simplify) the way how the server is started with C unit tests, so it is compatible with maven build now - the test case `testReadOnly` was failing with the previous PR because there was a bug in the C-client code, I fixed that - I also added new test case: `testReadOnlyWithSSL` **Testing this PR on linux:** ``` git clean -xdf # compile ZooKeeper server plus the C-client code mvn clean install -DskipTests -Pfull-build # compile and execute C-client unit tests cd zookeeper-client/ mvn clean install -Pfull-build ``` **Compile the code on windows (only cmake is supported):** - download C-Make:  https://cmake.org/download/ - Install community edition of Visual Studio: https://visualstudio.microsoft.com/downloads/ - Download OpenSSL (e.g. 1.0.2): https://slproweb.com/products/Win32OpenSSL.html (e.g. install it to `c:\OpenSSL-Win64`) - compile the java code using: `mvn clean install -DskipTests` - go to the Client folder: `cd zookeeper-client\zookeeper-client-c` - configure the project:  `cmake . -D WITH_OPENSSL=c:\OpenSSL-Win64` - build the project: `cmake --build .` **Testing the C-client with SSL manually:** - run the `zookeeper-client/zookeeper-client-c/ssl/gencerts.sh` to generate certificate files (e.g. copy it to an empty folder like `/tmp/ssl/` and start is) - start a ZooKeeper server, using some config file like this one: ``` tickTime=3000 initLimit=10 syncLimit=5 dataDir=/tmp/zkdata secureClientPort=22281 serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ssl.keyStore.location=/tmp/ssl/server.jks ssl.keyStore.password=password ssl.trustStore.location=/tmp/ssl/servertrust.jks ssl.trustStore.password=password ``` - start the command line client (cli.exe on windows, cli_mt or cli_st on linux): `./cli_mt --host localhost:22281 --ssl /tmp/ssl/server.crt,/tmp/ssl/client.crt,/tmp/ssl/clientkey.pem,password` Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com> Author: Mate Szalay-Beko <mszalay@cloudera.com> Reviewers: andor@apache.org Closes #1107 from symat/ZOOKEEPER-2122 and squashes the following commits: 08294ce91 [Mate Szalay-Beko] ZOOKEEPER-2122: update readme + use FQDN in SSL certs during testing 17e504a98 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into ZOOKEEPER-2122 317241d13 [Mate Szalay-Beko] ZOOKEEPER-2122: minor fix in SSL certificates used for testing 6f37b6653 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into HEAD 980914313 [Mate Szalay-Beko] ZOOKEEPER-2122: add SSL support for C-client
* ZOOKEEPER-3436: Enhance Mavenized Make C clientMate Szalay-Beko2019-10-101-61/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | Based on eolivelli 's [previous pull request](https://github.com/apache/zookeeper/pull/993) we enhanced now the maven C client build: - we can compile now the C-client without tests (using the -DskipTests option) - we can optionally enable the test coverage calculation for the C-client (-Pc-test-coverage) - we also package the C-client into the binary tarball (when -Pfull-build is used) - I also updated the README_packaging.md file to make user's life easier (happier? :) ) I tested the build on ubuntu 16.04. Author: Mate Szalay-Beko <mszalay@cloudera.com> Author: Enrico Olivelli <eolivelli@apache.org> Reviewers: nkalmar@apache.org, eolivelli@apache.org, andor@apache.org Closes #1078 from symat/ZOOKEEPER-3436 and squashes the following commits: 18250df41 [Mate Szalay-Beko] ZOOKEEPER-3436 remove the C binaries from the binary tarball; update the readme files 57b7cd61d [Mate Szalay-Beko] ZOOKEEPER-3436 fixing file permissions of usr/bin/* in binary tarball a2ac025a0 [Mate Szalay-Beko] ZOOKEEPER-3436 enhance / format README_packaging.md 30eea5e27 [Mate Szalay-Beko] ZOOKEEPER-3436 exclude README_packaging.md from license check 936550e48 [Mate Szalay-Beko] ZOOKEEPER-3436 fixing README_packaging.md f01029524 [Mate Szalay-Beko] ZOOKEEPER-3436 Enhance Mavenized Make C client 44f6ba5d2 [Enrico Olivelli] Better .gitignore 204603751 [Enrico Olivelli] Enhance Mavenized Make C client
* ZOOKEEPER-2282: C Client: chroot not stripped in asynchronous callbacksDamien Diederen2019-09-272-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is related to ZOOKEEPER-1027, which adjusted the result of `create` calls to account for the chroot path. It did not, however, correct the problem for asynchronous calls: $ cli_st $ENSEMBLE_HOSTS/rt-46444 create /x/fred Creating [/x/fred] node (mode: 0) [/x/fred]: rc = 0 name = /rt-46444/x/fred create2 /x/wilma Creating [/x/wilma] node (mode: 0) [/x/wilma]: rc = 0 name = /rt-46444/x/wilma ctime = Wed Aug 28 18:14:16 2019 czxid=1f00000041 mtime=Wed Aug 28 18:14:16 2019 mzxid=1f00000041 version=0 aversion=0 ephemeralOwner = 0 (It turns out that there was already a ticket, with a patch, for this issue. I am mentioning Andrew Grasso as a co-author. This commit also avoids the test failure, however.) Co-authored-by: Andrew Grasso <andrew-grassousers.noreply.github.com> Author: Damien Diederen <dd@crosstwine.com> Reviewers: eolivelli@apache.org, andor@apache.org Closes #1066 from ztzg/ZOOKEEPER-2282-c-client-async-chroot-paths
* ZOOKEEPER-2891: Invalid processing of zookeeper_close for mutli-requestAlexander A. Strelets2019-08-061-0/+151
| | | | | | | | | | | | | | | | | | | | When I call `zookeeper_close()` while there is a pending multi request, I expect the request completes with `ZCLOSING` status. But with the existing code I actually get the following: * the program exits with `SIGABRT` from `assert(entry)` in `deserialize_multi()` * and even if I remove this assertion and just break the enclosing loop, the returned status is `ZOK` but not `ZCLOSING` So, there are two defects with processing calls to `zookeeper_close()` for pending multi requests: improper assertion in implementation and invalid status in confirmation. Proposed changes remove these defects. For more details see: https://issues.apache.org/jira/browse/ZOOKEEPER-2891 Author: Alexander A. Strelets <streletsaa@gmail.com> Reviewers: Michael Han <hanm@apache.org> Closes #999 from xoiss/ZOOKEEPER-2891
* ZOOKEEPER-1634: hardening security by teaching server to enforce client ↵Michael Han2019-07-244-2/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | authentication. ## Motivation Previously ZooKeeper server is open to the world as the server does not enforce client authentication - anonymous clients are allowed to establish session with server. This behavior raises a couple of issues from the perspective of performance and security for example: * It is easy to launch a DDoS attack to server, by having a fleet of anonymous clients connect to the ensemble, as each session would consume valuable resources (socket, memory, etc) from server. * It is cumbersome to enforce certain security models with the presence of anonymous clients login - for example as clients are not trusted the root ACL has to be disabled for writes to world, among other configurations an admin has to do to secure a cluster in a multi-tenant environment. So the goal here is to address such issue by hardening ZooKeeper security to provide a more confined access option that user could opt-in, which in addition to the existing ACLs together could lead to more secured / resource optimal ensemble. ## Design Abstract * Introduce a new server side Java property that if set, ZooKeeper server will only accept connections and requests from clients that have authenticated with server via SASL. * Clients that are not configured with SASL authentication, or configured with SASL but fail authentication (i.e. with invalid credential) will not be able to establish a session with server. A typed error code (-124) will be delivered in such case, both Java and C client will close the session with server thereafter, without further attempts on retrying to reconnect. * This feature overrules the server property "zookeeper.allowSaslFailedClients". So even if server is configured to allow clients that fail SASL authentication to login, client will not be able to establish a session with server if this feature is enabled. * Only support SASL because only SASL authentication has the property that no operations will happen until SASL authentication process finished. Thus, the decision of whether to close the session or not can be quickly made on server side upon receiving a client connection request. We could later add support for other auth scheme via add_auth_info if that's desired (if we do, then a session has to be maintained until add_auth_info is invoked.). * As a side benefit, this PR fixes an issue mentioned in ZOOKEEPER-2346 by correctly propagate events from server to client side so a SASL auth failure will manifest as an auth / config failure rather than generic ConnectionLoss event. JIRA: https://issues.apache.org/jira/browse/ZOOKEEPER-1634 The PR also covers (or part of): https://issues.apache.org/jira/browse/ZOOKEEPER-2462 https://issues.apache.org/jira/browse/ZOOKEEPER-2526 https://issues.apache.org/jira/browse/ZOOKEEPER-2346 Author: Michael Han <lhan@twitter.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Andor Molnar <andor@apache.org> Closes #118 from hanm/ZOOKEEPER-1634
* ZOOKEEPER-2894: Memory and completions leak on zookeeper_closeAlexander A. Strelets2019-07-082-0/+118
| | | | | | | | | | | | | | | | | | | | | If I call `zookeeper_close()` when the call-stack does not pass through any of zookeeper mechanics (for example, some of completion handler called back by zookeeper) the following happens: 1. If there are requests waiting for responses in `zh.sent_requests` queue, they all are removed from this queue and each of them is "completed" with personal fake response having status `ZCLOSING`. Such fake responses are put into `zh.completions_to_process` queue. It's Ok 2. But then, `zh.completions_to_process` queue is left unhandled. **Neither completion callbacks are called, nor dynamic memory allocated for fake responses is freed** 3. Different structures within `zh` are dismissed and finally `zh` is freed Consequently we have a tiny **memory leak** and, which is much more important, **completions leak**. I.e. completions are not called at all. This is not the case if I call `zookeeper_close()` when the call-stack does pass through some of zookeeper mechanics. Here, zookeeper client handles completions properly. Proposed changes remove this defects. For more details see: https://issues.apache.org/jira/browse/ZOOKEEPER-2894 Author: Alexander A. Strelets <streletsaa@gmail.com> Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Michael Han <hanm@apache.org> Closes #1000 from xoiss/ZOOKEEPER-2894
* ZOOKEEPER-3362: Create a simple checkstyle fileEnrico Olivelli2019-05-081-0/+2
| | | | | | | | | | | | | | | | | | | | | Create a basic checkstyle file, in order to cover the minimal check on author tags. This is needed in order to drop old ANT based precommit job (see ZOOKEEPER-3351) We will not remove legacy checkstyle configuration file in zookeeper-server/src/test/resources/checkstyle.xml because it is referred by ANT build.xml files (even if we are not actually using that target). This task won't add a complete checkstyle configuration with usual checks because it would imply almost a change at every .java in the codebase. Author: Enrico Olivelli <eolivelli@apache.org> Reviewers: andor@apache.org Closes #909 from eolivelli/fix/checkstyle and squashes the following commits: fdda1571c [Enrico Olivelli] use a property in order to define checkstyle version 92192981f [Enrico Olivelli] Skip CPP tests when using -DskipTests b816e563e [Enrico Olivelli] Add cppunit for Travis faf37bb37 [Enrico Olivelli] ZOOKEEPER-3362 Create a simple checkstyle file
* ZOOKEEPER-2609: Add TTL Node APIs to C clientBalazs Meszaros2019-04-096-100/+502
| | | | | | | | | | | | | | | | | | | | | * added zoo_acreate_ttl, zoo_acreate2_ttl, zoo_create_ttl, zoo_create2_ttl methods, where ttl (int64_t) can be defined. * renamed flags to mode in zoo_*create* because it is not a bitmask on the server side. * added Create mode contants to match CreateMode.java enum fields. ZOO_SEQUENCE is kept for backward compatibility. ZOO_EPHEMERAL|ZOO_SEQUENCE is still usable, but ZOO_EPHEMERAL_SEQUENTIAL is preferred. * added / fixed doxygen comments for create methods. * modified tests/zkServer.sh to pass system properties to ZK. * added +t, +c shell commands to zookeeper cli. * added a test case for create ttl. Author: Balazs Meszaros <balazs.meszaros@cloudera.com> Reviewers: andor@apache.org Closes #891 from meszibalu/my/ttl
* ZOOKEEPER-3302: ZooKeeper C client does not compile on Fedora 29Enrico Olivelli2019-04-091-10/+33
| | | | | | | | | | | | | | | | | Use a safer value as limit for strncpy, taking into account the NULL terminator (see -Werror=stringop-truncation) See https://issues.apache.org/jira/browse/ZOOKEEPER-3302 for details about the error Author: Enrico Olivelli <eolivelli@apache.org> Reviewers: andor@apache.org Closes #846 from eolivelli/fix/ZOOKEEPER-3302-build-fedora-29 and squashes the following commits: 768ee50f6 [Enrico Olivelli] add comment 21cb65c1c [Enrico Olivelli] Drop debug 0c909e336 [Enrico Olivelli] ZOOKEEPER-3302 ZooKeeper C client does not compile on Fedora 29 2bb205f8d [Enrico Olivelli] ZOOKEEPER-3302 ZooKeeper C client does not compile on Fedora 29
* ZOOKEEPER-3122: Rename zookeeper-server to zookeeperNorbert Kalmar2019-04-031-1/+1
| | | | | | | | | | | To stay backward compatible and make dependency management less confusing, everyone using ZooKeeper can stay on "zookeeper" as artifactid. When java client is separated from server code, we can think on the module names to make sense. Author: Norbert Kalmar <nkalmar@yahoo.com> Reviewers: andor@apache.org Closes #890 from nkalmar/ZK-3122
* ZOOKEEPER-3122: Only deploy zk-server and zk-jute to maven repoEnrico Olivelli2019-04-031-1/+11
| | | | | | | | | | | | | | | | | Skip Maven deploy plugin for modules which are not to be deployed: - assembly - clients (C binding) - documentation - recipes Author: Enrico Olivelli <eolivelli@apache.org> Reviewers: andor@apache.org Closes #889 from eolivelli/fix/ZOOKEEPER-3122-fix-deploy (cherry picked from commit 57296927ec268930a6c2704104849d4d7dcb6721) Signed-off-by: Andor Molnar <andor@apache.org>
View Lorry Controller Status