delta/zookeeper.git/zookeeper-client/zookeeper-client-c/tests/zkServer.sh, branch master git.apache.org: repos/asf/zookeeper.git ZOOKEEPER-3561: Generalize target authentication scheme for ZooKeeper authentication enforcement. 2020-10-21T02:41:38+00:00 Mohammad Arshad arshad@apache.org 2020-10-21T02:41:38+00:00 1af3dcc633d4829864da74ca6b030428448fcc16 Added enforce.auth.enabled and enforce.auth.scheme to enforce any authentication scheme. Author: Mohammad Arshad <arshad@apache.org> Reviewers: Michael Han <hanm@apache.org>, Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@gmail.com>, Andor Molnár <andor@apache.org> Closes #1500 from arshadmohammad/ZOOKEEPER-3561-master 
Added enforce.auth.enabled and enforce.auth.scheme to enforce any authentication scheme.

Author: Mohammad Arshad <arshad@apache.org>

Reviewers: Michael Han <hanm@apache.org>, Damien Diederen <dd@crosstwine.com>, Enrico Olivelli <eolivelli@gmail.com>, Andor Molnár <andor@apache.org>

Closes #1500 from arshadmohammad/ZOOKEEPER-3561-master
ZOOKEEPER-1112: Add (Cyrus) SASL authentication support to C client library 2020-01-22T11:53:55+00:00 Damien Diederen dd@crosstwine.com 2020-01-22T11:53:55+00:00 c234848caef0cc920f19befd1b0b14251b17da92 This is a "respin" of https://github.com/apache/zookeeper/pull/1054, which I withdrew due to some annoying shortcomings. This changeset allows C clients to use SASL to authenticate with the ZooKeeper server. It is loosely based on patches #1 and #2 by Tom Klonikowski, at https://reviews.apache.org/r/2252/, but the result has been extensively reworked to follow the semantics of the Java client: * No SASL operations are exposed through the API; * The configuration is provided, and stored, at "handle init time"; * SASL authentication is automatically performed after each (re)connect. It introduces an optional dependency on the Cyrus SASL library, which can either be autodetected (default) or configured using the `--without-sasl`/`--with-sasl[=DIR]` flags, or -DWITH_CYRUS_SASL for CMake/Windows. `TestServerRequireClientSASLAuth.cc` has been renamed to `TestSASLAuth.cc`, and a test has been added which successfully (re)authenticates using the `DIGEST-MD5` mechanism. The code has also been used to successfully authenticate clients via `GSSAPI`/Kerberos. This commit also adds SASL support to the `cli.c` client. Co-authored-by: Tom Klonikowski <klonik_tinformatik.haw-hamburg.de> Author: Damien Diederen <dd@crosstwine.com> Reviewers: Mate Szalay-Beko <szalay.beko.mate@gmail.com>, Norbert Kalmar <nkalmar@apache.org> Closes #1134 from ztzg/ZOOKEEPER-1112-c-client-sasl-support-v2 
This is a "respin" of https://github.com/apache/zookeeper/pull/1054, which I withdrew due to some annoying shortcomings.

This changeset allows C clients to use SASL to authenticate with the ZooKeeper server.  It is loosely based on patches #1 and #2 by Tom Klonikowski, at https://reviews.apache.org/r/2252/, but the result has been extensively reworked to follow the semantics of the Java client:

  * No SASL operations are exposed through the API;

  * The configuration is provided, and stored, at "handle init time";

  * SASL authentication is automatically performed after each (re)connect.

It introduces an optional dependency on the Cyrus SASL library, which can either be autodetected (default) or configured using the `--without-sasl`/`--with-sasl[=DIR]` flags, or -DWITH_CYRUS_SASL for CMake/Windows.

`TestServerRequireClientSASLAuth.cc` has been renamed to `TestSASLAuth.cc`, and a test has been added which successfully (re)authenticates using the `DIGEST-MD5` mechanism.  The code has also been used to successfully authenticate clients via `GSSAPI`/Kerberos.

This commit also adds SASL support to the `cli.c` client.

Co-authored-by: Tom Klonikowski <klonik_tinformatik.haw-hamburg.de>

Author: Damien Diederen <dd@crosstwine.com>

Reviewers: Mate Szalay-Beko <szalay.beko.mate@gmail.com>, Norbert Kalmar <nkalmar@apache.org>

Closes #1134 from ztzg/ZOOKEEPER-1112-c-client-sasl-support-v2
ZOOKEEPER-2122: add SSL support for C-client 2019-11-21T08:54:01+00:00 Mate Szalay-Beko szalay.beko.mate@gmail.com 2019-11-21T08:54:01+00:00 dded710d07892c6b42380267cd1bb896b852a7ee This PR is based on the works of Asnish Amarnath and Suhas Dantkale. Most of the kudos should go to them and those who were reviewing all the previous PRs. **The PR includes the following changes from PR#639:** - OPENSSL 1.1.1 version support in C-client **The PR includes the following changes from PR#990:** - also supporting OPENSSL 1.0.2 - SSL connection on non-blocking socket is handled correctly - Support of Certificate Chains - Fix Memory leaks - Dynamically generated test certificates **The following new changes were added into the PR:** - fix CMake + VisualStudio2019 build with windows - fix C CLI to compile / work both with windows and linux (I tested them manually) - fix (and simplify) the way how the server is started with C unit tests, so it is compatible with maven build now - the test case `testReadOnly` was failing with the previous PR because there was a bug in the C-client code, I fixed that - I also added new test case: `testReadOnlyWithSSL` **Testing this PR on linux:** ``` git clean -xdf # compile ZooKeeper server plus the C-client code mvn clean install -DskipTests -Pfull-build # compile and execute C-client unit tests cd zookeeper-client/ mvn clean install -Pfull-build ``` **Compile the code on windows (only cmake is supported):** - download C-Make:  https://cmake.org/download/ - Install community edition of Visual Studio: https://visualstudio.microsoft.com/downloads/ - Download OpenSSL (e.g. 1.0.2): https://slproweb.com/products/Win32OpenSSL.html (e.g. install it to `c:\OpenSSL-Win64`) - compile the java code using: `mvn clean install -DskipTests` - go to the Client folder: `cd zookeeper-client\zookeeper-client-c` - configure the project:  `cmake . -D WITH_OPENSSL=c:\OpenSSL-Win64` - build the project: `cmake --build .` **Testing the C-client with SSL manually:** - run the `zookeeper-client/zookeeper-client-c/ssl/gencerts.sh` to generate certificate files (e.g. copy it to an empty folder like `/tmp/ssl/` and start is) - start a ZooKeeper server, using some config file like this one: ``` tickTime=3000 initLimit=10 syncLimit=5 dataDir=/tmp/zkdata secureClientPort=22281 serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ssl.keyStore.location=/tmp/ssl/server.jks ssl.keyStore.password=password ssl.trustStore.location=/tmp/ssl/servertrust.jks ssl.trustStore.password=password ``` - start the command line client (cli.exe on windows, cli_mt or cli_st on linux): `./cli_mt --host localhost:22281 --ssl /tmp/ssl/server.crt,/tmp/ssl/client.crt,/tmp/ssl/clientkey.pem,password` Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com> Author: Mate Szalay-Beko <mszalay@cloudera.com> Reviewers: andor@apache.org Closes #1107 from symat/ZOOKEEPER-2122 and squashes the following commits: 08294ce91 [Mate Szalay-Beko] ZOOKEEPER-2122: update readme + use FQDN in SSL certs during testing 17e504a98 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into ZOOKEEPER-2122 317241d13 [Mate Szalay-Beko] ZOOKEEPER-2122: minor fix in SSL certificates used for testing 6f37b6653 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into HEAD 980914313 [Mate Szalay-Beko] ZOOKEEPER-2122: add SSL support for C-client 
This PR is based on the works of Asnish Amarnath and Suhas Dantkale. Most of the kudos should go to them and those who were reviewing all the previous PRs.

**The PR includes the following changes from PR#639:**
- OPENSSL 1.1.1 version support in C-client

**The PR includes the following changes from PR#990:**
- also supporting OPENSSL 1.0.2
- SSL connection on non-blocking socket is handled correctly
- Support of Certificate Chains
- Fix Memory leaks
- Dynamically generated test certificates

**The following new changes were added into the PR:**
- fix CMake + VisualStudio2019 build with windows
- fix C CLI to compile / work both with windows and linux (I tested them manually)
- fix (and simplify) the way how the server is started with C unit tests, so it is compatible with maven build now
- the test case `testReadOnly` was failing with the previous PR because there was a bug in the C-client code, I fixed that
- I also added new test case: `testReadOnlyWithSSL`

**Testing this PR on linux:**
```
git clean -xdf

# compile ZooKeeper server plus the C-client code
mvn clean install -DskipTests -Pfull-build

# compile and execute C-client unit tests
cd zookeeper-client/
mvn clean install -Pfull-build
```

**Compile the code on windows (only cmake is supported):**
- download C-Make:  https://cmake.org/download/
- Install community edition of Visual Studio: https://visualstudio.microsoft.com/downloads/
- Download OpenSSL (e.g. 1.0.2): https://slproweb.com/products/Win32OpenSSL.html (e.g. install it to `c:\OpenSSL-Win64`)
- compile the java code using: `mvn clean install -DskipTests`
- go to the Client folder: `cd zookeeper-client\zookeeper-client-c`
- configure the project:  `cmake . -D WITH_OPENSSL=c:\OpenSSL-Win64`
- build the project: `cmake --build .`

**Testing the C-client with SSL manually:**
- run the `zookeeper-client/zookeeper-client-c/ssl/gencerts.sh` to generate certificate files (e.g. copy it to an empty folder like `/tmp/ssl/` and start is)
- start a ZooKeeper server, using some config file like this one:
```
tickTime=3000
initLimit=10
syncLimit=5
dataDir=/tmp/zkdata

secureClientPort=22281
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.keyStore.location=/tmp/ssl/server.jks
ssl.keyStore.password=password
ssl.trustStore.location=/tmp/ssl/servertrust.jks
ssl.trustStore.password=password
```
- start the command line client (cli.exe on windows, cli_mt or cli_st on linux): `./cli_mt --host localhost:22281 --ssl /tmp/ssl/server.crt,/tmp/ssl/client.crt,/tmp/ssl/clientkey.pem,password`

Author: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Author: Mate Szalay-Beko <mszalay@cloudera.com>

Reviewers: andor@apache.org

Closes #1107 from symat/ZOOKEEPER-2122 and squashes the following commits:

08294ce91 [Mate Szalay-Beko] ZOOKEEPER-2122: update readme + use FQDN in SSL certs during testing
17e504a98 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into ZOOKEEPER-2122
317241d13 [Mate Szalay-Beko] ZOOKEEPER-2122: minor fix in SSL certificates used for testing
6f37b6653 [Mate Szalay-Beko] Merge remote-tracking branch 'apache/master' into HEAD
980914313 [Mate Szalay-Beko] ZOOKEEPER-2122: add SSL support for C-client
ZOOKEEPER-1634: hardening security by teaching server to enforce client authentication. 2019-07-24T16:00:02+00:00 Michael Han lhan@twitter.com 2019-07-24T16:00:02+00:00 513df3da671c9937417bb7d92a55402520ba1292 ## Motivation Previously ZooKeeper server is open to the world as the server does not enforce client authentication - anonymous clients are allowed to establish session with server. This behavior raises a couple of issues from the perspective of performance and security for example: * It is easy to launch a DDoS attack to server, by having a fleet of anonymous clients connect to the ensemble, as each session would consume valuable resources (socket, memory, etc) from server. * It is cumbersome to enforce certain security models with the presence of anonymous clients login - for example as clients are not trusted the root ACL has to be disabled for writes to world, among other configurations an admin has to do to secure a cluster in a multi-tenant environment. So the goal here is to address such issue by hardening ZooKeeper security to provide a more confined access option that user could opt-in, which in addition to the existing ACLs together could lead to more secured / resource optimal ensemble. ## Design Abstract * Introduce a new server side Java property that if set, ZooKeeper server will only accept connections and requests from clients that have authenticated with server via SASL. * Clients that are not configured with SASL authentication, or configured with SASL but fail authentication (i.e. with invalid credential) will not be able to establish a session with server. A typed error code (-124) will be delivered in such case, both Java and C client will close the session with server thereafter, without further attempts on retrying to reconnect. * This feature overrules the server property "zookeeper.allowSaslFailedClients". So even if server is configured to allow clients that fail SASL authentication to login, client will not be able to establish a session with server if this feature is enabled. * Only support SASL because only SASL authentication has the property that no operations will happen until SASL authentication process finished. Thus, the decision of whether to close the session or not can be quickly made on server side upon receiving a client connection request. We could later add support for other auth scheme via add_auth_info if that's desired (if we do, then a session has to be maintained until add_auth_info is invoked.). * As a side benefit, this PR fixes an issue mentioned in ZOOKEEPER-2346 by correctly propagate events from server to client side so a SASL auth failure will manifest as an auth / config failure rather than generic ConnectionLoss event. JIRA: https://issues.apache.org/jira/browse/ZOOKEEPER-1634 The PR also covers (or part of): https://issues.apache.org/jira/browse/ZOOKEEPER-2462 https://issues.apache.org/jira/browse/ZOOKEEPER-2526 https://issues.apache.org/jira/browse/ZOOKEEPER-2346 Author: Michael Han <lhan@twitter.com> Reviewers: Enrico Olivelli <eolivelli@apache.org>, Andor Molnar <andor@apache.org> Closes #118 from hanm/ZOOKEEPER-1634 
## Motivation
Previously ZooKeeper server is open to the world as the server does not enforce client authentication - anonymous clients are allowed to establish session with server. This behavior raises a couple of issues from the perspective of performance and security for example:
* It is easy to launch a DDoS attack to server, by having a fleet of anonymous clients connect to the ensemble, as each session would consume valuable resources (socket, memory, etc) from server.
* It is cumbersome to enforce certain security models with the presence of anonymous clients login - for example as clients are not trusted the root ACL has to be disabled for writes to world, among other configurations an admin has to do to secure a cluster in a multi-tenant environment.

So the goal here is to address such issue by hardening ZooKeeper security to provide a more confined access option that user could opt-in, which in addition to the existing ACLs together could lead to more secured / resource optimal ensemble.

## Design Abstract
* Introduce a new server side Java property that if set, ZooKeeper server will only accept connections and requests from clients that have authenticated with server via SASL.
* Clients that are not configured with SASL authentication, or configured with SASL but fail authentication (i.e. with invalid credential) will not be able to establish a session with server. A typed error code (-124) will be delivered in such case, both Java and C client will close the session with server thereafter, without further attempts on retrying to reconnect.
* This feature overrules the server property "zookeeper.allowSaslFailedClients". So even if server is configured to allow clients that fail SASL authentication to login, client will not be able to establish a session with server if this feature is enabled.
* Only support SASL because only SASL authentication has the property that no operations will happen until SASL authentication process finished. Thus, the decision of whether to close the session or not can be quickly made on server side upon receiving a client connection request. We could later add support for other auth scheme via add_auth_info if that's desired (if we do, then a session has to be maintained until add_auth_info is invoked.).
* As a side benefit, this PR fixes an issue mentioned in ZOOKEEPER-2346 by correctly propagate events from server to client side so a SASL auth failure will manifest as an auth / config failure rather than generic ConnectionLoss event.

JIRA: https://issues.apache.org/jira/browse/ZOOKEEPER-1634
The PR also covers (or part of):
https://issues.apache.org/jira/browse/ZOOKEEPER-2462
https://issues.apache.org/jira/browse/ZOOKEEPER-2526
https://issues.apache.org/jira/browse/ZOOKEEPER-2346

Author: Michael Han <lhan@twitter.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Andor Molnar <andor@apache.org>

Closes #118 from hanm/ZOOKEEPER-1634
ZOOKEEPER-2609: Add TTL Node APIs to C client 2019-04-09T09:36:55+00:00 Balazs Meszaros balazs.meszaros@cloudera.com 2019-04-09T09:36:55+00:00 3b93f7671464bef1824288b13ee66dec547bb97e * added zoo_acreate_ttl, zoo_acreate2_ttl, zoo_create_ttl, zoo_create2_ttl methods, where ttl (int64_t) can be defined. * renamed flags to mode in zoo_*create* because it is not a bitmask on the server side. * added Create mode contants to match CreateMode.java enum fields. ZOO_SEQUENCE is kept for backward compatibility. ZOO_EPHEMERAL|ZOO_SEQUENCE is still usable, but ZOO_EPHEMERAL_SEQUENTIAL is preferred. * added / fixed doxygen comments for create methods. * modified tests/zkServer.sh to pass system properties to ZK. * added +t, +c shell commands to zookeeper cli. * added a test case for create ttl. Author: Balazs Meszaros <balazs.meszaros@cloudera.com> Reviewers: andor@apache.org Closes #891 from meszibalu/my/ttl 
* added zoo_acreate_ttl, zoo_acreate2_ttl, zoo_create_ttl,
  zoo_create2_ttl methods, where ttl (int64_t) can be defined.
* renamed flags to mode in zoo_*create* because it is not a bitmask on the
  server side.
* added Create mode contants to match CreateMode.java enum fields.
  ZOO_SEQUENCE is kept for backward compatibility.
  ZOO_EPHEMERAL|ZOO_SEQUENCE is still usable, but ZOO_EPHEMERAL_SEQUENTIAL
  is preferred.
* added / fixed doxygen comments for create methods.
* modified tests/zkServer.sh to pass system properties to ZK.
* added +t, +c shell commands to zookeeper cli.
* added a test case for create ttl.

Author: Balazs Meszaros <balazs.meszaros@cloudera.com>

Reviewers: andor@apache.org

Closes #891 from meszibalu/my/ttl
ZOOKEEPER-3029: add pom.xml for jute, client and server 2018-12-20T06:24:17+00:00 Norbert Kalmar nkalmar@yahoo.com 2018-12-20T06:24:17+00:00 05b774a1b05374618300f657c9c91b0d5c6ddf71 Work In Progress Creating the maven build for zookeeper-server, zookeeper-jute and zookeeper-client (which only contains C client right now, and it hasn't been implemented yet - maven is not calling make or autoconf) Author: Norbert Kalmar <nkalmar@yahoo.com> Reviewers: andor@apache.org Closes #708 from nkalmar/ZOOKEEPER-3029r and squashes the following commits: 4b1354f41 [Norbert Kalmar] ZOOKEEPER-3029 - add java clover coverage 9c4591ce5 [Norbert Kalmar] ZOOKEEPER-3029 - fix pom issues and clover classpath 5b9350abd [Norbert Kalmar] ZOOKEEPER-3029 - fix C maven test classpath issues 69a2c51b6 [Norbert Kalmar] ZOOKEEPER-3029 - change git-wip-us git to gitbox daf273a7e [Norbert Kalmar] ZOOKEEPER-3029 - remove build.xml from c-client, use maven e328cb812 [Norbert Kalmar] ZOOKEEPER-3029 - fix cppunit tests 5133abe46 [Norbert Kalmar] ZOOKEEPER-3029 - fix portAssigment bug with maven eb7b4abfb [Norbert Kalmar] ZOOKEEPER-3029 - fix jute dependency import d786b90a2 [Norbert Kalmar] ZOOKEEPER-3029 - Build C library and fix tests 438cd43de [Norbert Kalmar] ZOOKEEPER-3029 - add pom.xml for jute, client and server 
Work In Progress
Creating the maven build for zookeeper-server, zookeeper-jute and zookeeper-client (which only contains C client right now, and it hasn't been implemented yet - maven is not calling make or autoconf)

Author: Norbert Kalmar <nkalmar@yahoo.com>

Reviewers: andor@apache.org

Closes #708 from nkalmar/ZOOKEEPER-3029r and squashes the following commits:

4b1354f41 [Norbert Kalmar] ZOOKEEPER-3029 - add java clover coverage
9c4591ce5 [Norbert Kalmar] ZOOKEEPER-3029 - fix pom issues and clover classpath
5b9350abd [Norbert Kalmar] ZOOKEEPER-3029 - fix C maven test classpath issues
69a2c51b6 [Norbert Kalmar] ZOOKEEPER-3029 - change git-wip-us git to gitbox
daf273a7e [Norbert Kalmar] ZOOKEEPER-3029 - remove build.xml from c-client, use maven
e328cb812 [Norbert Kalmar] ZOOKEEPER-3029 - fix cppunit tests
5133abe46 [Norbert Kalmar] ZOOKEEPER-3029 - fix portAssigment bug with maven
eb7b4abfb [Norbert Kalmar] ZOOKEEPER-3029 - fix jute dependency import
d786b90a2 [Norbert Kalmar] ZOOKEEPER-3029 - Build C library and fix tests
438cd43de [Norbert Kalmar] ZOOKEEPER-3029 - add pom.xml for jute, client and server
ZOOKEEPER-3032: MAVEN MIGRATION - zookeeper-server 2018-10-19T12:39:50+00:00 Norbert Kalmar nkalmar@yahoo.com 2018-10-19T12:39:50+00:00 cb9f303bda9137d1aebe8eff3eab85c8a59f3cdd Separating the java code is not feasible. Moving common and client back to server. Author: Norbert Kalmar <nkalmar@yahoo.com> Reviewers: andor@apache.org Closes #672 from nkalmar/ZOOKEEPER-3032r 
Separating the java code is not feasible. Moving common and client back to server.

Author: Norbert Kalmar <nkalmar@yahoo.com>

Reviewers: andor@apache.org

Closes #672 from nkalmar/ZOOKEEPER-3032r
ZOOKEEPER-3032: MAVEN MIGRATION - move java server, client 2018-10-05T12:25:43+00:00 Norbert Kalmar nkalmar@yahoo.com 2018-10-05T12:25:43+00:00 faa7cec71fddfb959a7d67923acffdb67d93c953 Author: Norbert Kalmar <nkalmar@yahoo.com> Reviewers: hanm@apache.org, andor@apache.org Closes #633 from nkalmar/ZOOKEEPER-3032 and squashes the following commits: 3f9a0eca [Norbert Kalmar] ZOOKEEPER-3032 fix flaky QuorumPeerMainTest.testLeaderElectionWithDisloyalVoter_stillHasMajority 27295ed6 [Norbert Kalmar] ZOOKEEPER-3032 - MAVEN MIGRATION - move java server, client 
Author: Norbert Kalmar <nkalmar@yahoo.com>

Reviewers: hanm@apache.org, andor@apache.org

Closes #633 from nkalmar/ZOOKEEPER-3032 and squashes the following commits:

3f9a0eca [Norbert Kalmar] ZOOKEEPER-3032 fix flaky QuorumPeerMainTest.testLeaderElectionWithDisloyalVoter_stillHasMajority
27295ed6 [Norbert Kalmar] ZOOKEEPER-3032 - MAVEN MIGRATION - move java server, client
ZOOKEEPER-3031: MAVEN MIGRATION - Step 1.4 - move client dir 2018-08-21T05:30:53+00:00 Norbert Kalmar nkalmar@yahoo.com 2018-08-21T05:30:53+00:00 176bd68222f3a3bddac9deb412eee7f538cb01ca Author: Norbert Kalmar <nkalmar@yahoo.com> Reviewers: andor@apache.org Closes #597 from nkalmar/ZOOKEEPER-3031 
Author: Norbert Kalmar <nkalmar@yahoo.com>

Reviewers: andor@apache.org

Closes #597 from nkalmar/ZOOKEEPER-3031