1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
|
# Apache Thrift - integration test suite
This is the cross everything integration test suite for Apache Thrift.
## Run
### A. Using Make
The test can be executed by:
make cross
This starts the [test.py](test.py) script which does the real cross test with
different transports, protocols and languages.
Note that this skips any language that is not built locally. It also skips
tests that are known to be failing. If you need more control over which tests
to run, read following section.
### B. Using test script directly
Alternatively, you can invoke [test.py](test.py) directly. You need to run`make
precross` once before executing it for the first time.
For example, if you changed something in `nodejs` library and need to verify
the patch, you can skip everything except `nodejs` itself and some reference
implementation (currently `cpp` and `java` are recommended) like this:
./configure --without-c_glib --without-erlang --without-lua ...
make precross -j8
test/test.py --server cpp,java --client nodejs
test/test.py --server nodejs --client cpp,java
Another useful flag is --regex. For example, to run all tests that involve
Java TBinaryProtocol:
test/test.py --regex "java.*binary"
## Test case definition file
The cross test cases are defined in [tests.json](tests.json).
The root element is collection of test target definitions.
Each test target definition looks like this:
{
"name": "somelib",
"client": {
"command": ["somelib_client_executable"],
"workdir": "somelib/bin",
"protocols": ["binary"],
"transports": ["buffered"],
"sockets": ["ip"],
},
"server": {
"command": ["somelib_server_executable"],
"workdir": "somelib/bin",
"protocols": ["binary"],
"transports": ["buffered"],
"sockets": ["ip", "ip-ssl"],
}
}
Either client or server definition or both should be present.
Parameters that are common to both `client` and `server` can be put to target
definition root:
{
"name": "somelib",
"workdir": "somelib/bin",
"protocols": ["binary"],
"transports": ["buffered"],
"sockets": ["ip"],
"client": { "command": ["somelib_client_executable"] },
"server": {
"command": ["somelib_server_executable"],
"sockets": ["ip-ssl"]
}
}
For the complete list of supported keys and their effect, see source code
comment at the opt of [crossrunner/collect.py](crossrunner/collect.py).
## List of known failures
Since many cross tests currently fail (mainly due to partial incompatibility
around exception handling), the test script specifically report for "not known
before" failures.
For this purpose, test cases known to (occasionally) fail are listed in
`known_failures_<platform>.json` where `<platform>` matches with python
`platform.system()` string.
Currently, only Linux version is included.
FYI, the file is initially generated by
test/test.py --update-expected-failures=overwrite
after a full test run, then repeatedly
test/test.py --skip-known-failures
test/test.py --update-expected-failures=merge
to update the known failures, run
make fail
## Test executable specification
### Command line parameters
Unit tests for languages are usually located under lib/<lang>/test/
cross language tests according to [ThriftTest.thrift](ThriftTest.thrift) shall be
provided for every language including executables with the following command
line interface:
**Server command line interface:**
$ ./TestServer -h
Allowed options:
-h | --help produce help message
--port=arg (9090) Port number to listen
--domain-socket=arg Unix Domain Socket (e.g. /tmp/ThriftTest.thrift)
--pipe=arg Windows Named Pipe (e.g. MyThriftPipe)
--server-type=arg (simple) type of server, "simple", "thread-pool",
"threaded", or "nonblocking"
--transport=arg (buffered) transport: buffered, framed, http, anonpipe, zlib
--protocol=arg (binary) protocol: binary, compact, header, json
--multiplex Add TMultiplexedProtocol service name "ThriftTest"
--abstract-namespace Create the domain socket in the Abstract Namespace
(no connection with filesystem pathnames)
--ssl Encrypted Transport using SSL
--zlib Wrapped Transport using Zlib
--processor-events processor-events
-n=arg | --workers=arg (=4) Number of thread pools workers. Only valid for
thread-pool server type
**Client command line interface:**
$ ./TestClient -h
Allowed options:
-h | --help produce help message
--host=arg (localhost) Host to connect
--port=arg (9090) Port number to connect
--domain-socket=arg Domain Socket (e.g. /tmp/ThriftTest.thrift),
instead of host and port
--pipe=arg Windows Named Pipe (e.g. MyThriftPipe)
--anon-pipes hRead hWrite Windows Anonymous Pipes pair (handles)
--abstract-namespace Create the domain socket in the Abstract Namespace
(no connection with filesystem pathnames)
--transport=arg (buffered) Transport: buffered, framed, http, evhttp, zlib
--protocol=arg (binary) Protocol: binary, compact, header, json
--multiplex Add TMultiplexedProtocol service name "ThriftTest"
--ssl Encrypted Transport using SSL
--zlib Wrap Transport with Zlib
-n=arg | --testloops=arg (1) Number of Tests
-t=arg | --threads=arg (1) Number of Test threads
If you have executed the **make check** or **make cross** then you will be able to browse
[gen-html/ThriftTest.html](gen-html/ThriftTest.html) with the test documentation.
### Return code
The return code (exit code) shall be 0 on success, or an integer in the range 1 - 255 on errors.
In order to signal failed tests, the return code shall be composed from these bits to indicate
failing tests:
#define TEST_BASETYPES 1 // 0000 0001
#define TEST_STRUCTS 2 // 0000 0010
#define TEST_CONTAINERS 4 // 0000 0100
#define TEST_EXCEPTIONS 8 // 0000 1000
#define TEST_UNKNOWN 64 // 0100 0000 (Failed to prepare environment etc.)
#define TEST_TIMEOUT 128 // 1000 0000
#define TEST_NOTUSED 48 // 0011 0000 (reserved bits)
Tests that have not been executed at all count as errors.
**Example:**
During tests, the test client notices that some of the Struct tests fail.
Furthermore, due to some other problem none of the Exception tests is executed.
Therefore, the test client returns the code `10 = 2 | 8`, indicating the failure
of both test 2 (TEST_STRUCTS) and test 8 (TEST_EXCEPTIONS).
## SSL
Test Keys and Certificates are provided in multiple formats under the following
directory [test/keys](keys)
|
