1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
#UI Tests
## Running Espresso tests locally
This test project comes with all the required Android Testing Support Library dependencies
in the Gradle file. Tests are under the `app/src/androidTest` folder.
Note that before running your tests, you might want to turn off animations on your test device.
It's a known issue that leaving system animations turned on in a test device
(window animation scale, transition animation scale, animator duration scale)
might cause unexpected results, or may lead tests to fail.
To create a new run configuration:
* Click on Run -> Edit Configurations...
* Click on the plus sign and then on "Android Tests"
* Give a name to the configuration, e.g. `TestAppTests`
* Choose the `MapboxGLAndroidSDKTestApp` module
* Choose `android.support.test.runner.AndroidJUnitRunner` as the instrumentation runner
* Click OK to save the new configuration
You can now run this configuration from the main toolbar dropdown menu.
## Running Espresso tests on AWS Device Farm
On a terminal, within `mapbox-gl-native/android/java`,
run the tests (`cC` stands for `connectedCheck`):
```
$ ./gradlew cC -p MapboxGLAndroidSDKTestApp
```
Then:
* Go to your AWS Console and choose Device Farm.
* Create a new project, e.g. `MapboxGLAndroidSDKTestApp`
* On step 1, upload the APK in `mapbox-gl-native/platform/android/MapboxGLAndroidSDKTestApp/build/outputs/apk/MapboxGLAndroidSDKTestApp-debug-unaligned.apk`
* On step 2, choose Instrumentation, test filter is `com.mapbox.mapboxgl.testapp.MainActivityTest` and upload the APK in `mapbox-gl-native/platform/android/MapboxGLAndroidSDKTestApp/build/outputs/apk/MapboxGLAndroidSDKTestApp-debug-androidTest-unaligned.apk`
* On step 3, choose a device pool. E.g. Top Devices
* On step 4, customize your device state (if needed)
* Finally, confirm the configuration and run the tests.
On Step 2, you can also separate by commas different classes: `com.mapbox.mapboxgl.testapp.MainActivityTest,com.mapbox.mapboxgl.testapp.MainActivityScreenTest`
If you have no tests for your app, or want to test some random user behavior,
you can just choose "Built-in: Fuzz" in step 2.
### Code coverage
You can generate JaCoCo reports from espresso tests by
- adding this to build.gradle:
```java
buildTypes {
debug {
// Run code coverage reports by default on debug builds.
testCoverageEnabled = true
}
}
```
- running the gradle task `createMockDebugCoverageReport` when executing tests.
## Running Espresso test automatically on AWS Device Farm
To automatically execute Espresso tests as part of our CI build, we have created a Python script called [`devicefarm.py`](https://github.com/mapbox/mapbox-gl-native/blob/master/platform/android/tests/scripts/devicefarm.py).
This script is responsible for:
- uploading an APK + test APK
- scheduling tests
- exiting with a return code
- 0 -> all tests have passed
- 1 otherwise
### Requirements
* [Boto 3](http://boto3.readthedocs.org)
* [Requests](http://www.python-requests.org)
### Running the script
A sample run would be as follows:
```
$ python devicefarm.py \
--project-arn "arn:aws:devicefarm:us-west-2:XXXXX" \
--device-pool-arn "arn:aws:devicefarm:us-west-2::devicepool:YYYYY" \
--app-apk-path app/build/outputs/apk/app-debug-unaligned.apk \
--test-apk-path app/build/outputs/apk/app-debug-androidTest-unaligned.apk
```
Where you need to insert your actual project and device ARNs. We follow Boto 3
conventions to to [set up the AWS credentials](https://github.com/boto/boto3#quick-start).
You can build the `app-debug-androidTest-unaligned.apk` package with Gradle:
```
./gradlew assembleAndroidTest
```
To run tests locally, you can use `./gradlew assemble` to build the app APK, and
`./gradlew test --continue` to run unit tests. Finally, `./gradlew connectedAndroidTest`
will run the Espresso tests in a local device.
A sample output would be as follows:
```
Starting upload: ANDROID_APP
Uploading: ../app/build/outputs/apk/app-debug-unaligned.apk
Checking if the upload succeeded.
Upload not ready (status is INITIALIZED), waiting for 5 seconds.
Starting upload: INSTRUMENTATION_TEST_PACKAGE
Uploading: ../app/build/outputs/apk/app-debug-androidTest-unaligned.apk
Checking if the upload succeeded.
Upload not ready (status is INITIALIZED), waiting for 5 seconds.
Scheduling a run.
Checking if the run succeeded.
Run not completed (status is SCHEDULING), waiting for 60 seconds.
Run not completed (status is RUNNING), waiting for 60 seconds.
Run not completed (status is RUNNING), waiting for 60 seconds.
Run not completed (status is RUNNING), waiting for 60 seconds.
Run not completed (status is RUNNING), waiting for 60 seconds.
Run not completed (status is RUNNING), waiting for 60 seconds.
Run completed: PASSED
```
### Available commands
You can use the `--help` command to get a list of all available options:
```
$ python devicefarm.py --help
usage: Device Farm Runner [-h] [--project-arn PROJECT_ARN]
[--device-pool-arn DEVICE_POOL_ARN]
[--app-apk-path APP_APK_PATH]
[--test-apk-path TEST_APK_PATH]
Runs the Espresso tests on AWS Device Farm.
optional arguments:
-h, --help show this help message and exit
--project-arn PROJECT_ARN
The project ARN (Amazon Resource Name) (default: None)
--device-pool-arn DEVICE_POOL_ARN
The device pool ARN (Amazon Resource Name) (default:
None)
--app-apk-path APP_APK_PATH
Path to the app APK (default: None)
--test-apk-path TEST_APK_PATH
Path to the tests APK (default: None)
```
|
