summaryrefslogtreecommitdiff
path: root/platform/android/tests
diff options
context:
space:
mode:
authorTobrun <tobrun.van.nuland@gmail.com>2016-08-17 08:53:33 +0200
committerGitHub <noreply@github.com>2016-08-17 08:53:33 +0200
commitfff0ebaa1754aabd9584c994624b7b948bec95af (patch)
tree745006b1a761856e6651738cd7d26e0e08c6cbbf /platform/android/tests
parent0062cf2f949fe1b0080e2de45aaf0b3d5e028bf3 (diff)
downloadqtlocation-mapboxgl-fff0ebaa1754aabd9584c994624b7b948bec95af.tar.gz
5859 generate sanity tests (#5870)
* [android] #5859 - generate espresso tests [android] #5859 - cleanup xml files, use all the same resource id for map view [android] #5859 - update scripts [android] #5859 - update Activity to match mapboxMap naming [android] #5859 - make SurfaceViewMediaControlActivity compatible [android] #5859 - make MarkerViewScaleActivity test generation compatible [android] #5859 - make PolygonActivity test generation compatible [android] #5859 - integration within make file [android] #5859 - made CameraPostion generation safe [android] #5859 - generate sanity test [android] #5859 - add generated test cases [android] #5859 - add documentation * [android] #5859 - remove generated files, introduce gen package, added gitignore to package * [android] #5859 - regenerated test code * [android] #5859 - added AWS gradle plugin + bitrise configuration * [android] #5859 - fixes running locally without having the properties configured * [android] #6010 - remove python from android sdk * [android] #5859 - update docs about aws gradle plugin
Diffstat (limited to 'platform/android/tests')
-rw-r--r--platform/android/tests/docs/ACTIVITY_SANITY_TEST.md17
-rw-r--r--platform/android/tests/docs/UI_TESTS.md89
-rw-r--r--platform/android/tests/scripts/devicefarm.py132
3 files changed, 20 insertions, 218 deletions
diff --git a/platform/android/tests/docs/ACTIVITY_SANITY_TEST.md b/platform/android/tests/docs/ACTIVITY_SANITY_TEST.md
new file mode 100644
index 0000000000..14e4335111
--- /dev/null
+++ b/platform/android/tests/docs/ACTIVITY_SANITY_TEST.md
@@ -0,0 +1,17 @@
+# Activity Sanity Tests
+
+This type of test checks if an Activity doesn't crash at start up. For this project this type of test is usefull since a lot of core crashes occurs at this point and are ideal to test regressions. The only run for 1 sec each so there are fast and fairly reliable. We are currently generating the test stack from every Activity found in the following package:
+
+> com.mapbox.mapboxsdk.testapp
+
+The generation is done by executing:
+
+> make android-generate-test
+
+This command underneath executes the following js file found in:
+
+> node scripts/android-generate-test
+
+and uses the following ejs file as a template.
+
+>platform/android/MapboxGLAndroidSDKTestApp/src/androidTest/java/com/mapbox/mapboxsdk/activity/activity.junit.ejs
diff --git a/platform/android/tests/docs/UI_TESTS.md b/platform/android/tests/docs/UI_TESTS.md
index 4d7addea0d..9341cf1e1e 100644
--- a/platform/android/tests/docs/UI_TESTS.md
+++ b/platform/android/tests/docs/UI_TESTS.md
@@ -59,92 +59,9 @@ You can generate JaCoCo reports from espresso tests by
- running the gradle task `createMockDebugCoverageReport` when executing tests.
## Running Espresso test automatically on AWS Device Farm
-To automatically execute Espresso tests as part of our CI build, we have created a Python script called [`devicefarm.py`](https://github.com/mapbox/mapbox-gl-native/blob/master/platform/android/tests/scripts/devicefarm.py).
-
-This script is responsible for:
- - uploading an APK + test APK
- - scheduling tests
- - exiting with a return code
- - 0 -> all tests have passed
- - 1 otherwise
-
-### Requirements
-
- * [Boto 3](http://boto3.readthedocs.org)
- * [Requests](http://www.python-requests.org)
-
-### Running the script
-
- A sample run would be as follows:
-
- ```
- $ python devicefarm.py \
- --project-arn "arn:aws:devicefarm:us-west-2:XXXXX" \
- --device-pool-arn "arn:aws:devicefarm:us-west-2::devicepool:YYYYY" \
- --app-apk-path app/build/outputs/apk/app-debug-unaligned.apk \
- --test-apk-path app/build/outputs/apk/app-debug-androidTest-unaligned.apk
- ```
-
- Where you need to insert your actual project and device ARNs. We follow Boto 3
- conventions to to [set up the AWS credentials](https://github.com/boto/boto3#quick-start).
-
- You can build the `app-debug-androidTest-unaligned.apk` package with Gradle:
-
- ```
- ./gradlew assembleAndroidTest
- ```
-
- To run tests locally, you can use `./gradlew assemble` to build the app APK, and
- `./gradlew test --continue` to run unit tests. Finally, `./gradlew connectedAndroidTest`
- will run the Espresso tests in a local device.
-
- A sample output would be as follows:
-
- ```
- Starting upload: ANDROID_APP
- Uploading: ../app/build/outputs/apk/app-debug-unaligned.apk
- Checking if the upload succeeded.
- Upload not ready (status is INITIALIZED), waiting for 5 seconds.
- Starting upload: INSTRUMENTATION_TEST_PACKAGE
- Uploading: ../app/build/outputs/apk/app-debug-androidTest-unaligned.apk
- Checking if the upload succeeded.
- Upload not ready (status is INITIALIZED), waiting for 5 seconds.
- Scheduling a run.
- Checking if the run succeeded.
- Run not completed (status is SCHEDULING), waiting for 60 seconds.
- Run not completed (status is RUNNING), waiting for 60 seconds.
- Run not completed (status is RUNNING), waiting for 60 seconds.
- Run not completed (status is RUNNING), waiting for 60 seconds.
- Run not completed (status is RUNNING), waiting for 60 seconds.
- Run not completed (status is RUNNING), waiting for 60 seconds.
- Run completed: PASSED
- ```
-
-### Available commands
-
- You can use the `--help` command to get a list of all available options:
-
- ```
- $ python devicefarm.py --help
- usage: Device Farm Runner [-h] [--project-arn PROJECT_ARN]
- [--device-pool-arn DEVICE_POOL_ARN]
- [--app-apk-path APP_APK_PATH]
- [--test-apk-path TEST_APK_PATH]
-
- Runs the Espresso tests on AWS Device Farm.
-
- optional arguments:
- -h, --help show this help message and exit
- --project-arn PROJECT_ARN
- The project ARN (Amazon Resource Name) (default: None)
- --device-pool-arn DEVICE_POOL_ARN
- The device pool ARN (Amazon Resource Name) (default:
- None)
- --app-apk-path APP_APK_PATH
- Path to the app APK (default: None)
- --test-apk-path TEST_APK_PATH
- Path to the tests APK (default: None)
- ```
+To run tests on AWS device farm you need to execute `./gradlew devicefarmUpload`.
+You can configure the different steps in the testapp `build.gradle`.
+AWS credentials are found in bitrise.
diff --git a/platform/android/tests/scripts/devicefarm.py b/platform/android/tests/scripts/devicefarm.py
deleted file mode 100644
index 6c0a337403..0000000000
--- a/platform/android/tests/scripts/devicefarm.py
+++ /dev/null
@@ -1,132 +0,0 @@
-'''
-Uploads an APK and a test APK to AWS Device Farm and runs the Espresso tests.
-Exit code is 0 if all tests pass, 1 otherwise. See README.md for details.
-'''
-
-from time import sleep
-import argparse
-import boto3
-import requests
-
-'''
-Parser
-'''
-
-parser = argparse.ArgumentParser(
- prog='Device Farm Runner',
- description='Runs the Espresso tests on AWS Device Farm.',
- formatter_class=argparse.ArgumentDefaultsHelpFormatter,
- add_help=True)
-
-parser.add_argument('--project-arn',
- type=str, help='The project ARN (Amazon Resource Name)')
-parser.add_argument('--device-pool-arn',
- type=str, help='The device pool ARN (Amazon Resource Name)')
-parser.add_argument('--app-apk-path',
- type=str, help='Path to the app APK')
-parser.add_argument('--test-apk-path',
- type=str, help='Path to the tests APK')
-
-args = vars(parser.parse_args())
-project_arn = args.get('project_arn')
-device_pool_arn = args.get('device_pool_arn')
-app_apk_path = args.get('app_apk_path')
-test_apk_path = args.get('test_apk_path')
-
-# Check
-if not project_arn or not device_pool_arn:
- raise Exception('You need to set both the project and the device pool ARN.')
-elif not app_apk_path or not test_apk_path:
- raise Exception('You need to set both the app and test APK path.')
-
-'''
-The AWS Device Farm client
-'''
-
-client = boto3.client('devicefarm', region_name='us-west-2')
-
-'''
-Methods
-'''
-
-def upload_apk(apk_name, apk_path, apk_type):
- print 'Starting upload: %s' % apk_type
- result = client.create_upload(
- projectArn=project_arn, name=apk_name, type=apk_type)
- presigned_url = result.get('upload').get('url')
- upload_arn = result.get('upload').get('arn')
-
- # PUT the file content and wait
- put_file(apk_path=apk_path, presigned_url=presigned_url)
- wait_for_upload(upload_arn=upload_arn)
- return upload_arn
-
-def put_file(apk_path, presigned_url):
- print 'Uploading: %s' % apk_path
- with open(apk_path, 'rb') as f:
- data = f.read()
- result = requests.put(presigned_url, data=data)
- if result.status_code != 200:
- raise Exception(
- 'PUT failed with status code: %s' % result.status_code)
-
-def wait_for_upload(upload_arn):
- print 'Checking if the upload succeeded.'
- succeeded = False
- while not succeeded:
- result = client.get_upload(arn=upload_arn)
- status = result.get('upload').get('status')
- succeeded = (status == 'SUCCEEDED')
- if status == 'FAILED':
- raise Exception('Upload failed.')
- elif not succeeded:
- print 'Upload is not ready (status is %s), waiting for 5 seconds.' % status
- sleep(5)
-
-def schedule_run(app_arn, test_arn):
- print 'Scheduling a run.'
- result = client.schedule_run(
- projectArn=project_arn,
- appArn=app_arn,
- devicePoolArn=device_pool_arn,
- name='automated_run',
- test={ 'type': 'INSTRUMENTATION', 'testPackageArn': test_arn})
- run_arn = result.get('run').get('arn')
- return_code = wait_for_run(run_arn=run_arn)
- return return_code
-
-def wait_for_run(run_arn):
- print 'Checking if the run succeeded.'
- return_code = ''
- succeeded = False
- while not succeeded:
- result = client.get_run(arn=run_arn)
- status = result.get('run').get('status')
- return_code = result.get('run').get('result')
- succeeded = (status == 'COMPLETED')
- if not succeeded:
- print 'Run not completed (status is %s), waiting for 60 seconds.' % status
- sleep(60)
- return return_code
-
-'''
-Main flow
-'''
-
-# 1. Upload the app APK
-app_arn = upload_apk(
- apk_name='app-debug-unaligned.apk',
- apk_path=app_apk_path,
- apk_type='ANDROID_APP')
-
-# 2. Upload the test APK
-test_arn = upload_apk(
- apk_name='app-debug-androidTest-unaligned.apk',
- apk_path=test_apk_path,
- apk_type='INSTRUMENTATION_TEST_PACKAGE')
-
-# 3. Schedule the run
-return_code = schedule_run(app_arn=app_arn, test_arn=test_arn)
-exit_code = 0 if return_code == 'PASSED' else 1
-print 'Run completed: %s' % return_code
-exit(exit_code) \ No newline at end of file