Automated UI tests on Android

I recently fought the Android emulator a lot to get my UI tests to run automatically during the build of Rabbit Escape, so I thought I’d better write down what I did before I forget.

I already have tests that drive the Android UI (see e.g. SmokeTest.java – it’s not pretty, but it seems reliable, and it catches real problems) – this post is about how to run them automatically during our build.

Note that to run the Android emulator I’m fairly sure you need a windowing environment, so I don’t think this could be moved to a headless build server. If course, you could always fight some kind of framebuffer thing.

Here is the part of our Makefile that launches the tests:

android-smoke-tests:
	@echo ". Running Android smoke tests"
	./build-scripts/android-start-emulator "android-8" "3.2in QVGA (ADP2)"
	./build-scripts/android-test "free" "app-free-debug"
	./build-scripts/android-test "" "app-paid-debug"
	./build-scripts/android-stop-emulator

and here is ./build-scripts/android-start-emulator – it starts up and emulator, waits for it to be ready, and unlocks its screen.:

#!/bin/bash

set -x
set -u
set -e

# Args

TARGET="$1"   # E.g. "android-8"
DEVICE="$2"   # E.g. "3.2in QVGA (ADP2)"

# Setup

ADB="${HOME}/Android/Sdk/platform-tools/adb"
EMULATOR="${HOME}/Android/Sdk/tools/emulator"
ANDROID="${HOME}/Android/Sdk/tools/android"
NAME="rabbitescape-${TARGET}"
TMP="/data/local/tmp"

${ANDROID} create avd \
    --force \
    --name "${NAME}" \
    --target "${TARGET}" \
    --abi "armeabi" \
    --device "${DEVICE}"

# Start the emulator
${EMULATOR} -avd "${NAME}" &

# Wait for the device to boot and unlock it
${ADB} wait-for-device shell < ${TMP}/zero
getprop dev.bootcomplete > ${TMP}/bootcomplete
while cmp ${TMP}/zero ${TMP}/bootcomplete; do
{
    echo -n "."
    sleep 1
    getprop dev.bootcomplete > ${TMP}/bootcomplete
}; done
echo "Booted."
exit
ENDSCRIPT

echo "Waiting 30 secs for us to be really booted"
sleep 30

echo "Unlocking screen"
${ADB} shell "input keyevent 82"

Now here is android-test – it launches the JUnit test code on the running emulator::

#!/bin/bash

set -x
set -u
set -e

PKGSUFFIX="$1"   # E.g. "free"
APKNAME="$2"     # E.g. "app-free-debug"

APPID="net.artificialworlds.rabbitescape${PKGSUFFIX}"
TESTAPPID="net.artificialworlds.rabbitescape${PKGSUFFIX}.test"
APK="rabbit-escape-ui-android/app/build/outputs/apk/${APKNAME}.apk"
TESTAPK="rabbit-escape-ui-android/app/build/outputs/apk/${APKNAME}-androidTest.apk"
ADB="${HOME}/Android/Sdk/platform-tools/adb"
DIR="/data/local/tmp/${APPID}"
TESTDIR="/data/local/tmp/${TESTAPPID}"

function run_test()
{
    TMPFILE=$(mktemp)

    ${ADB} shell am instrument \
        -w \
        -r \
        -e class "$1" \
        "${TESTAPPID}/android.test.InstrumentationTestRunner" \
    | tee ${TMPFILE}

    egrep "OK (.* tests?)" ${TMPFILE}
}

${ADB} push "${APK}" "${DIR}"
${ADB} push "${TESTAPK}" "${TESTDIR}"

${ADB} shell pm install -r "${DIR}"
${ADB} shell pm install -r "${TESTDIR}"

run_test rabbitescape.ui.android.DialogsTest
run_test rabbitescape.ui.android.SmokeTest
run_test rabbitescape.ui.android.TestAndroidConfigUpgradeTo1

And here is android-stop-emulator – it shuts down the emulator:

#!/bin/bash

set -x
set -u

echo -e "auth $(cat ~/.emulator_console_auth_token)\nkill" \
    | telnet localhost 5554

echo "Emulator stopped."

Submitting a package to F-Droid

Here’s what I needed to get a dev environment for F-Droid up and running on Ubuntu 16.10, using F-Droid server version 0.7.0 (commit id 8147f9235), so that I could submit a package for inclusion in the F-Droid repository.

Doing this is apparently the best way to get your own package into the repository, since you can provide a direct merge request for the metadata about your package, making it easy for the maintainers.

References:

Setup

Before you start, manually install the Android SDK at ~/Android/Sdk/ – see Download Android Studio. I installed version 23.0.2, but you will probably have a later one and may need to adjust the version number below.

Note: If you’re only planning to contribute a package I’m fairly certain you don’t need to install the Android SDK at all – you can just use the build server by running ./makebuildserver as I outline below.

Also before you start, if you want to contribute to the server project you should fork the F-Droid server project by going to gitlab.com/fdroid/fdroidserver and clicking Fork. When you’ve done that, the git clone command below will need to change to clone your own fork via SSH, instead of the HTTPS one cloning the main repo that is shown below. Do the same for the F-Droid data project, which holds the information about the packages in F-Droid. It’s the data project where you will want to make changes if you are submitting a package.

Run these commands:

# Prerequisites
sudo apt-get install openjdk-8-jdk subversion git git-svn mercurial bzr virtualbox ruby ruby-dev vagrant python3 python3-paramiko python3-pil python3-pyasn1-modules python3-clint
vagrant plugin install vagrant-cachier
ln -s ~/Android/Sdk/build-tools/23.0.2/aapt ~/Android/Sdk/platform-tools/

# Get the code
cd ~/code
git clone https://gitlab.com/fdroid/fdroidserver.git
git clone https://gitlab.com/fdroid/fdroiddata.git
echo 'export PATH="~/code/fdroidserver:$PATH"' >> ~/.profile
source ~/.profile

# Config
cd fdroiddata
cp ../fdroidserver/examples/config.py ./
chmod 0600 config.py
echo 'sdk_path = "$HOME/Android/Sdk"' >> config.py

# Set up Vagrant build box
cd ../fdroidserver
cp ./examples/makebuildserver.config.py ./
./makebuildserver
# Now wait several hours for this to finish

# Build a package (the F-Droid client) just to check it works
cd ../fdroiddata
mkdir repo
fdroid update --create-key
fdroid readmeta  # Should give no output if it worked
fdroid build --server org.fdroid.fdroid

Make your own package

Below I’m using my own package, Rabbit Escape, as an example. Its Android code is inside rabbit-escape-ui-android/app, whereas many programs will just have it directly in a directory called “app”.

Rabbit Escape also builds non-Android-specific Java and other things during its build, so your package may be simpler.

cd ../fdroiddata
fdroid import --url https://github.com/andybalaam/rabbit-escape --subdir rabbit-escape-ui-android/app

Now edit the new file that was created – in my case it was called metadata/net.artificialworlds.rabbitescape.txt.

I set the following info:

Provides:net.artificialworlds.rabbitescape
Categories:Games
License:GPLv2+
Author Name:Andy Balaam and the Rabbit Escape developers
Author Email:rabbitescape@artificialworlds.net
Web Site:http://artificialworlds.net/rabbit-escape
Source Code:https://github.com/andybalaam/rabbit-escape
Issue Tracker:https://github.com/andybalaam/rabbit-escape/issues

Name:Rabbit Escape
Summary:Lemmings-like puzzle/action game
Description:
140 levels of puzzling action!

blah blah blah
.

Repo Type:git
Repo:https://github.com/andybalaam/rabbit-escape
Binaries:https://github.com/andybalaam/rabbit-escape/releases/download/v%v/rabbit-escape-%v.apk

Build:0.10.1,101
    commit=v0.10.1
    subdir=rabbit-escape-ui-android/app
    gradle=paid
    build=cd ../.. && make android-pre

Auto Update Mode:Version v%v
Update Check Mode:Tags v\d+\.\d+(\.\d+)?
Current Version:0.10.1
Current Version Code:101

For more info, see the F-Droid manual.

And then checked it all worked with:

cd ../fdroiddata
fdroid lint net.artificialworlds.rabbitescape
fdroid readmeta
fdroid checkupdates net.artificialworlds.rabbitescape
fdroid rewritemeta net.artificialworlds.rabbitescape

When I got the version stuff right the checkupdates command printed:

INFO: Processing net.artificialworlds.rabbitescape...
INFO: ...updating to version 0.10.1 (101)
INFO: Finished.

Then I made sure it built OK:

fdroid build --server -v -l net.artificialworlds.rabbitescape

Actually, it didn’t work, and I decided I had to request a new package (sox) be installed in the build machine environment (in the fdroidserver project). The relevant commit is here: 19e372026. Actually though, after discussion with the F-Droid devs we agreed I’d be better off not using sox during the build, so I didn’t need this.

Side note: if you do end up needing to modify the build environment for F-Droid, make sure you delete the fdroiddata/buildserver directory when you re-try your build. That one had me stuck for a few days, with the old environment being used no matter what caches I cleared and vagrant commands I ran.

And now I was ready to request my package be included in F-Droid by committing and pushing the changes I had made to the fdroiddata project to my forked repo, and clicking the Merge Request button in the gitlab UI. My merge request is here: gitlab.com/fdroid/fdroiddata/merge_requests/1965

Resources for year 6 teachers on coding and programming

I have been introducing some year 6 (UK) teachers to coding by showing them how to lay out a simple web page by writing HTML. I promised I would find some links to resources for them, so here it is:

HTML and JavaScript

My examples of how to write HTML are at github.com/andybalaam/html-examples

There are several web sites that allow you to experiment with writing HTML and JavaScript and seeing the results immediately:

I also made some videos about how to make a snowflake animation in both JavaScript and Scratch here: Snowflake Christmas card.

Raspberry Pi

The Raspberry Pi is a cheap education-focussed computer that looks like a piece of circuit board the size of a credit card.

Their educational resources are at: raspberrypi.org/resources.

There are lots of great videos about how to do different things with the Raspberry Pi, including the ones by The Raspberry Pi Guy.

There are also my (boring, but comprehensive) videos teaching you to write a simple game in Python, from a starting point of no programming experience at all: My First Raspberry Pi Game.

Graphical programming

There are several tools and sites for learning programming by dragging and dropping blocks instead of typing code:

  • Scratch – creative, unguided, a bit old-fashined looking but tried-and-trusted
  • code.org – fun, attractive guided tasks featuring Disney characters, Minecraft etc.
  • blockly – guided tasks with good progression
  • codeforlife.education – I’ve not used this, but it looks like it could have potential

Other languages

Setting up a sane Maven project

Today I have spent hours trying to get wrangle Maven into letting me set up a sane Java project. The hardest parts were enforcing warnings-as-errors, and getting Maven to shut up a bit.

Some code that warns

My first problem was writing some Java code that causes the compiler to emit warnings. For some reason I can’t understand, my Java 1.8 compiler was not emitting warnings (even with -Xlint:all) for unused variables, use of deprecated methods, or unknown @SuppressWarnings types (suggestions from SO 1752607).

Instead, I had to use an unnecessary cast:

$ cat src/tast/java/ExampleTest.java
public class ExampleTest {
    public void warn() {
        String fixmePlease = (String)"Hello";
    }
}

Now, finally, I got a warning:

$ javac -Xlint:all src/test/ExampleTest.java
src/test/ExampleTest.java:3: warning: [cast] redundant cast to String
        String s = (String) "Hello!";
                   ^
1 warning

Maven compiler settings for warnings-as-errors

I tried a vast set of combinations of properties like maven.compiler.failOnWarning and maven.compiler.fork (as specified in the maven compiler plugin docs) before giving up on properties. Making a property called maven.compiler.failOnWarning seems to have no effect whatsoever, ever.

So I decided I must try the (very verbose) plugin tag containing a configuration tag, as also specified in the maven compiler plugin docs. After a lot of messing about with flags that seemed incompatible, and Maven silently ignoring things it didn’t understand, I came to a working config.

On the way, I discovered that setting the “fork” property to true is a non-starter, because Maven simply drops the compiler output in that case, meaning you can’t see what is going wrong when it does.

Finally, I had a pom file like this:

cat pom.xml
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>Example</artifactId>
    <packaging>jar</packaging>
    <version>0.0.1</version>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <failOnWarning>true</failOnWarning>
                    <showWarnings>true</showWarnings>
                    <compilerArgs>
                        <arg>-Xlint:all</arg>
                    </compilerArgs>
                </configuration>
            </plugin>
        </plugins>
    </build>
     <dependencies>
         <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <version>4.12</version>
             <scope>test</scope>
         </dependency>
     </dependencies>
</project>

Which manages successfully to tell the compiler to show all warnings, and to fail when it sees one. (Thanks to SO 9192613, among others.)

I dare not change anything, for fear that it will stop working again without any explanation.

Quieting Maven

If you tell Maven to be quiet with -q or -e it will merrily fail the build because of a warning, but not tell you what the warning was.

Maven does not appear to have a command line option to set the log level to display warnings and errors only, but you can force it to do so by setting the environment variable MAVEN_OPTS like this:

MAVEN_OPTS=MAVEN_OPTS=-Dorg.slf4j.simpleLogger.defaultLogLevel=warn mvn clean test-compile

(Thanks to SO 4782089.)

And, with some guesswork (partly based on Configuring Maven) I found that if I put something similar in .mvn/jvm.config I didn’t have to type it every time:

$ cat .mvn/jvm.config 
-Dorg.slf4j.simpleLogger.defaultLogLevel=warn

Failing on warnings, and seeing them!

I don’t know whether to feel triumphant or defeated, but, it works!

$ mvn clean test-compile 
[WARNING] COMPILATION WARNING : 
[WARNING] src/test/java/ExampleTest.java:[3,20] redundant cast to java.lang.String
[ERROR] COMPILATION ERROR : 
[ERROR] src/test/java/ExampleTest.java: warnings found and -Werror specified
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:testCompile (default-testCompile) on project unmatcheddrrepair: Compilation failure
[ERROR] src/test/java/ExampleTest.java: warnings found and -Werror specified
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I wish I could suppress double-reporting of the error, and the extra rubbish at the end (and the super-long absolute paths of each file that push the actual errors off the side of the screen), but at this point, I must try and do what I was trying to do in the first place.

Guess how much I am liking Maven at this point.

Writing a unit test in Elm

Series: Snake in Elm, Elm makes me happy, Elm Basics, Elm Unit Test

I’ve been having fun with Elm programming recently. Elm is a replacement for JavaScript that is pure functional and highly practical.

Here’s how to go from nothing installed at all to writing a unit test that passes, in just over 10 minutes.

The source code is here: github.com/andybalaam/elm-unit-test-example