Brain Flush

January 10, 2010

Introducing Calculon – a Java DSL for Android Activity testing

Filed under: Mobile Devices, Software Development & Programming — Tags: , , — Matthias @ 1:05 pm

I have been working on a new test library for Google Android lately, a project which I’m very excited about, and which I feel has loads of potential: Calculon, an Android testing library which lets you write functional tests for your Activities using a very concise and natural syntax.

I am a big fan of test-driven development, and I think there’s no arguing about the massive benefits you get from backing your software up with an automated test suite. The support for writing tests in Android, however, is lacking at best. Even though there has been a test runner since the 1.0 days, writing unit or functional tests in Android is far from being a pleasant experience. For me, the two most important things about tests are that they must be easy to write, and more importantly, easy to read.

That being said, here is a classic example of why Android testing sucks. Our test story is:

Assert that a click on ‘button’ launches a new BarActivity.

Without Calculon, what you do is this:

public void whatInHeavensNameDoesThisEvenTest() {
    final Button b = (Button) getActivity().findViewById(R.id.button);
    ActivityMonitor monitor = getInstrumentation().addMonitor(
        BarActivity.class.getCanonicalName(), null, false);
    getInstrumentation().runOnMainSync(new Runnable() {
        public void run() {
            b.performClick();
        }
    });
    getInstrumentation().waitForIdleSync();
    assertTrue(getInstrumentation().checkMonitorHit(monitor, 1));
}

So, I first have to retrieve a reference to the button that’s supposed to launch a new BarActivity. Then I have to create what Android calls an ActivityMonitor, that’s simply an object which just sits there and waits until an Activity is being launched that matches the description it was configured with. Then I have to perform the actual button click, but sorry, not allowed on the instrumentation thread, so I have to runOnMainSync(). We’re not yet there though: we have to sit and wait, because launching a new activity could take a while, so we also have to waitForIdleSync(). The assertion that follows also couldn’t be less expressive.

When I first wrote this test, my single thought was: Where the heck is my story? It’s buried under a heap of ugly boilerplate code.

I don’t care that the test runner cannot perform a view event on the instrumentation thread. I don’t care that it needs to sit and wait for the Activity to launch. I don’t care about ActivityMonitor and its crummy API. I just want to look at the test, and understand what it’s doing.

Let’s reiterate our test story:

Assert that a click on ‘button’ launches a new BarActivity.

With Calculon, this test is written as:

assertThat(R.id.button).click().starts(BarActivity.class);

Oh the beauty. Our test story was a single sentence, so is the test. That’s how it should be, really. There’s nothing worse than seeing someone else’s tests failing on the build server and having to dissect them line by line to figure out what it’s actually testing.

So let’s move on and look at some nifty Calculon tests. And remember: “Calculon never does two takes!”

Testing with Calculon

Calculon is in a very early stage of development, and currently it only supports writing functional Activity tests, using Android’s ActivityInstrumentationTestCase2. Here is what a Calculon test looks like:

public class FooTest extends FunctionalTest<FooActivity> {
    public FooTest() {
        super("com.example", FooActivity.class);
    }

    public void testStuff() {
        ...
    }
}

Nothing special here, really. You just inherit from a Calculon FunctionalTest.

Assertions

At the heart of a Calculon test are assertions. To create a new assertion, you use the assertThat() method:

public void testStuff() {
    // testing against activities
    assertThat()...
    assertThat(getActivity())...
    assertThat(someOtherActivity)...

    // testing against views
    assertThat(R.id.some_button)...
    assertThat(someButton)...

    // of course all Junit assertions work as well
    assertEquals(...)
    assertNotNull(...)
    ...
}

For now, there are two basic kinds of assertions: ActivityAssertion and ViewAssertion. There is also a third kind, ActionAssertion, but it takes a somewhat special role and is discussed at the end.

Activity Assertions

An ActivityAssertion is an assertion on the state of an Activity and is used like this:

public void testStuff() {
    // testing for an orientation
    assertThat().inPortraitMode();

    // testing for views
    assertThat().viewExists(R.id.some_button);

    // testing for input actions
    assertThat().keyDown(KeyEvent.KEYCODE_BACK)...

    // testing for custom predicates
    assertThat().satisfies(new Predicate<Activity>() {
        public boolean check(Activity target) {
            return target.isTaskRoot();
        }
    });
}

View Assertions

A ViewAssertion is basically the same, just that it tests the state and behavior of a view:

public void testStuff() {
    // testing for view state
    assertThat(R.id.button).isVisible();
    assertThat(R.id.button).isGone();

    // testing for input actions
    assertThat(R.id.button).keyDown(KeyEvent.KEYCODE_BACK)...
    assertThat(R.id.button).click()...
    assertThat(R.id.button).longClick()...

    // testing for custom predicates
    assertThat(R.id.button).satisfies(new Predicate<View>() {
        public boolean check(View target) {
            return target.getVisibility() == View.VISIBLE;
        }
    });
}

Action Assertions

This is a special kind of assertion. You always use it when you want to assert that some action (e.g. a click or a key press) does something. You can use it on both activity and view assertions, and it also allows you to delegate an assertion to another object:

public void testStuff() {
    // testing for actions that launch a new activity
    assertThat(R.id.b1).click().starts(BarActivity.class);

    // testing for actions that finish an activity
    assertThat(R.id.b2).keyDown(KeyEvent.KEYCODE_Q).finishesActivity();

    // testing for actions that change something
    assertThat(R.id.b3).click().implies(R.id.b2).isGone();
    assertThat(R.id.b4).click().implies(getActivity()).inLandscapeMode();
    assertThat(R.id.b5).click().implies(new Predicate<Model>() {
        public boolean check(Model target) {
            return target.someAttribute() == 5;
        }
    });
}

There’s more to come!

This is only a very early version. I intend to expand this library significantly over time. Of course it’s open source, so contributions are always welcome. You can fork the project on GitHub, add your changes, and send me a pull request.

Advertisements

March 18, 2008

Using cURL for Testing Web Applications

Filed under: Software Development & Programming — Tags: , , , , , , , , — Matthias @ 11:05 am

This may be old news to some, but although I had heard of using cURL for downloading files off the internet from a command line (as a wget alternative), I didn’t know that it was actually capable of doing much, much more. In fact, cURL is a very good allrounder at doing anything related to transmitting and receiving data using popular protocols such as HTTP(S), (S)FTP, TELNET, LDAP and more. In particular, you can use it to test your Web application with very little effort!

I am currently programming for a Java Servlet based Web application, where I need to assemble HTTP requests on the client side and dissect them on the server side. More precisely, I have a servlet which extracts data from an HTTP multipart request (using Apache Commons FileUpload) and hand it over to the logic for further processing. Of course, I have to write lots of code for checking whether requests are valid and so forth. So, in order to test this code, I want to send “malformed” requests, by which I mean requests that do not carry data expected by the application in its current state. Instead of setting up a complex test environment or even modifying your client code to send bad data, you can simply use cURL.

cURL uses a simple command line interface; I will show you the most important flags. Let’s assume we have a simple RESTful Web application which manages books. We could use cURL to retrieve the list of available books like this:

curl http://localhost:8080/mybookstore/books -v

This will issue an HTTP GET on the specified URL. The -v flag tells cURL to be verbose, which means it will print all sent and received HTTP headers, the payload of the server reply, plus some additional status information to the standard output. The complete output might look something like this:

* About to connect() to localhost port 8080 (#0)
* Trying 127.0.0.1… connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mybookstore/books HTTP/1.1
> User-Agent: curl/7.18.0 (i586-pc-mingw32msvc) libcurl/7.18.0 zlib/1.2.3
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/xml;charset=utf-8
< …
< <books><book author=”David Flanagan”>Java In A Nutshell</book><book author=”Stephen Hawking”>A Brief History Of Time</book>…</books>

The > symbol indicates data going to the server, while < indicates data coming from the server. If that’s not verbose enough for you, try using the --trace-ascii <filename> argument instead. It will log the client/server conversation to the specified file.

So, what about sending data? It’s actually just as easy. Suppose we want to store a new book by supplying its title and author:

curl http://localhost:8080/mybookstore/books -F “title=Wikinomics” -F “author=D. Tapscott, A. D. Williams”

What this does is issueing an HTTP POST to the specified address, submitting the given data using the multipart/form-data MIME type. Now that was easy! If you don’t want to type all the input data on the command line, you can also let cURL read data from a file by prefixing the value with @ followed by the file name, which makes perfect sense when transmitting binary data or long text documents. As to the way how cURL transmits the data, you can also use the -d flag, which will result in the data being POST-ed as an application/x-www-form-urlencoded string, or use the same -d flag in combination with -G to issue a GET request instead and sending the data as a URL parameter (query string).

Another noteworthy aspect about cURL is that you can use it to manage cookies. Let’s assume our book server requires the user to login first, and that the server remembers us by setting a session cookie. Ponder the commands below, where we use cURL first to authenticate with our server, and then add a new book in the same session:

curl http://localhost:8080/mybookstore/login -d “username=john” -d “password=doe” -c cookies.txt
curl http://localhost:8080/mybookstore/books -b cookies.txt -F “title=…” …

In the first line, we tell cURL to use the file cookies.txt as its “cookie jar”, which means that it will store all cookies set by the server in this file. We can leverage the session data stored in that file (for a Servlet container this would be a JSESSIONID) in consecutive requests in order to remain authenticated, by setting the value of the -b flag to the cookie jar. You can also use key/value pairs as an argument to -b, but using an intermediary file is more convenient.

cURL is free software and can be downloaded for a broad range of platforms from the cURL website.

Blog at WordPress.com.