Brain Flush

May 3, 2009

Introducing Signpost: Easy OAuth for Java and Apache HttpComponents (Android, too)

Filed under: Software Development & Programming — Tags: , , , , , , , — Matthias @ 10:58 am

I would like to announce a project I started, which I hope may be useful to Java (particularly Google Android) application developers who need to communicate with web services that leverage the OAuth protocol for accessing protected resources.

What is Signpost?

Signpost is the easy and intuitive solution for signing HTTP messages in conformance with the OAuth Core 1.0 standard. Signpost has been designed to work in conjunction with the Apache HttpComponents library, a proven, well-established HTTP library for the Java programming language.

Signpost is still in beta stage, which means it may contain bugs. The project is currently hosted at Google Code and can be downloaded, distributed and used under the terms of the Apache License, version 2.

Why another OAuth library for Java?

It was my discontendness with Netflix’s OAuth implementation which ultimately drove me to develop my own solution. My biggest gripes with it were:

  • Its clumsy, overly complicated API
  • Its tendency to do more than what’s actually in the standard (why does it implement an own HTTP layer? OAuth is about message signing, not about message sending)
  • Its limitations resulting from the last point, particularly:
    • Its ignorance towards RESTful web services (a 201 is treated as an error)
    • Its inability to process other HTTP verbs than GET (you must subclass in order to POST or PUT a resource)
    • Its inability to send more complex messages (don’t even try to send multipart requests)

Signpost attempts to avoid these issues as described below (a brief remark: Despite those issues, I highly appreciate Netflix’s work on the original implementation. In fact, Signpost is to some degree based on code from the reference implementation).

Goals of Signpost

Signpost has been designed with several principal goals in mind:

Simplicity

Using Signpost is as simple as it could possibly get — all actions are executed with only a few lines of code. For example, here is how you would sign an HTTP message using Signpost (assuming you have already created the involved HttpClient and OAuthConsumer objects):

        // create an HTTP request to a protected resource
        HttpGet request = new HttpGet("http://example.com/protected");

        // sign the request (consumer is a Signpost OAuthConsumer)
        consumer.sign(request);

        // send the request
        HttpResponse response = httpClient.execute(request);

Signpost exposes a minimalistic API designed for two purposes: Signing HTTP messages and requesting tokens from an OAuth service provider. Everything else is beyond the scope of the OAuth specification, and is thus left to the HTTP messaging layer, where it belongs.

For more exhaustive examples, please refer to GettingStarted.

Unobtrusiveness

Signpost tries to be as unobtrusive as possible. Unlike the OAuth reference implementation for Java, Signpost does not wrap the entire HTTP layer and hides its features from the client. Instead, you simply pass an HttpRequest object to it, and Signpost will sign the message using the credentials it was configured with. This means that all the power and flexibility of the Apache HttpComponents? is still at your fingertips!

Limitations

Simplicity doesn’t come free. Thus, Signpost currently makes certain assumptions to reduce the complexity of both the implementation and the API.

Deviations from the OAuth standard

  • Additional service provider parameters for retrieving request tokens are currently unsupported (cf. section 6.1)
  • Message signing using public key encryption (as per section 9.3) is currently unsupported. Message signing using the PLAINTEXT and HMAC-SHA1 is supported, however.
  • The OAuth standard demands that OAuth request parameters may be put in the URI query string or in the message payload. Signpost will never do that; instead, all OAuth protocol parameters are written to the HTTP Authorization header field. Anything you put there will be overwritten by Signpost.
  • Signpost does not support writing OAuth protocol params to the WWW-Authenticate header field

I believe that even with those restrictions in place, Signpost will work for 99% of its users. Trading in some flexibility only relevant for edge cases was a design decision. If that doesn’t work for your setup, then Signpost is probably not the best choice.

Thread Safety

Signpost is not thread safe and probably will never be. Signpost objects are very lightweight, so you are adviced to create an OAuthConsumer and OAuthProvider for every thread in your application that must send signed HTTP requests.

Google Android

Signpost works flawlessly in conjunction with Android, Google’s software stack for mobile devices. In fact, Signpost has already signed thousands of HTTP requests at this very moment, as it is an integral part of Qype Radar, our geo-sensitive mobile application for Android that finds the best places near you.

Since Android already ships with a recent version of the Apache HttpComponents?, you merely need to add the Signpost JAR to your Android application, and you’re all set.

Advertisements

April 8, 2009

Android In-Sync: Handling concurrent tasks in Google Android

UPDATE 15/12/2009: While the problem described here still applies to the latest version of Android, I discourage anyone from using the code linked here. I’ve bundled a better solution with my Droid-Fu library, which is a little more intrusive, but leads to much leaner code for the application developer.

Today I want to talk a little bit about concurrency in Android applications, and the problems it poses on the developer. If you have used Android on your phone before, it’s likely that you have stumbled upon applications which load data off the internet, or perform other time consuming operations. The problem with time consuming operations is that, well, they consume time, and if they aren’t perfomed concurrently to Android’s user interface thread (the main thread), then the UI will lock up — certainly not a good user experience. So, it’s pretty obvious that on internet phones like those based on Android, highly concurrent applications are more the rule than the exception.

I guess I don’t have to mention that developing concurrent applications is everything but simple. Keeping threads that share data in sync is not a trivial task and prone to errors. What makes it even more difficult in Android is the fact that while your application is loading data, it may suddenly be interrupted by an incoming phone call or because the user decided to flip the screen into a different orientation. You may think that your thread will get paused while the activity that created it is brought to the background (or even gets destroyed). That’s not what happens though: any thread will continue running until it completes, even if your activity or service is not alive anymore. And that’s where the pain starts.

Problem Statement

Consider the following news reader application (just as an example). To keep things simple, the application only supports one operation: Showing the latest news articles on the screen using pagination. When started the first time, the application loads the first 10 articles off the internet in a separate thread and renders a nifty spinner visual to keep the user happy. When the user wants to read the next 10 articles, a button is pressed, the spinner appears again, and the next 10 articles are loaded, etc. pp. We have two separate threads running concurrently in this application: The main UI thread that renders all the article items and the spinner graphic, and the thread that downloads data off the web and posts that data back to the UI thread upon completion, so the UI can update itself accordingly.

Now, what happens if a phone call comes in while we see the spinner graphic? What happens is that Android will pause your application (probably even destroy it) in order to launch the call activity on top of it, but your download thread will continue running in the background. It’s difficult to tell what is going to happen when that thread tries to post its result to an activity that doesn’t even exist anymore, but most probably that application will crash (for example when the download result handler in that activity tries to display a toast or dialog, which will be attached to a window that has already been destroyed). If that thread doesn’t terminate and still holds a reference to the calling activity, it may even produce a memory leak. Long story short, you will definitely want to introduce some mechanism that keeps the communication between activities, services, and any threads they run, in sync.

Solution

Lucky for you, it just so happens that I have written a module that does exactly that. Based on my former work on this problem and an excellent article by Eric Burke (who already presented an almost-working solution), I have come up with a Task class that you can use to dispatch long running operations from your activities and services and which also handles all the inconveniences arising from situations like resuming from an interruption (phone calls and the likes).

And here is how you use it:

public class Concurrency extends Activity implements TaskListener<String> {

    private static final int TASK1 = 0;

    private static final int TASK2 = 1;

    private Task<String> task1, task2;

    private Callable<String> callable1 = new Callable<String>() {

        public String call() throws Exception {
            try {
                System.out.println("task1 starting");
                Thread.sleep(3000);
            } catch (InterruptedException e) {
                System.out.println("task1 finished");
            }
            return "task1 result";
        };
    };

    private Callable<String> callable2 = new Callable<String>() {

        public String call() throws Exception {
            try {
                System.out.println("task2 starting");
                Thread.sleep(6000);
            } catch (InterruptedException e) {
                System.out.println("task2 finished");
            }
            return "task2 result";
        };
    };

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
    }

    @Override
    protected void onPause() {
        super.onPause();

        task1.unregisterCallback();
        task2.unregisterCallback();
    }

    @Override
    protected void onResume() {
        super.onResume();

        task1 = Task.getOrCreate(this, TASK1);
        task2 = Task.getOrCreate(this, TASK2);

        switch (task1.state()) {
        case NOT_STARTED:
            task1.run(this, callable1);
            break;
        case RUNNING:
            System.out.println("task1 still running");
            break;
        case COMPLETED:
            System.out.println("task1 completed in background, result: "
                    + task1.getResult());
            break;
        }

        switch (task2.state()) {
        case NOT_STARTED:
            task2.run(this, callable2);
            break;
        case RUNNING:
            System.out.println("task2 still running");
            break;
        case COMPLETED:
            System.out.println("task2 completed in background, result: "
                    + task2.getResult());
            break;
        }

    }

    @Override
    public void onTaskFinished(Task<String> task) {

        if (task.failed()) {
            System.err.println("task" + task.getTaskId() + " failed. Reason: "
                    + task.getError().getMessage());
        } else {
            System.out.println("task" + task.getTaskId() + " finish handler: "
                    + task.getResult());
        }
    }

    @Override
    public boolean onKeyDown(int keyCode, KeyEvent event) {

        if (keyCode == KeyEvent.KEYCODE_BACK) {
            Task.cancelAll(this);
        }

        return super.onKeyDown(keyCode, event);
    }
}

Let’s walk through that code bit by bit. First, our activity defines two task objects, task1 and task2, with IDs TASK1 and TASK2. What those tasks are supposed to do is defined using two Callable objects, callable1 and callable2. Everything that happens inside the call() methods of those objects will be executed in a separate thread. We also have to tell those tasks what will happen should they complete. We do this by implementing the TaskListener interface, which currently only defines a single method: onTaskFinished(Task). We can check in that handler whether the task succeeded or not by calling its failed() method. If any exception was thrown during the execution of call(), this method will return true and the exception can be retrieved by calling its getError() method. Otherwise, the return value of getResult() is guranteed to be whatever you return in the callable. The Task class is generic: You instantiate it using the return type of the callable you pass to it. This ensures type safety when working with the result object. The same holds for TaskListener.

A closer look to onResume() reveals that everytime our activity is resumed, those task objects are either already in memory or will be created for us by calling Task.getOrCreate(). If we already started that task during a previous life-cycle of our activity, we can poll its status to check whether it has already completed or if it’s still running. The former is the case if the thread had terminated while our activity was paused or even completely destroyed; in that case, we can simply pick up whatever result the task came up with in the meantime. We also call Task.unregisterCallback() in onPause() in order to avoid being called back by the task when the activity goes poof (not doing so may result in memory leaks, as described above).

Right now, the activity will pick up any results of a task with a certain ID when being restarted, even when explicitly restarted by the user. If that’s not what you want, you can make a call to Task.cancelAll() in the key handler for the ‘back’ key. That way you can ensure that all tasks (or their results) are discarded when explicitly exiting the activity. You can also cancel a single task using task.cancel(). A canceled task will never post any result or error data back to the caller.

How it works

Internally, task state is maintained in a static hash mapping callers to their list of tasks. This assures that tasks are kept in memory as long as the Task class itself (or until they terminate of course). The Task class does all the locking, state updates and callback invocations for us; we can even create a ProgressDialog and assign it to a Task using Task.setProgressDialog(). The dialog will then automatically be displayed when the task starts running, and will close when the task finishes. A task (or more precisely: a list of tasks) is bound to its caller (the calling activity or service) by the caller’s ComponentName. That means, you can think of tasks being associated to a calling class, rather than a calling object. The task class will take care of removing tasks that finished and which have been posted back to the caller, but it will preserve all completed (uncanceled) tasks until the caller claims its results, in case the caller wasn’t reachable while the task was finishing.

You can download the Task module for free here.

March 15, 2009

Understanding Android Themes and Styles

Filed under: Mobile Devices, Software Development & Programming — Tags: , , — Matthias @ 8:10 pm

I don’t know how many hours I’ve spent struggling with Android’s theme engine, trying to figure out — mostly by trial and error — how styles are applied and how to properly define them (let’s be honest: the documentation on styles in Android is a bit lacking). I thought it may be worth sharing what I’ve found out so far, in order to save my fellow Android developers from baldness due to ripped out hair (I sure lost some). This is a work in progress (so is Android), and some information may be inaccurate. If you find anything that you think is wrong or unclear, please let me know by dropping a line in the comments section. On another note, I assume that you have read the reference documentation on Android styles, if not, you should do that now.

What are Android styles?

A style in Android is a collection of attribute/value pairs applied to a view, an Activity or the entire application (which means, all Activities in your application). Styles for Activities are called themes, but do not get confused: they are syntactically equivalent to styles (because they are styles), they merely have a different scope. At this point I want to clarify something upfront: Themes or styles in Android have nothing to do with user styles known from applications such as, say, Firefox. Android users can not change the looks of Android applications by downloading themes from the Market or creating their own (yet). Android themes are of interest for developers only, think of them as syntactic sugar, a way of making your application’s view code more DRY.

How do I define custom styles?

Let’s look into a simple theme definition (stored in res/values/styles.xml):

<?xml version="1.0" encoding="utf-8"?>
<resources>
  <style name="MyTheme" parent="android:Theme.Light">
    <item name="android:windowNoTitle">true</item>
    <item name="android:windowBackground">@color/translucent_blue</item>
    <item name="android:listViewStyle">@style/MyListView</item>
  </style>

  <style name="MyListView" parent="@android:style/Widget.ListView">
    <item name="android:listSelector">@drawable/list_selector</item>
  </style>
</resources>

First, we declare a new theme called MyTheme, which inherits from another theme called Theme.Light in the android namespace (specified by the ‘parent’ attribute). This means that all styles we do not explicitly specify in our custom theme will be merged down from the definition of android:Theme.Light (the curious among you may want to download the Android source code and go to frameworks/base/core/res/res/values to see how it is defined; all stock styles are defined in themes.xml and styles.xml in that folder).

In our theme definition, we also set some custom styles using the ‘item’ element. We set the windowNoTitle attribute to true, so all activities that use our theme won’t have a title bar. We also set the window background color to a custom color called ‘translucent_blue’; this is a resource reference (indicated by the @ symbol) to a color definition in values/colors.xml. We could also have passed a reference to a drawable here, or have provided a hex color value directly. Finally, we set the default style for ListView widgets to a custom style called MyListView. Its style definition follows the exact same structure as the MyTheme definition, just that this time it is not a theme, but simply a style that can be applied to ListView objects. It inherits from the default styles for ListView but replaces the default selector image with a custom drawable.

There are two important things to understand here. First, those two style definitions are completely independent of each other (well, apart from the fact that we reference MyListView in MyTheme of course). This means, that if I remove the reference to MyListView from MyTheme, I can still use that style by applying it manually to a ListView declaration in a layout file. Think of styles simply as a bunch of attributes and values you would have otherwise typed directly into your view definition, so instead of writing

<ListView android:listSelector="@drawable/list_selector" />

we write

<ListView style="@style/MyListView" />

or better yet, we let this style be applied automatically to all ListView widgets by setting it in a theme definition (as seen above). (Note the missing ‘android:’ namespace reference here; this is intentional, the ‘style’ attribute is not defined in the ‘android’ namespace. You will get an error if you try to set a style using ‘android:style’, there is no such attribute.)

That way we never have to touch the ListView definition anymore; we can do all its styling from a single point, the style sheet. This helps in leaving the code for defining structure in your layouts, while leaving the code for defining appearance in your style sheets — a good example for separation of concerns. More importantly, your style definitions can be reused, which is particularly useful if styles are shared between several views.

The other important thing to understand is that styles do not have any type semantics. When we create a style called MyListView which inherits from Widget.ListView, then the intuition is that we are creating a style that is only supposed to apply to ListView objects. That’s not entirely correct though. There is no mechanism that will check whether we indeed apply that style to a ListView or even prevent us from applying it to an entirely different view type. It just so happens that Widget.ListView defines some attributes that only make sense when being applied to a ListView (such as listDivider), this doesn’t stop us, however, from creating a style that totally makes sense for any widget type (maybe because it only uses attributes defined in the View class, the parent class of all views). The bottom line is that you have to figure out yourself whether a style you define makes sense when you apply it to a view. Android will not do any sanity checks for you. In the worst case, the targeted view will expose odd behavior when being rendered, but more probably nothing will happen at all.

So what can I style, and from what can I inherit?

To get going with styles, it’s of course important to know two things:

  1. what’s there to be styled, and
  2. what styles are already there to inherit from

The easy answer is: Anything in android.R.styleable can be used inside a style body as a subject for styling (using the item tag), while anything in android.R.style can be used as parent styles to inherit from. So, to stick with the ListView example style, the android:listSelector style item can be found in android.R.styleable, while android:style/Widget.ListView is defined in android.R.style. Those two files are therefore what you want to turn your attention to first when defining custom styles. You should generally always inherit all default style items first, and then overwrite them one by one with your custom items.

That’s basically already it, a simple yet powerful style framework if used correctly!

Useful Tidbits

Text Appearance

Did you find yourself defining text appearance like font size or text color over and over again in your layouts? Don’t do that, use text appearances instead. Text appearances are styles, too, and Android already defines some for you which you can override (they are, of course, defined in R.style). This helps tremendously in keeping your view code DRY, and encourages you to get some structure into your different font styles used throughout your app by grouping them into styles you can reference and reuse. If, for example, you want to change the default text appearance in your app, simply define a custom text appearance in your stylesheet and set it as the default in your theme:

<?xml version="1.0" encoding="utf-8"?>
<resources>
  <style name="MyTheme" parent="android:Theme.Light">
    <item name="android:textAppearance">@style/MyDefaultTextAppearance</item>
  </style>

  <style name="MyDefaultTextAppearance" parent="@android:style/TextAppearance">
    <item name="android:textSize">12sp</item>
    <item name="android:textColor">#333</item>
    <item name="android:textStyle">bold</item>
  </style>
</resources>

Make yourself familiar with all the text appearances Android already defines, and customize them for your application as you see fit.

Colors

Avoid setting color values directly in layouts or styles. Define them in a color resource file instead, and only use references to those color values from your layouts and styles. This helps in isolating them from the rest of your view code and makes your styles less brittle if you should decide to change a color afterwards. For simple, non-stateful colors, you can simply create a colors.xml file in res/values:

<?xml version="1.0" encoding="UTF-8"?>
<resources>
    <color name="default_text_color">#FFEAEAEA</color>
    ...
</resources>

Often, however, colors can be more complex. If, for instance, you use a custom list selector image, it may happen that list text is readable when not selected, but becomes difficult or impossible to read when the custom selector is drawn (e.g. because the coloring is too similar). In that case, you need a ColorStateList, which you can use to automatically change colors for the view it’s applied to, based on that view’s state. Start by creating a new file, e.g. ‘stateful_text_color.xml’ in your res/colors directory. The resource root is ‘selector’, which is what you always use when creating resources that change with a view’s state:

<selector xmlns:android="http://schemas.android.com/apk/res/android">
  <item android:state_enabled="false" android:color="#333" />
  <item android:state_window_focused="false" android:color="#CCC" />
  <item android:state_pressed="true" android:color="#FFF" />
  <item android:state_selected="true" android:color="#FFF" />
  <item android:color="@color/default_text_color" /> <!-- not selected -->
</selector>

You get the idea. You can set this stateful color resource like any other color, it’s simply defined in a different way.

Also keep in mind that there are already color definitions shipped with Android. The most useful of them is probably android:color/transparent, which corresponds to the color #00000000. The first 8 bit, i.e. the first two hex digits, represent the alpha channel which is used to define opacity. A value of zero means zero opacity, or 100% transparency; the remaining color bits are therefore of no relevance, because they will be invisible. You can then turn the background of your views transparent by doing something like:

<style name="MyListView" parent="@android:style/Widget.ListView">
  <item name="android:background">@android:color/transparent</item>
</style>

That’s it for now. If I find out more about styles in Android, I’ll update this post accordingly. Feel free to contribute in the comments section, too.

December 4, 2008

How To Gracefully Recover From File Upload Errors In Grails

Filed under: Software Development & Programming — Tags: , , — Matthias @ 4:45 pm

Problem statement

It is often desirable to let users upload files to your website, such as custom profile images. This is not a trivial task however, since it is prone to errors (e.g. file sizes exceeding maximum values, or certain file types not being allowed by the system).

Grails supports file uploads through its Spring layer, which in turn uses the Apache Commons FileUpload library, a Java based library that brings support for handling HTTP requests of the multipart/form-data MIME type (that’s the format of requests submitted from an HTML form). However, there is one major problem: Any exceptions that occur while uploading files (e.g. due to oversize files) are thrown in a Servlet specific to that library, and the request never reaches any of your Grails controllers. This means that if the user tries to upload a file that is too large, the backend will bail out with an internal exception and the container will display a 500 to the user.

That’s obviously not what we want. Consider for example a signup page where the user can upload a custom profile picture. If the maximum allowed size for that picture is, say, 1MB, we don’t want the user to face a 500, but instead gracefully handle that error and set an error flag on the signup data (e.g. using Spring’s Errors.reject()).

Solution

Grails manages a Spring bean for handling file uploads called ‘multipartResolver’, which is an instance of Spring’s CommonsMultipartResolver. We can set e.g. the maximum allowed size for uploaded files on that bean using the maxUploadSize property (you could do that e.g. in BootStrap.groovy), but that alone does not solve the problem mentioned above. What we want to do is subclass CommonsMultipartResolver, handle any exceptions we do not want to reach the user, and redirect to our own error handling logic. In the case for handling exceeding file sizes, our custom resolver could look like this:

public class CustomMultipartResolver extends CommonsMultipartResolver {

    static final String FILE_SIZE_EXCEEDED_ERROR = "fileSizeExceeded"

    public MultipartHttpServletRequest resolveMultipart(HttpServletRequest request) {

        try {
            return super.resolveMultipart(request)
        } catch (MaxUploadSizeExceededException e) {
            request.setAttribute(FILE_SIZE_EXCEEDED_ERROR, true)
            return new DefaultMultipartHttpServletRequest(request, [:], [:])
        }
    }
}

In the controller responsible for processing the request, we can then do as follows:


def handleRequest = {
    def user = new User(params)
    if (request.getAttribute(CustomMultipartResolver.FILE_SIZE_EXCEEDED_ERROR)) {
        user.errors.reject("user.picture.fileSizeTooLarge")
        return [user: user]
    } else {
        // OK, continue processing
    }
}

Of course, since we replaced the implementation of Grails’ multipartResolver, we have to tell Grails we’re using a custom class. We do that via Spring in resources.groovy or resources.xml:

<beans xmlns="...">
    <bean id="multipartResolver" class="com.example.CustomMultipartResolver">
        <!-- limit uploads to 1MB -->
<property name="maxUploadSize" value="1048576" />
    </bean>
</beans>

And that’s it. If the user now uploads an image which is larger than 1MB, the website will display a validation error not visually distinguishable from other validation errors which are actually produced by Spring.

References:
http://blog.bruary.net/2008/03/grails-ajax-file-upload-progressbar.html

October 17, 2008

Talking to Web Servers via HTTP in Android 1.0

Filed under: Mobile Devices, Software Development & Programming — Tags: , , , — Matthias @ 9:08 pm

If you have worked with Google Android prior to the 1.0 release, you probably have noticed that Google has upgraded the Apache HttpClient module in Android 1.0 from 3.x to a recent alpha release of version 4, which is a complete rewrite and brings with it a host of API changes. Unfortunately, the alpha version included in Android 1.0 is both terribly documented and lacking an important feature: supporting multipart requests using the multipart/form-data MIME type. Sending GET requests has changed significantly, too. Where you were formerly be able to add request parameters to a GetMethod object, you now will have to build a query string on your own and initialize an HttpGet object with it.

Here is how HTTP GET works in Android 1.0:

HttpClient httpClient = new DefaultHttpClient();

StringBuilder uriBuilder = new StringBuilder(SERVICE_ENDPOINT);
uriBuilder.append("?param0=" + param0);
uriBuilder.append("&param1=" + param1);
uriBuilder.append("&paramN=" + paramN);

HttpGet request = new HttpGet(uriBuilder.toString());
HttpResponse response = httpClient.execute(request);

int status = response.getStatusLine().getStatusCode();

// we assume that the response body contains the error message
if (status != HttpStatus.SC_OK) {
    ByteArrayOutputStream ostream = new ByteArrayOutputStream();
    response.getEntity().writeTo(ostream);
    Log.e("HTTP CLIENT", ostream.toString()));
} else {
    InputStream content = response.getEntity().getContent();
    // <consume response>
    content.close(); // this will also close the connection
}

Sending a multipart request first involves some environmental setup, since the required libraries are not bundled with Android 1.0. First, go to the Apache HttpClient download page and download the distribution called “Binary with dependencies”. In that package you’ll find two libraries: apache-mime4j-0.4.jar and httpmime-4.0-beta1.jar. Copy these files to e.g. the lib/ folder of your Android project and add them to the build path. You can now use a MultipartEntity to send multipart POSTs as such:

HttpClient httpClient = new DefaultHttpClient();

HttpPost request = new HttpPost(SERVICE_ENDPOINT);
MultipartEntity entity = new MultipartEntity();
entity.addPart("param0", new StringBody("value0"));
entity.addPart("param1", new StringBody(Double.toString(1.0)));
entity.addPart("paramN", new FileBody(new File("/bar/baz")));
request.setEntity(entity);

HttpResponse response = httpClient.execute(request);
int status = response.getStatusLine().getStatusCode();

if (status != HttpStatus.SC_OK) {
    // see above   
} else {
    // see above
}

And that’s that. I assume Google has included this early build of HttpClient 4 in Android 1.0 so that they will remain API-stable for the years to come, without being stuck with a legacy HTTP component.

References:
http://groups.google.com/group/android-developers/browse_frm/thread/e4230ed22c196772
http://wiki.apache.org/HttpComponents/HttpCoreTutorial
http://hc.apache.org/httpcomponents-client/examples.html

August 15, 2008

Handling Long Running Operations In Google Android

Filed under: Mobile Devices, Software Development & Programming — Tags: , , — Matthias @ 12:27 pm

User interface responsiveness is a crucial thing for all applications that require user interaction, but maybe even more so when programming for mobile handsets. Google’s Android programming environment unfortunately does not provide any mechanism for handling operations that may require a fair amount of time to complete, but which in itself are not meant to be implemented as Android Services. An example for this would be network I/O in an Activity, such as posting data to (or retrieving data from) a remote Web server. Because the Android runtime will terminate any Activity that does not respond within a couple of seconds, it is impossible (and simply a bad idea anyway) to perform such tasks from within the UI main thread.

That being said, I have come up with a generic class that handles long running operations by spawning a separate thread and passing back any result or error data to the main thread using a callback mechanism. An animated progress dialog will be displayed while the operation is running. That way the user is kept informed about any program activity that may take some time to complete.

The class can be used as follows:

public class MyActivity implements LongRunningActionCallback<Void> {

    private LongRunningActionDispatcher<Void> dispatcher;

    private void startLongRunningOperation() {
        // the first argument is a reference to the current Context, in this
        // case the current Activity. The second argument is a reference to
        // the object implementing the callback method.
        this.dispatcher = new LongRunningActionDispatcher<Void>(this, this);
        dispatcher.startLongRunningAction(new Callable<Void>() {
            public Void call() throws Exception {
                // perform your actions that take a long time
                return null;
            }
        }, "Dialog Title", "Dialog message");
    }

    // the callback
    public void onLongRunningActionFinished(Void result, Exception error) {
        if (error != null) {
            // handle error
        } else {
            // success, work with the result, if any
        }
    }
}

This will spawn a progress dialog (not indicating any actual progress in percentage, it’s just a “busy” dialog) with the given title and message. If an exception occurred in the Callable you provided, it will be passed as the error argument to the callback, so you should always check whether it’s non-null.

Below are the source codes for both LongRunningActionDispatcher and LongRunningActionCallback.

import java.util.concurrent.Callable;

import android.app.ProgressDialog;
import android.content.Context;
import android.os.Handler;
import android.util.Log;

/**
 * Use this class if you need to dispatch expensive (long running) operations
 * from your Activity. The long running operation is provided to
 * {@link startLongRunningAction} as a {@link Callable}. The result of the
 * operation and any potential exception that occurred during the call are
 * passed to {@link LongRunningActionCallback.onLongRunningActionFinished},
 * which will be called on successful or unsuccessful completion of the
 * Callable.
 *
 * LICENSE STATEMENT (DO NOT REMOVE):
 * This code is in the public domain. You may use, alter, and redistribute it
 * free of any charges or obligations, with the following exceptions:
 * 1. You are not allowed to remove the statement naming the original author.
 * 2. You are not allowed to remove this license statement.
 *
 * @author Matthias Kaeppler
 */
public final class LongRunningActionDispatcher<ResultType> {

    private Context context;

    private LongRunningActionCallback<ResultType> callback;

    /**
     * A progress dialog shown during long-lasting operations
     */
    private ProgressDialog progressDialog;

    private Handler finishedHandler = new Handler();

    public LongRunningActionDispatcher(Context context,
            LongRunningActionCallback<ResultType> callback) {
        this.context = context;
        this.callback = callback;
    }

    /**
     * Invoke this method to start long running operations which may block your
     * activity and therefore the main UI thread. A progress dialog will be
     * shown while the operation is executing.
     *
     * @param callable
     *            The callable
     * @param progressDialogTitle
     *            The progress dialog title
     * @param progressDialogMessage
     *            The progress dialog message
     */
    public void startLongRunningAction(final Callable<ResultType> callable,
            String progressDialogTitle, String progressDialogMessage) {

        progressDialog = ProgressDialog.show(context, progressDialogTitle,
                progressDialogMessage, true, false);

        new Thread(new Runnable() {

            public void run() {
                ResultType result = null;
                Exception error = null;
                try {
                    result = callable.call();
                } catch (Exception e) {
                    Log.e("ERROR", e.getMessage());
                    error = e;
                }

                final ResultType finalResult = result;
                final Exception finalError = error;
                finishedHandler.post(new Runnable() {

                    public void run() {
                        onLongRunningActionFinished(finalResult, finalError);
                    }
                });
            }
        }).start();
    }

    private void onLongRunningActionFinished(ResultType result, Exception error) {
        progressDialog.dismiss();
        callback.onLongRunningActionFinished(result, error);
    }
}
/**
 * LICENSE STATEMENT (DO NOT REMOVE):
 * This code is in the public domain. You may use, alter, and redistribute it
 * free of any charges or obligations, with the following exceptions:
 * 1. You are not allowed to remove the statement naming the original author.
 * 2. You are not allowed to remove this license statement.
 *
 * @author Matthias Kaeppler
 */
public interface LongRunningActionCallback<ResultType> {

    /**
     * Called when the callable provided to
     * {@link LongRunningActionDispatcher.startLongRunningAction} completes.
     *
     * @param <ResultType>
     *            The result type of callable.call()
     * @param result
     *            Whatever the callable returns if it completes successfully, or
     *            null if an exception was thrown
     * @param error
     *            Whatever the callable throws if it executes in error, or null
     *            if it completed successfully
     */
    void onLongRunningActionFinished(ResultType result, Exception error);
}

July 31, 2008

Grails Quick Tip #2: Getting Access to SessionFactory and MessageSource

Filed under: Software Development & Programming — Tags: , — Matthias @ 9:52 am

Sometimes it is necessary to reach beyond the dynamics of Grails’ finder methods and other injected features, and instead make use of the underlying objects that implement this behavior directly. An example would be the need to access a Hibernate session directly, maybe because  you are writing a service object in plain Java, which means none of the features that rely on the dynamic nature of Groovy are available to you. These include:

  • the current Hibernate session
  • the Spring message source for accessing text in various translations
  • a logger object

All these objects are injected by Grails into your services automatically — that is, if you follow the conventions and are programming in Groovy. If you’re not, you’re pretty much on your own, because the Grails reference guide does not make any mention of these objects and how to access them.

However, I figured they’d be simply Spring beans as well (with the exception of log objects, see below), and due to the consistent conventions in Grails I soon discovered that they are called sessionFactory and messageSource respectively. So in order to access them directly from within a Groovy service object, just do this:

class MyService {
   def sessionFactory
   def messageSource

   def serviceMethod = {
      def session = sessionFactory.getCurrentSession()
      ...
   }
}

That doesn’t work with Java services of course. In that case, go to resources.groovy and add a bean declaration:

 myService(com.example.services.MyService) {
    sessionFactory = ref("sessionFactory")
    messageSource = ref("messageSource")
 }

Note that log objects have to be created manually, because they are not handled by the Spring container. You could for instance create a Log4J object, as shown below. Do not forget to add the setters for the Spring beans, so Grails can set them for you automatically:

import org.apache.log4j.Logger;
import org.hibernate.SessionFactory;
import org.springframework.context.MessageSource;

public class MyService {
    // instantiates a Log4J logger object
    private Logger log = Logger.getLogger(getClass().getName());

    // a Hibernate SessionFactory
    private SessionFactory sessionFactory;

    // a Spring MessageSource
    private MessageSource messageSource;

    [...]

    // used by the Spring IoC container to inject the session factory bean
    public void setSessionFactory(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    // used by the Spring IoC container to inject the message source bean
    public void setMessageSource(MessageSource messageSource) {
        this.messageSource = messageSource;
    }
}

It may be a good idea to use a class like this as a base class for all your plain Java services, so they can benefit from these objects, too.

May 28, 2008

Grails + Eclipse Quick Tip #1

Filed under: Software Development & Programming — Tags: , — Matthias @ 9:07 am

More Automated Builds

here is a little hint on how to make building your Grails apps from within Eclipse a little less explicit. The goal is to avoid being asked to manually issue a “grails package” after each re-build. Plus, if you followed my hints on how to let Grails and Eclipse compile to the same directory structure, you won’t need to issue “grails clean” anymore. Simply running Project –> Clean… and then running the Grails launcher will suffice for successful compilation and deployment if you follow these instructions.

  1. Go to you project properties and select the Builders entry. You should see two builders: Groovy builder and Java builder.
  2. If you haven’t yet configured an external tools launcher for a “grails package”, click New…, otherwise import that builder
  3. If you have to create the builder, set the following fields:
    • set Location to the folder where the Grails executables reside on your system
    • set Working Directory to ${workspace_loc:/your_project_name}
    • set Arguments to “package”
    • In the Build Options tab, check the fields After a “Clean” and During manual builds
  4. move the new builder to the bottom

I would also advice to create a keyboard shortcut to issue a manual build. For myself, I reassigned Ctrl+B to “Build project” instead of “Build all”.

When you now “Clean” your project and run the “run-app” launcher (I assume you have one!), Eclipse will first build your project and package it for deployment. The packaging is conditional though: if the sources are up-to-date, the new builder will be excluded when launching the app.

May 3, 2008

Grails and Eclipse: Not So Groovy

Filed under: Software Development & Programming — Tags: , , , — Matthias @ 7:16 pm

Grails is a Web application framework for rapid prototyping based on the Groovy scripting language, a Java dialect that compiles to Java byte code. Grails strongest advantage over similar frameworks (e.g. AppFuse, to name a more popular one) is its consequent use of the Convention Over Configuration paradigm, making it almost a no-brainer to set up a full featured Web app using Grails. Grails also relies heavily on the dynamic nature of Groovy to provide useful utility functionality or inject commonly used features such as logging objects and CRUD accessors for domain objects at runtime.

While Grails is an excellent piece of software, it is completely beyond me why apparently almost zero effort has been undergone to make it work flawlessly with one of the most (if not THE) widely used IDEs out there: the Eclipse IDE.

From day one, I had nothing but trouble running Grails applications in Eclipse. There is a document over at grails.org which has instructions for Grails + Eclipse, but it is erroneous and contradictory. While I still suggest to read that document, I will offer solutions to most of the problems you will probably still encounter, even after following that guide.

All instructions below worked for Grails 1.0.2 and Eclipse 3.3 on Ubuntu Linux 8.04.

The Groovy Eclipse Plugin

I assume that the Eclipse Groovy plugin has been installed.

By default, the output directory setting for Groovy compilations is left empty. That means your Groovy files in Eclipse will be compiled to your Grails project root. That is bad, as it confuses Grails. Set it to a different folder, e.g. bin-groovy. In any case, regardless what the Eclipse guide over at grails.org says, leave the “Disable Groovy Compiler Generating Class Files” box unchecked, but check “Disable Check Package Matches Source Directory”. Two reasons: If you check the first one, you won’t be able to debug your Groovy sources, but of course you want that. If you leave the second unchecked, you will get errors related to Groovy’s Spring DSL config in resources.groovy. This error is safe to ignore however.

Make sure to follow the instructions for debugging Groovy applications in Eclipse, too.

Fixing The Class-File Hell: 1st Try

Grails will generate an Eclipse .project file for you after issueing ‘grails create-app’. However, there are several issues with that file. First of all, Grails always compiles your Groovy and Java sources to the directory ~/.grails/<ver>/projects/<proj>/classes, but the generated project file sets the default output folder for Java class files to web-app/WEB-INF/classes. Furthermore, the Groovy plugin may compile to yet another directory, say bin-groovy. In other words, by default an imported Grails app is doomed to make you trouble, because the ‘grails’ command will always compile to a different folder than Eclipse does, and you will end up in a class-file mess where Eclipse will not recognize any changes made by Grails and vice versa.

The solution obviously is to make both Grails, the Eclipse Java compiler and the Groovy Eclipse plugin all compile to the same output directory. However, this is not as easy as it sounds. The best solution would be to tell Grails to compile to a different directory, but I haven’t found a way to do so.

The next best solution would be to create two symbolic links in your project root: bin and bin-groovy, both pointing to the Grails output directory (see above, it’s in your home directory under .grails/) and then tell the Eclipse Java compiler to compile to bin/ (via Project -> Properties -> Java Build Path -> Default Output Folder) and the Groovy plugin to compile to bin-groovy (via Project -> Properties -> Groovy Project Properties -> Groovy compiler output location).

While this solves the class-file hell, it leads to a weird problem: At least for me, I wasn’t able to run my Grails app from its Eclipse launcher anymore without the launch failing with a cryptic error message:

Failed to compile configuration file <proj_path>/grails-app/conf/Config.groovy: No signature of method: groovy.util.ConfigSlurper.parse() is applicable for argument types: (Config)

As of now, no one on the grails-user mailing list was able to say what is causing this, but of course it’s a show stopper, because you won’t be able to debug your application anymore (please note that this only affects Eclipse, your app would still start from the command line using ‘grails run-app’).

The third solution is somewhat ugly, but it works.

Fixing The Class-File Hell: 2nd Try

This approach is as follows: Instead of symlinking from bin and bin-groovy to the Grails output folder, we do it vice versa:

  1. leave the Java output folder to its default value: web-app/WEB-INF/classes
  2. the Groovy class files must also go there, so set a bin-groovy symlink to the same directory
  3. add bin-groovy to the build path: Project -> Properties -> Java Build Path -> Libraries -> Add Class Folder
  4. go to ~/.grails/<ver>/projects/<proj>, remove the classes/ folder and create a symlink “classes” to the full path of the web-app/WEB-INF/classes directory (e.g. ~/workspace/<proj>/web-app/WEB-INF/classes)

What we have done now is to turn around the sym-linkage: Grails will now compile to web-app/WEB-INF/classes. You may ask: What is the difference to the first approach?! Actually, there shouldn’t be any difference, but now you will notice that the ConfigSlurper error is gone, and you can run your app again from within Eclipse. THAT is why we do it (in lack of a better explanation…).

Unfortunately we’re still not 100% done. The reason is: As soon as you issue a ‘grails clean’, the symlink will become invalid and any “creational” follow-up commands like ‘grails compile’ will fail. The problem here is that ‘grails clean’ does not merely empty the ‘classes’ directory, but removes it alltogether. So we need to tell Grails not to remove the directory upon clean, but only its contents. A little background: Whenever you issue a command like this, you are effectively executing a Gant script (that’s the Groovy DSL for Apache Ant). These scripts reside in your Grails installation directory, in the scripts/ subfolder. Find the Clean.groovy script and perform the following changes:

  1. Find the target cleanCompiledSources
  2. Change the first call to Ant.delete to this code:
    Ant.delete(includeEmptyDirs:true) {
       fileset(dir:"${webInf}/classes", includes:"**/*")
    }
  3. Remove/comment the line Ant.delete(dir:classesDirPath)

save the file and exit. When you now run ‘grails clean’, only the content of classes/ will be removed, not the directory itself.

And that’s it!

Grails and Eclipse should now love each other. Ain’t that a happy ending?

March 18, 2008

Using cURL for Testing Web Applications

Filed under: Software Development & Programming — Tags: , , , , , , , , — Matthias @ 11:05 am

This may be old news to some, but although I had heard of using cURL for downloading files off the internet from a command line (as a wget alternative), I didn’t know that it was actually capable of doing much, much more. In fact, cURL is a very good allrounder at doing anything related to transmitting and receiving data using popular protocols such as HTTP(S), (S)FTP, TELNET, LDAP and more. In particular, you can use it to test your Web application with very little effort!

I am currently programming for a Java Servlet based Web application, where I need to assemble HTTP requests on the client side and dissect them on the server side. More precisely, I have a servlet which extracts data from an HTTP multipart request (using Apache Commons FileUpload) and hand it over to the logic for further processing. Of course, I have to write lots of code for checking whether requests are valid and so forth. So, in order to test this code, I want to send “malformed” requests, by which I mean requests that do not carry data expected by the application in its current state. Instead of setting up a complex test environment or even modifying your client code to send bad data, you can simply use cURL.

cURL uses a simple command line interface; I will show you the most important flags. Let’s assume we have a simple RESTful Web application which manages books. We could use cURL to retrieve the list of available books like this:

curl http://localhost:8080/mybookstore/books -v

This will issue an HTTP GET on the specified URL. The -v flag tells cURL to be verbose, which means it will print all sent and received HTTP headers, the payload of the server reply, plus some additional status information to the standard output. The complete output might look something like this:

* About to connect() to localhost port 8080 (#0)
* Trying 127.0.0.1… connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mybookstore/books HTTP/1.1
> User-Agent: curl/7.18.0 (i586-pc-mingw32msvc) libcurl/7.18.0 zlib/1.2.3
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/xml;charset=utf-8
< …
< <books><book author=”David Flanagan”>Java In A Nutshell</book><book author=”Stephen Hawking”>A Brief History Of Time</book>…</books>

The > symbol indicates data going to the server, while < indicates data coming from the server. If that’s not verbose enough for you, try using the --trace-ascii <filename> argument instead. It will log the client/server conversation to the specified file.

So, what about sending data? It’s actually just as easy. Suppose we want to store a new book by supplying its title and author:

curl http://localhost:8080/mybookstore/books -F “title=Wikinomics” -F “author=D. Tapscott, A. D. Williams”

What this does is issueing an HTTP POST to the specified address, submitting the given data using the multipart/form-data MIME type. Now that was easy! If you don’t want to type all the input data on the command line, you can also let cURL read data from a file by prefixing the value with @ followed by the file name, which makes perfect sense when transmitting binary data or long text documents. As to the way how cURL transmits the data, you can also use the -d flag, which will result in the data being POST-ed as an application/x-www-form-urlencoded string, or use the same -d flag in combination with -G to issue a GET request instead and sending the data as a URL parameter (query string).

Another noteworthy aspect about cURL is that you can use it to manage cookies. Let’s assume our book server requires the user to login first, and that the server remembers us by setting a session cookie. Ponder the commands below, where we use cURL first to authenticate with our server, and then add a new book in the same session:

curl http://localhost:8080/mybookstore/login -d “username=john” -d “password=doe” -c cookies.txt
curl http://localhost:8080/mybookstore/books -b cookies.txt -F “title=…” …

In the first line, we tell cURL to use the file cookies.txt as its “cookie jar”, which means that it will store all cookies set by the server in this file. We can leverage the session data stored in that file (for a Servlet container this would be a JSESSIONID) in consecutive requests in order to remain authenticated, by setting the value of the -b flag to the cookie jar. You can also use key/value pairs as an argument to -b, but using an intermediary file is more convenient.

cURL is free software and can be downloaded for a broad range of platforms from the cURL website.

« Newer PostsOlder Posts »

Create a free website or blog at WordPress.com.