Brain Flush

March 18, 2008

Using cURL for Testing Web Applications

Filed under: Software Development & Programming — Tags: , , , , , , , , — Matthias @ 11:05 am

This may be old news to some, but although I had heard of using cURL for downloading files off the internet from a command line (as a wget alternative), I didn’t know that it was actually capable of doing much, much more. In fact, cURL is a very good allrounder at doing anything related to transmitting and receiving data using popular protocols such as HTTP(S), (S)FTP, TELNET, LDAP and more. In particular, you can use it to test your Web application with very little effort!

I am currently programming for a Java Servlet based Web application, where I need to assemble HTTP requests on the client side and dissect them on the server side. More precisely, I have a servlet which extracts data from an HTTP multipart request (using Apache Commons FileUpload) and hand it over to the logic for further processing. Of course, I have to write lots of code for checking whether requests are valid and so forth. So, in order to test this code, I want to send “malformed” requests, by which I mean requests that do not carry data expected by the application in its current state. Instead of setting up a complex test environment or even modifying your client code to send bad data, you can simply use cURL.

cURL uses a simple command line interface; I will show you the most important flags. Let’s assume we have a simple RESTful Web application which manages books. We could use cURL to retrieve the list of available books like this:

curl http://localhost:8080/mybookstore/books -v

This will issue an HTTP GET on the specified URL. The -v flag tells cURL to be verbose, which means it will print all sent and received HTTP headers, the payload of the server reply, plus some additional status information to the standard output. The complete output might look something like this:

* About to connect() to localhost port 8080 (#0)
* Trying 127.0.0.1… connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mybookstore/books HTTP/1.1
> User-Agent: curl/7.18.0 (i586-pc-mingw32msvc) libcurl/7.18.0 zlib/1.2.3
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/xml;charset=utf-8
< …
< <books><book author=”David Flanagan”>Java In A Nutshell</book><book author=”Stephen Hawking”>A Brief History Of Time</book>…</books>

The > symbol indicates data going to the server, while < indicates data coming from the server. If that’s not verbose enough for you, try using the --trace-ascii <filename> argument instead. It will log the client/server conversation to the specified file.

So, what about sending data? It’s actually just as easy. Suppose we want to store a new book by supplying its title and author:

curl http://localhost:8080/mybookstore/books -F “title=Wikinomics” -F “author=D. Tapscott, A. D. Williams”

What this does is issueing an HTTP POST to the specified address, submitting the given data using the multipart/form-data MIME type. Now that was easy! If you don’t want to type all the input data on the command line, you can also let cURL read data from a file by prefixing the value with @ followed by the file name, which makes perfect sense when transmitting binary data or long text documents. As to the way how cURL transmits the data, you can also use the -d flag, which will result in the data being POST-ed as an application/x-www-form-urlencoded string, or use the same -d flag in combination with -G to issue a GET request instead and sending the data as a URL parameter (query string).

Another noteworthy aspect about cURL is that you can use it to manage cookies. Let’s assume our book server requires the user to login first, and that the server remembers us by setting a session cookie. Ponder the commands below, where we use cURL first to authenticate with our server, and then add a new book in the same session:

curl http://localhost:8080/mybookstore/login -d “username=john” -d “password=doe” -c cookies.txt
curl http://localhost:8080/mybookstore/books -b cookies.txt -F “title=…” …

In the first line, we tell cURL to use the file cookies.txt as its “cookie jar”, which means that it will store all cookies set by the server in this file. We can leverage the session data stored in that file (for a Servlet container this would be a JSESSIONID) in consecutive requests in order to remain authenticated, by setting the value of the -b flag to the cookie jar. You can also use key/value pairs as an argument to -b, but using an intermediary file is more convenient.

cURL is free software and can be downloaded for a broad range of platforms from the cURL website.

January 18, 2008

Integrating Direct Web Remoting (DWR) in Mozilla Thunderbird

Direct Web Remoting, or DWR for short, is a Web technology for exposing server-side Java interfaces via a dynamically generated JavaScript API. A specific Servlet will translate incoming JavaScript calls to their Java equivalents which makes integration of applications programmed in client-side JavaScript with Web applications written in server-side Java a no-brainer.

DWR calls are typically dispatched from a JavaScript/JScript program running in a Web browser such as Mozilla Firefox or Microsoft’s Internet Explorer, but since all Mozilla products are built upon the Gecko layout engine and XUL (which are programmed using JavaScript modules), you can leverage DWR to dispatch remote calls from e.g. a Mozilla Thunderbird extension just as easily. Integrating DWR into a XUL based extension is actually pretty straight forward, but there are couple of subtleties to be aware of.

I will hereafter assume that a Web application called ‘MyWebApp’ has been deployed on localhost port 8080, that it’s up and running, that it exposes an interface called ‘MyService’ via DWR. Refer to the DWR documentation if you are unsure how to do that. It is further assumed that the DWR servlet is reached at dwr/ relative to the base URL. All following code examples refer to DWR version 1.1, which unfortunately has already been superseded by version 2.x at the time of this writing. If required at all, changing the code to run with newer versions of DWR should be easy enough however.

In order for your XUL component to be able to access `MyService’, add the following lines to your XUL file (e.g. myOverlay.xul):

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/engine.js"/>

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/util.js"/>

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/interface/MyService.js"/>

Don’t go looking for MyService.js in your file system, it’s not there. DWR will take care of dynamically creating it once your client requests it from the server. You can find both engine.js and util.js in the dwr.jar file which should reside at WEB-INF/lib of your deployed Web application.

Now you’re already prepared to invoke your service. However, when run outside a Web browser, as is the case with e.g. a Thunderbird extension, there is a catch: DWR won’t correctly resolve the target URL for your server, because it operates on relative URLs by default. You can fix that problem by modifying the MyService._path variable, which was auto-generated by DWR. Ponder the example program below, which invokes MyService.service() from a JavaScript program running in a Thunderbird extension:

function callService() {

    MyService._path = "http://localhost:8080/MyWebApp/dwr";

    MyService.service({
        callback: function(returnData) {
            // perform logic
        },

        errorHandler: function(error) {
            // handle error
        }
    });
}

That’s it basically; of course you may want to add additional call variables or use another invocation style altogether, but you get the idea.

Create a free website or blog at WordPress.com.