Brain Flush

March 18, 2008

Using cURL for Testing Web Applications

Filed under: Software Development & Programming — Tags: , , , , , , , , — Matthias @ 11:05 am

This may be old news to some, but although I had heard of using cURL for downloading files off the internet from a command line (as a wget alternative), I didn’t know that it was actually capable of doing much, much more. In fact, cURL is a very good allrounder at doing anything related to transmitting and receiving data using popular protocols such as HTTP(S), (S)FTP, TELNET, LDAP and more. In particular, you can use it to test your Web application with very little effort!

I am currently programming for a Java Servlet based Web application, where I need to assemble HTTP requests on the client side and dissect them on the server side. More precisely, I have a servlet which extracts data from an HTTP multipart request (using Apache Commons FileUpload) and hand it over to the logic for further processing. Of course, I have to write lots of code for checking whether requests are valid and so forth. So, in order to test this code, I want to send “malformed” requests, by which I mean requests that do not carry data expected by the application in its current state. Instead of setting up a complex test environment or even modifying your client code to send bad data, you can simply use cURL.

cURL uses a simple command line interface; I will show you the most important flags. Let’s assume we have a simple RESTful Web application which manages books. We could use cURL to retrieve the list of available books like this:

curl http://localhost:8080/mybookstore/books -v

This will issue an HTTP GET on the specified URL. The -v flag tells cURL to be verbose, which means it will print all sent and received HTTP headers, the payload of the server reply, plus some additional status information to the standard output. The complete output might look something like this:

* About to connect() to localhost port 8080 (#0)
* Trying 127.0.0.1… connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mybookstore/books HTTP/1.1
> User-Agent: curl/7.18.0 (i586-pc-mingw32msvc) libcurl/7.18.0 zlib/1.2.3
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/xml;charset=utf-8
< …
< <books><book author=”David Flanagan”>Java In A Nutshell</book><book author=”Stephen Hawking”>A Brief History Of Time</book>…</books>

The > symbol indicates data going to the server, while < indicates data coming from the server. If that’s not verbose enough for you, try using the --trace-ascii <filename> argument instead. It will log the client/server conversation to the specified file.

So, what about sending data? It’s actually just as easy. Suppose we want to store a new book by supplying its title and author:

curl http://localhost:8080/mybookstore/books -F “title=Wikinomics” -F “author=D. Tapscott, A. D. Williams”

What this does is issueing an HTTP POST to the specified address, submitting the given data using the multipart/form-data MIME type. Now that was easy! If you don’t want to type all the input data on the command line, you can also let cURL read data from a file by prefixing the value with @ followed by the file name, which makes perfect sense when transmitting binary data or long text documents. As to the way how cURL transmits the data, you can also use the -d flag, which will result in the data being POST-ed as an application/x-www-form-urlencoded string, or use the same -d flag in combination with -G to issue a GET request instead and sending the data as a URL parameter (query string).

Another noteworthy aspect about cURL is that you can use it to manage cookies. Let’s assume our book server requires the user to login first, and that the server remembers us by setting a session cookie. Ponder the commands below, where we use cURL first to authenticate with our server, and then add a new book in the same session:

curl http://localhost:8080/mybookstore/login -d “username=john” -d “password=doe” -c cookies.txt
curl http://localhost:8080/mybookstore/books -b cookies.txt -F “title=…” …

In the first line, we tell cURL to use the file cookies.txt as its “cookie jar”, which means that it will store all cookies set by the server in this file. We can leverage the session data stored in that file (for a Servlet container this would be a JSESSIONID) in consecutive requests in order to remain authenticated, by setting the value of the -b flag to the cookie jar. You can also use key/value pairs as an argument to -b, but using an intermediary file is more convenient.

cURL is free software and can be downloaded for a broad range of platforms from the cURL website.

January 18, 2008

Integrating Direct Web Remoting (DWR) in Mozilla Thunderbird

Direct Web Remoting, or DWR for short, is a Web technology for exposing server-side Java interfaces via a dynamically generated JavaScript API. A specific Servlet will translate incoming JavaScript calls to their Java equivalents which makes integration of applications programmed in client-side JavaScript with Web applications written in server-side Java a no-brainer.

DWR calls are typically dispatched from a JavaScript/JScript program running in a Web browser such as Mozilla Firefox or Microsoft’s Internet Explorer, but since all Mozilla products are built upon the Gecko layout engine and XUL (which are programmed using JavaScript modules), you can leverage DWR to dispatch remote calls from e.g. a Mozilla Thunderbird extension just as easily. Integrating DWR into a XUL based extension is actually pretty straight forward, but there are couple of subtleties to be aware of.

I will hereafter assume that a Web application called ‘MyWebApp’ has been deployed on localhost port 8080, that it’s up and running, that it exposes an interface called ‘MyService’ via DWR. Refer to the DWR documentation if you are unsure how to do that. It is further assumed that the DWR servlet is reached at dwr/ relative to the base URL. All following code examples refer to DWR version 1.1, which unfortunately has already been superseded by version 2.x at the time of this writing. If required at all, changing the code to run with newer versions of DWR should be easy enough however.

In order for your XUL component to be able to access `MyService’, add the following lines to your XUL file (e.g. myOverlay.xul):

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/engine.js"/>

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/util.js"/>

<script type="application/x-javascript"
    src="http://localhost:8080/MyWebApp/dwr/interface/MyService.js"/>

Don’t go looking for MyService.js in your file system, it’s not there. DWR will take care of dynamically creating it once your client requests it from the server. You can find both engine.js and util.js in the dwr.jar file which should reside at WEB-INF/lib of your deployed Web application.

Now you’re already prepared to invoke your service. However, when run outside a Web browser, as is the case with e.g. a Thunderbird extension, there is a catch: DWR won’t correctly resolve the target URL for your server, because it operates on relative URLs by default. You can fix that problem by modifying the MyService._path variable, which was auto-generated by DWR. Ponder the example program below, which invokes MyService.service() from a JavaScript program running in a Thunderbird extension:

function callService() {

    MyService._path = "http://localhost:8080/MyWebApp/dwr";

    MyService.service({
        callback: function(returnData) {
            // perform logic
        },

        errorHandler: function(error) {
            // handle error
        }
    });
}

That’s it basically; of course you may want to add additional call variables or use another invocation style altogether, but you get the idea.

January 17, 2008

Mozilla Thunderbird Extension Development – Environment Setup

It took me a while to setup my Thunderbird installation in order to make it ready for convenient extension development, by which I mean having all the tools and configuration in place. So I thought I’d share the most useful steps you want to perform in order to make your life easier.

You should basically do two things: One, reconfigure Thunderbird to be more developer-friendly, and two, install some cool extensions.

First, I recommend installing the following plugins:

  1. ChromEdit Plus – allows for easy access to your configuration files. Works in Firefox, too.
  2. Venkman – A JavaScript debugger. Must have. For Firefox, I recommend Firebug instead.
  3. Console² – An advanced error console. Can interpret JavaScript commands on-the-fly.
  4. Chrome List – Lets you inspect chrome packages of all installed components. It’s somewhat buggy for me though, sometimes the contents of XUL files are not displayed in the viewer.

With these plugins installed, you already have a solid basis to built your hacking on.

Now to the configuration. You can do this with ChromEdit now, or if you haven’t installed it, fall back to your favorite text editor. Open the ChromEdit Plus menu, select ChromEdit and navigate to the ‘user.js’ tab. From here you can edit your ‘user.js’ file located in your profile folder (on Linux, it’s in your home directory, on Windows, it’s in ‘Documents and Settings/<username>/Application Data/Thunderbird/Profiles/<long_UUID>’ ). If it does not yet exist, and it doesn’t by default, go ahead and create it). Add the following lines (it’s a JavaScript file, so it’s simply JavaScript syntax):

user_pref(“javascript.options.showInConsole”, true);
user_pref(“nglayout.debug.disable_xul_cache”, true);
user_pref(“browser.dom.window.dump.enabled”, true);
user_pref(“javascript.options.strict”, true);

The following explanations are taken from the MDC:

  • javascript.options.showInConsole = true. Logs errors in chrome files to the Error Console.
  • nglayout.debug.disable_xul_cache = true. Disables the XUL cache so that changes to windows and dialogs do not require a restart. This assumes you’re using directories rather than JARs. Changes to XUL overlays will still require reloading of the document overlaid.
  • browser.dom.window.dump.enabled = true. Enables the use of the dump() statement to print to the standard console. See window.dump for more info. You can also use nsIConsoleService from privileged script.
  • javascript.options.strict = true. Enables strict JavaScript warnings in the Error Console. Note that since many people have this setting turned off when developing, you will see lots of warnings for problems with their code in addition to warnings for your own extension. You can filter those with Console2.

In order for the Thunderbird standard console to appear, run Thunderbird with the ‘-console’ command line argument. The dumped statements won’t appear in Console² though.

You may also want to refer to Setting up extension development environment on MDC as a primer, I particularly recommend the steps outlined for externalizing your development files into a separate folder, so you won`t have to reinstall the plugin each time you change a line of code.

December 30, 2007

Ubuntu Linux on a Samsung R55 Notebook

Filed under: Hardware & Technology, Linux & Open Source — Tags: , , , — Matthias @ 1:24 pm

In an attempt of shameless self-promotion, I’d like to point out my website covering the installation and potential issues of running Ubuntu Linux on Samsung’s R55 notebook series.

http://www.infodump.de/samsung-r55-linux

It focuses on an older version of Ubuntu, but I will update it once the next LTS release of Ubuntu hits the server somewhere around April 2008.

Create a free website or blog at WordPress.com.