nosewheelie

Technology, mountain biking, politics & music.

Archive for the ‘Java’ Category

Processing Large Datasets in Real Time

without comments

Introduction

This is an article I wrote in late 2007 for the Workingmouse wiki. With the WM wiki no longer running, I’ve republished it here for posterity. Some content may no longer be current or relevant.

Tom Adams, Workingmouse, December 2007

I’ve had the good fortune to recently complete a project at Veitch Lister Consulting (VLC) (a transport planning consultancy) processing large datasets in real-time. This article is a summary of the technical aspects of the project, I won’t be talking about the process we used, suffice to say it was XP-like, supported by tools such as my BDD framework Instinct and web-based agile project management tools. The project ran for around 7-9 weeks, the team was comprised of five developers (3 full-time on the project) consisting of two Workingmouse developers, myself and Sanjiv Sahayam, and three VLC developers, Jamie Cook, Glen Maddern and Nick Partridge.

As a consultant we usually work in corporate environments and as such are bound by the architectural constraints of the organisation. This usually includes the usual “enterprise” constraints such as language (Java 1.4 is common), application servers (usually WebLogic or WebSphere) and databases (Oracle or DB2). However this was one of those rare projects where you have no technical constraints and have the fortune to be working with great people.

Finding a working solution

Going in, we had three basic requirements, 1) must be callable from a Rails-based web app, 2) must return responses to typical requests in real-time to the webapp (30 seconds was our target) and 3) must query against a dataset that was initially estimated at around 45 TB, but later came down to around 100 GB.

Our technical approach initially was two-fold, firstly to understand the nature of the problem and size of the datasets, and secondly to research potential solutions to the problem. I won’t talk too much about the nature of the problem domain and type of data, except to say that the data is the output from transportation simulations and ended up being around 100 GB (of raw uncompressed text), which has since been converted into a binary format. To use SQL nomenclature, our data was divided into three tables, our requests to the data storage layer consisted of a single query consisting of two joins between these three tables. The system then performed further data analysis based on the results of the query. We had complete control over the data generation (this company wrote the piece that generates the data) and storage format, something that we could exploit to provide performance far above conventional generic storage engines (i.e. our solution can not be generalised). The system we were developing was completely read only, which simplified our design considerably.

Our dataset was initially estimated to be around 45 TB (based on extrapolations from the data generation component), which led us to think we’d need to distribute the data across multiple machines. Our first investigations (e.g. Hadoop) were based around this. Later, as we worked out we could batch process the majority of the data offline (as an input to the real-time system) and input only a subset, we were able to look towards conventional tools such as SQL databases.

As there were enough problems to solve already, we didn’t want to write our own storage engine. We looked towards conventional SQL databases to achieve this, including:

We spent several weeks tuning the data, the queries, indices and the databases themselves, however we were not able to get the performance that we required out of any of the conventional SQL databases. To be fair, we didn’t complete our investigations into DB2, and Oracle failed to complete its install on two separate occasions. PostgreSQL was the worst performer and MySQL, while good for the smaller datasets we threw at it (~10 GB), would not have been performant enough even if it maintained linear scalability with the larger dataset. We also looked at distributed memory solutions like those provided by GigaSpaces and Terracotta however we no longer needed to distribute the data, so they weren’t suited.

After switching away from traditional SQL databases, we looked at a number of column databases including:

However these were either unsupported and outdated (C-Store), core-dumped when queried (two versions of MonetDB, both source & binaries) or were not feature rich enough for our needs at the time (HBase).

We also briefly looked at commercial column stores such as:

In the end we ruled these out also as we were starting to think we could easily solve our problem using a plain vanilla filesystem and some simple parsing code, and we didn’t want to deal with the sales teams at these companies (I’ve personally been there before and knew the process would not be pretty).

The data storage solution we came up with was laughable simply. We stored CSV files on a filesystem (ext3 on our dev boxes) and indexed into the data using directories. As we had to search linearly (table scan equivalent) through some of the CSV files we split these files into chunks. We were thus able to distribute the requests based on chunk to each of our nodes. The downside of this approach was that the chunking was a manual process, we had to select the chunk size (31 initially) up front, there was no dynamic indexing. The upside was that we didn’t need to maintain the indexing scheme (using B or AVL trees) and the files could be batch updated with only a small configuration change (number of chunks). We’ve found that our data storage was extremely performant, contributing only 18% to the total time it takes to process a request, the remainder of the time is spent in analysing the data and distribution overhead. Subsequent conversion of the CSV files to a binary format result in even faster processing as we no longer need to parse out the CSV delimiters.

Our initial performance tests of the core algorithm showed that we’d need to distribute the computation (and potentially the data) in order to achieve our “real-time” goal. Based on this we looked at languages such as Erlang and Scala for their Actor concurrency model, both languages also have ways of distributing work across multiple machines. Although our algorithm (and data) was parallelisable, we didn’t feel that it was amenable to a high level of concurrency, and our research showed us that we could build the system using Java-based tools. This along with the fact that none of the developers were proficient with either Erlang or Scala swayed us back towards Java (even considering all of its problems). Workingmouse does perform research around functional languages including the newly developed Scalaz library, however at the time only one of our guys was highly proficient in Scala and we were on a very tight time line. The risk associated with four out of five in the team learning a new language and supporting tools was thought to be too great.

Now that we had a candidate language, we looked at a bunch of Java-based “grid” or parallel computing frameworks including:

These tools perform different functions, some distribute only the data and some distribute only the computation. By now we were fairly sure that we only needed to distribute the computation, so technologies such as Coherence, Terracotta and GigaSpaces were not appropriate. During this time we’d been prototyping simple algorithms using Hadoop and GridGain. We’d found GridGain very easy to get started and performance tests showed that it added only 10-200 ms to the overhead of calls when compared to local (in-JVM) invocations. Hadoop was promising, however it was also quite complicated to setup (installation and creation of readers for input) and use, and provided components that we didn’t need, such as the distributed filesystem. We were also unsure how suited it was for interactive use, it looked to be optimised for long running batch operations. In the end, the ease of use and flexibility of GridGain made it a good candidate for the initial distribution, and continues to be used to date.

GridGain

GridGain has what it calls SPIs (service provider interfaces) which are a fancy way of saying an API whose implementation may be provided by a third party and which I thought had long since been relegated to the bad old days of XML parsers. These make it easy to swap in different implementations of node discovery or node-to-node communication for example. We used this to great effect when we discovered that Amazon’s EC2 did not support IP multicast, we easily swapped this out for a JGroups implementation, which has since been swapped out for a JMS-based implementation.

For its good points however, there are some downsides (I’m being quite picky here, it’s a really good framework to use and works quite well).

  • We initially had some issues with GridGain, such as jobs being lost and nodes being lost off the grid. To counter this, we went back and wrote a sample project that performs word counts, and encountered no issues, so these looked to be an issue with our code.
  • The error messages that come out of GridGain are horrible. They’ve attempted to clean up the stack traces, but have made it worse, as they’ve interrupted the normal flow of Java stack traces, and interspersed it with marketing and documentation material. This makes reading errors quite hard.
  • The code itself is quite poor, we had lots of trouble following the flow of execution from the startup script to discover how nodes are actually started. This complicated our understanding of the system and its nomenclature. Most methods are huge, tens of lines long with nested try-catch blocks with variables initialised outside the try-catch block and later used. I know this is normal coding practice for Java developers but there are much better and cleaner ways to handle this. The whole code base needs a good refactor. I actually think this is a reflection on their non-test driven approach to development. I read a post on InfoQ regarding TDD from one of the developers, who said they’d tried it but basically given up. This reflects in poor design and tight coupling of the classes making them hard to use outside their original context (see Spring example below).
  • The documentation while quite extensive, does not cover things such as the architectural overview, what a “grid” is (it’s a node), how the queueing mechanism works, etc. All stuff you’re going to need if you use GridGain in anger.
  • While GridGain provides SPIs, if you need to change the way GridGain works outside of these SPIs, it’s not very easy to do so. We encountered two instance of this. Firstly, we found the built in JBoss serialisation orders of magnitude slower than the default Java 1.6 serialisation for the objects we were sending (lists of simple structures containing 2-3 integers) and it produced an order of magnitude more bytes on the stream. There doesn’t appear to be a way to replace this without re-implementing a lot of code. Secondly, when we changed to JGroups for discovery, we needed to pass the master node (our single node that submits jobs to the grid) a JGroups configuration file, which must to be an absolute path. As we were running inside a servlet container we couldn’t be sure the webapp had been exploded to the filesystem, and even if we were exploded we also had no easy way of knowing (from a Spring config file) where the file lived. Paths are not resolved relative to the Spring config file but relative to the GridGain home directory. This meant we had to ship a JGroups config file in a known location, outside of our normal distribution package (the WAR file). This file must also be a real file on the filesystem, you can’t pass a stream or a resource on the classpath.
  • Because the documentation was sparse, we spent a lot of time proving theories about how the “grid” behaved under certain circumstances to try to get an understanding of how it all fitted together: Can we run multiple nodes that submit jobs to the same grid (yes), does it load balance these (unsure), what constitutes a “grid” (it’s the discovery SPI, whatever other nodes a node can see are the “grid”), can we run multiple nodes in the same VM (no, though some documentation claims you can), how does the queueing work, what happens if we saturate the grid with jobs, does GridGain pre-allocate jobs to nodes or can it take advantage of new nodes coming up after a task has been split (we don’t think it can, but are unsure), can nodes participate on more than one “grid”, etc.
  • As GridGain uses serialisation to send objects across the wire, everything must be Serializable. If it’s not, you’ll get obscure errors which are hard to trace back to serialisation issues. We ended up writing a test utility to ensure that classes we expected to be Serializable (the task, job and job params) were. We also used FindBugs in the build to ensure we didn’t miss any inadvertently.
  • We needed to be able to submit a number of tasks to the grid concurrently (where each task was a user request on the Rails app). Our initial tests showed that this bombarded the nodes with jobs, until they were unable to cope, causing the master node to fail over and eventually fail (as there were no nodes “available”). There is an SPI that partially addresses this (the collision SPI), however it only addresses “collisions” (i.e. messages arriving at the same time) on the consumer end (the processing/worker node) of the connection. There does not seem to be a way to batch up messages on the master (producer) node. This becomes a problem when the code submitting tasks to the grid runs concurrently (like a web service receiving multiple requests). We hadn’t needed to address this yet so didn’t look too hard for GridGain solutions, but other possibilities include rolling your own queue (perhaps via java.util.concurrent) or batching up requests on the Rails side. This also has affects on the architecture of your system. As GridGain (seems to) sends the jobs out as soon as they’re split, only nodes that are available at the time of the initial send are available to participate in the task. So if a node fails and another comes online, it does not seem to pick up the jobs, increasing the overall processing time of the task.
  • GridGain includes a peer class loading facility, which basically means whenever a class is needed by the JVM, the classloader looks in the local classpath first, and if not found, will pull the class of any other nodes in the grid that have it available (caching it locally). This is god for development, where you can make a change to your job or job parameters class and have them automatically re-synced to all the nodes. However we were having issues with class loading (which turned out to be serialization and keeping old classes in the classpath) and wanted to turn this off. Although the documentation claims it can be disabled, we couldn’t get our grid to work without it on.
  • Submission of a task requires the task class, you cannot give it an instance, which implies a no-args constructor on the task class. This is a bit odd.
  • This isn’t really an issue, but the IP multicast stuff works too well locally. It’s great to get up and going but you can easily throw jobs onto nodes that you didn’t intend to. The ability to integrate it into an automated build also suffers because of this. We ended up using custom multicast address per local developer machine (auto-generated from the machine’s IP). Other discovery SPIs should be similarly configurable, JGroups & JMS can use localhost for example.
  • And lastly, but perhaps worst of all, the nomenclature is all wrong. The entity GridGain calls a “grid” is really a “node” on the grid. This confused us for a couple of weeks, and caused us to incorrectly name the “grid” with a single name, when in fact all you are doing is naming nodes. We spent the time talking about the “conceptual grid”, where nodes all communicated based on the “grid” name. This had an impact on our initial architecture, whereby we thought we could have nodes participating in more than one grid at the same time. This was appealing to us as we basically had two different kinds of requests, and we thought we may be able to dynamically partition our nodes into either RequestA grid or RequestB grid. This was not the case, the “grid” is defined by the discovery SPI not the “grid” (node) name.

We also encountered issues with the discovery SPI, initially driven by EC2 not supporting IP multicast, and later by issues with JGroups. With our configuration of JGroups, we found that with larger grids (8 or 16 nodes), often the master node wouldn’t discover every other node available. We could often rectify the situation by starting our nodes in a specific order (all processing nodes, then the master node) – but we would still occasionally miss a grid node or two. What was more worrying was that with long-running CPU-intensive tasks some of the nodes would drop off the grid (according to the master node’s logs). We probably would’ve persisted with JGroups even with this problem, except that once a node dropped off the grid it was never re-discovered by the master node. Eventually the grid would dwindle to our single non-processing node (our master node was non-processing) and fall over. Because of this we ended up swapping the JGroups discovery implementation out for a JMS-based one. Given we already had a JMS service running inside the application server, switching to it was fairly painless. Grid discovery with JMS seems to work just as well as IP multicast, and there does not appear to be anymore overhead on our processing times.

Results

After we’d found an initial working solution it was time to tweak it. We’d actually been doing this all along, however we now had a couple of solid weeks to spend solely on performance tuning. We had two lines of attack here, firstly we looked at tuning the JVM and its garbage collector configuration (I’d had great success with this in the past), and secondly we looked at profiling the code for CPU hotspots and memory usage. Our biggest wins turned out to be reducing the memory overhead and optimising our hashCode() and equals() implementations. Things we thought would hold us back (before profiling) turned out to be not that big an issue at all, our text-based CSV reading for example was contributing only around 2% to the overall processing time.

Part of our algorithm called for loading 3.1 million objects into memory (and retaining them for fast lookup). Each of these objects implemented a base class (Primordial) which gave us sensible (field-based) toString(), hashCode() and equals() methods. This greatly simplified development, but meant that for every object, we also had an instance of Primordial and its parent Object also in memory, giving us 9.3 million objects. Primordial also had two instance fields that delegate to provide equality and to string capabilities, giving us another 6.2 million objects, 15.5 million objects in total, requiring more than half a GB of RAM. By moving these instance fields into our objects and making them static (class fields) we reduced the object count to 6,200,002 objects and significantly improved the memory and CPU characteristics of the application as we were no longer letting the garbage attempt to reclaim space it couldn’t.

Our algorithm also makes heavy use of maps and sets, so our hashCode() and equals() methods were getting a workout. The Primordial implementation uses field values (obtained reflectively) to provide sensible defaults for these methods. While this is normally fine (we used this approach in a large batch processing application for several years with no problems), for this algorithm it proved an issue. By writing custom implementations of hashCode() and equals() for the two or three objects that required it we were able to drop our processing time by a fifth (from 25 seconds to 5 seconds).

I should note here that we didn’t perform any premature optimisations, we profiled with real requests against real data, giving us the true problems not the things we thought would be problems. In all we spent around a week and a half tuning and were able to decrease the total process from around 50 seconds to around 1 second (for a single node on the local machine). We were lucky however in that we always had performance at the forefront and had chosen a good architecture, one that didn’t need to change following performance testing (I had thought we’d need to do some radical rework to achieve desired performance).

As a sample of the numbers we were getting, here is a performance graph that comes from us trying to determine how the number of nodes affects processing time. These numbers were measured before we’d done the performance optimisations mentioned above, the current system is around 5 times faster than this. Processing time in milliseconds is on the Y-axis and the number of GridGain nodes is on the X-axis, the results for 31 nodes are extrapolated.

Grid performance results over multiple nodes

The red and blue lines show the average processing time as the number of nodes is increased (averaged across 5 and 20 requests respectively). For a single node the system performs the query in 30 seconds, for 16 nodes the system takes 7 seconds.

The green line (time delta) shows the improvement we get in going from N nodes to N + 1 nodes. For a small number of nodes this is very significant – going from 2 to 3 nodes reduces the total time by 9086 ms – however becomes less significant as more nodes are added. Going from 8 to 11, 11 to 16 and 16 to 31 nodes only drops the processing time by 1243 ms, 1255 ms and 1219 ms respectively. So for this test, almost doubling the number of nodes from 16 to 31 (our maximum number of nodes for our given chunk size discussed above) only improves processing time by one second. At some point you approach a limit where adding more nodes does not decrease (single-request) processing time significantly. It may however provide more resilience so that the grid can handle more concurrent requests and cope better with failure.

The yellow line shows the theoretical performance we should be getting by increasing the number of nodes (assuming no network etc. overhead). If we average the total overhead out across the all the nodes, as we increase the number of nodes this blows out from 95 milliseconds for 3 nodes to 210 milliseconds for 16 nodes. This overhead corresponds roughly to the GridGain overhead we saw in our GridGain sample project.

Deployment

After we’d developed a workable solution we needed a hosting environment. The guys we were working with were pretty keen not to have to host the grid themselves and were looking at Amazon’s EC2 for a low cost easy ramp up solution. The process of setting up nodes was pretty painless; we had registered, set up a node and documented the issues in about a day.

The only real issues we had with EC2 is that it doesn’t support IP multicast so we had to change the way our nodes found each other, IPs are also not static so management of clusters could become unwieldily. We also had issues with the IP address the machine thought it was on (the internal IP) and its external address, which meant code that returned the address of the server (the WSDL generation) returned the wrong (internal) address to external clients.

There’s issues with persistent data (there is none) so you need to store anything you want somewhere else like S3. Instances boot up from images (AMIs) so you can keep stuff there also, but it doesn’t get saved when the instance goes down, and there’s a limit on the amount of data you can store. We used S3 to store our data and were planning on automating the copying of the data across to nodes on boot. The EC2 VMs are quite nice, instances are easy to manage and work as advertised. The small instance can be a bit slow, we found them slower than our desktop machines, but the large instances are very nice, and 64-bit.

To ease deployment, we used Capistrano to deploy our code, start and stop our nodes and the app server. Some of the guys also wrote scripts to automate the booting of our images, returning the (dynamic) IPs for SSH access.

Conclusions

Based on our work, here are a few items to consider in summary.

  • Conventional SQL databases are not the solution to all data storage problems, often, you can do better using just the filesystem, if you don’t require a general solution.
  • Because of their nature current Map/Reduce frameworks (i.e. Hadoop) may not be appropriate for all problems. If you don’t need a distributed filesystem, there might be other solutions.
  • Discover how your application behaves and what it needs from a grid. Does it need to be distributed? If so, do you need a computation grid or a data grid?
  • Beware of the vendors marketing spin.
  • If you go with GridGain, invest the time to learn how it works, especially for your problem space. Choose the size of your split (how much work gets done on a node) appropriately. The longer a job runs, the more chance a failing job has of delaying the overall processing time as it may fail at the end of the job, requiring a resend to another node. Hadoop seems to cater for this by allowing idle nodes to pick up work not yet allocated.
  • Keep your nodes as consistent as possible, if you can keep them exactly the same all the better, it eases deployment and management of nodes.
  • Get an end-to-end solution going as soon as possible, it’ll flush out lots of issues.
  • Automation is a good thing.
  • Small teams can achieve a lot in a short period of time.
  • This is the first project I’ve used the XP idea of a metaphor on (thanks Andrew!) and it worked really well. Ours was “lightning”, we wanted the application to be small, simple and lightweight. Requests should flow in and out of the system as fast and simply as possible. This guided technology choices and design decisions.
  • Premature optimisation is a bad thing. Keeping your code well structured will aid in refactoring it if and when you meet real problems, not perceived ones.

Video

Nick Partridge and I gave a talk on this topic at the February 2008 Queensland Java User’s Group, a video of this presentation is available here: Off the Grid – Introduction to Grid Computing using GridGain.

Written by Tom Adams

March 30th, 2009 at 9:03 pm

Posted in Agile,HPC,Instinct,Java

Handling nulls

without comments

I got some feedback to my Error handling with Either, or, Why Either rocks! post, basically saying it was incomprehensible.

So, here’s a nice introductory post on Option: Handling nulls. Should be a gentler introduction.

Written by Tom Adams

August 21st, 2008 at 11:42 am

Deploying a Maven artefact without a repository using Ant

without comments

You know the drill, you’re working on an open source project (or two), and users start demanding Maven support. Why you ask yourself? Why, would anyone want to use Maven. But then, they assure you that it’s not by choice. That yes, they know Maven sucks, that it makes the simple hard and moderate impossible. But they’ve been forced to use it, and they’d like help. Now you know Ant has problems, but Maven? Seriously?

Seriously. So you decide to add in Maven support. After all, you’re running the project for the people, and having Maven support would help adoption, so it’s all good (and you get to write lame blog posts too!).

Now here’s where the fun begins…

The problem is that you’re using a tool other than Maven to build the project(s), and you have no Maven repository into which to deploy your project artefacts. The example I’ll show is using Google Code, but this could be any Subversion service, the process should be similar – with the exception of the SVN commands – for exposing a repository over HTTP. I’m also using Ant as the build tool, again, this should be replicable in other tools.

To save you the hassle of find them yourself, here’s the two links you’ll need. Firstly, here’s the Guide to uploading artifacts to the Central Repository, and secondly, here’s the Maven Deploy Plugin Usage overview. Be warned though, like most things Maven, this documentation is pretty scarce, for example what URL schemes does the deploy-file plugin take? Will it accept svn://... URLs? Who knows, but this is what worked for me.

All in all, the process is fairly simple once you figure it all out. The thing we’ll be building is called a Maven Bundle, and is simply a jar (zip) file containing certain content. We’ll then “deploy” this into a local “repository”, creating the necessary directory structure.

  1. Ensure your project is creating three artefacts, a main jar, a sources jar, and a javadoc jar. The last two (source & javadoc) are optional, however they seem to be standard for most Maven bundles, and Maven aware tools (such as IntelliJ) will automatically link the source and javadoc jars to the main classes.

    As detailed in the guide, they need to be named as follows:

    ${artifactId}-${version}.jar
    ${artifactId}-${version}-sources.jar
    ${artifactId}-${version}-javadoc.jar
    

    Note that if your project also contains Scala code, and Scaladoc (as Functional Java & Instinct do), I’ve not looked into how to get these hooked up into the Maven bundle. Lift is using Maven so I guess it’s possible…

  2. Create a Maven POM file, the guide linked above shows you the minimum you’ll need to get started (there are rules in the guide, that tell you what you must and can’t have in a POM for deployment). Here’s Instinct’s POM:

    <?xml version="1.0"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelversion>4.0.0</modelversion>
        <groupid>com.googlecode.instinct</groupid>
        <artifactid>instinct-core</artifactid>
        <packaging>jar</packaging>
        <name>Instinct Core</name>
        <description>Instinct Behaviour Driven Development (BDD) Framework</description>
        <version>0.1.9</version>
        <url>http://instinct.googlecode.com/</url>
        <licenses>
            <license>
                <name>The Apache Software License, Version 2.0</name>
                <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
                <distribution>repo</distribution>
            </license>
        </licenses>
        <scm>
            <url>http://instinct.googlecode.com/svn/</url>
        </scm>
        <dependencies>
          ...
        </dependencies>
    </project>
    
  3. You’ll now need to build your bundle, the contents should look like this:

    pom.xml
    ${artifactId}-${version}.jar
    ${artifactId}-${version}-sources.jar
    ${artifactId}-${version}-javadoc.jar
    

    Here’s some Ant XML showing how this can be done:

    <property name="build.dir" value="build"/>
    <property name="release.dir" value="${build.dir}/release"/>
    <property name="maven-bundle.jar" value="instinct-0.1.9-bundle.jar"/>
    <target name="-maven-bundle" depends="-jar,-sources-jar,-javadoc">
        <mkdir dir="${release.dir}"/>
        <jar destfile="${release.dir}/${maven-bundle.jar}">
            <fileset dir="${basedir}" includes="pom.xml"/>
            <fileset dir="${build.dir}" includes="${project.jar}"/>
            <fileset dir="${build.dir}" includes="${sources.jar}"/>
            <fileset dir="${build.dir}" includes="${javadoc.jar}"/>
        </jar>
    </target>
    

    And here’s what it builds:

    $ unzip -l build/release/instinct-0.1.9-bundle.jar Archive:  build/release/instinct-0.1.9-bundle.jar
      Length     Date   Time    Name
     --------    ----   ----    ----
            0  08-08-08 22:48   META-INF/
           98  08-08-08 22:48   META-INF/MANIFEST.MF
         2136  08-08-08 12:36   pom.xml
       378565  08-08-08 22:48   instinct-0.1.9.jar
       315685  08-08-08 15:28   instinct-0.1.9-sources.jar
       693521  08-08-08 22:48   instinct-0.1.9-javadoc.jar
     --------                   -------
      1390005                   6 files
    
  4. Create a spot in your svn for the maven artefacts, this will become your “repository”. I added mine here: http://instinct.googlecode.com/svn/artifacts/maven/.
  5. Take a checkout of this directory, you’ll need to store it somewhere local to the project, you’ll need its path later. I checked mine out into a peer directory to my project, so from the project it’s accessed as “../artifacts/maven”. This will be a local version of your repository.
  6. You’ll now need to “deploy” your classes jar into this local repository, as follows (documented in the deploy plugin usage):

    $ mvn deploy:deploy-file -Durl=file://../artifacts/maven -DrepositoryId=local-svn-artifacts -Dfile=instinct-core-0.1.9.jar -DpomFile=pom.xml
    

    I tried to get the deploy-file goal to deploy to a remove SVN directly with no luck, YMMV. Again, the documentation is lacking here.

    Note that the deploy-file goal does not accept bundles, it needs to be just your classes jar. If you want your sources and javadoc there as well, you’ll need to copy and hash them also (see Ant target below).

    Alternatively, you can have your build tool do this for you:

    <property name="maven-bundle.jar" value="${project.shortname}-${project.version.full}-bundle.jar"/>
    <property name="maven-repo.dir" value="${basedir}/../artifacts/maven"/>
    <property name="maven-repo-release.dir" value="${maven-repo.dir}/com/googlecode/instinct/instinct-core/${project.version.full}"/>
    <exec dir="${basedir}" executable="mvn" failonerror="true" os="Mac OS X,Linux">
      <arg value="deploy:deploy-file"/>
      <arg value="-Durl=file://${maven-repo.dir}"/>
      <arg value="-DrepositoryId=local-svn-artifacts"/>
      <arg value="-Dfile=${build.dir}/${project.jar}"/>
      <arg value="-DpomFile=${basedir}/pom.xml"/>
    </exec>
    <copy file="${release.dir}/${maven-bundle.jar}" todir="${maven-repo-release.dir}"/>
    <copy file="${build.dir}/${sources.jar}" todir="${maven-repo-release.dir}"/>
    <copy file="${build.dir}/${javadoc.jar}" todir="${maven-repo-release.dir}"/>
    <checksum file="${maven-repo-release.dir}/${sources.jar}" algorithm="MD5" forceOverwrite="yes" fileext=".md5"/>
    <checksum file="${maven-repo-release.dir}/${sources.jar}" algorithm="SHA" forceOverwrite="yes" fileext=".sha1"/>
    <checksum file="${maven-repo-release.dir}/${javadoc.jar}" algorithm="MD5" forceOverwrite="yes" fileext=".md5"/>
    <checksum file="${maven-repo-release.dir}/${javadoc.jar}" algorithm="SHA" forceOverwrite="yes" fileext=".sha1"/>
    
  7. Check the structure of the local repository, it should look something like this:

    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.jar
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-javadoc.jar.sha1
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-sources.jar
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.jar.md5
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-javadoc.jar
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-sources.jar.md5
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-bundle.jar
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-javadoc.jar.md5
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.jar.sha1
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.pom
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.pom.md5
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9.pom.sha1
    maven/com/googlecode/instinct/instinct-core/0.1.9/instinct-core-0.1.9-sources.jar.sha1
    maven/com/googlecode/instinct/instinct-core/maven-metadata.xml
    maven/com/googlecode/instinct/instinct-core/maven-metadata.xml.md5
    maven/com/googlecode/instinct/instinct-core/maven-metadata.xml.sha1
    
  8. If you’re happy with the local repository, commit it into your (svn) repository. Instinct’s 0.1.9 release was deployed into this URL: http://instinct.googlecode.com/svn/artifacts/maven/com/googlecode/instinct/instinct-core/0.1.9/.
  9. You’re now done with the technical side of building a bundle. You’ll then need to log a JIRA issue for the Maven developers to upload your bundle (following the instructions in the guide). Then wait, and hope… Here’s the Instinct 0.1.9 request.

There is another way that users can download your Maven artefacts without them needing to be deployed to the central Maven repository, it’s detailed in the introduction to Maven repositories. Basically client projects of your project need to add a repository to their POM as follows (substituting the ID and URL belong for that of your repository):

<project>
  ...
  <repositories>
    <repository>
      <id>instinct-repository</id>
      <url>http://instinct.googlecode.com/svn/artifacts/maven/</url>
    </repository>
  </repositories>
  ...
</project>

Note that the process I’ve described of uploading to the central repository process is a manual one. Looking at the Maven issues for bundle uploads, it appears that if you don’t provide rsync or ssh access to your repository, you need to make manual upload requests. Others who’ve followed a similar process to this one are asking whether the Maven sync can happen automatically, to which I’ve not seen an answer. The documentation isn’t overly clear on this last point. A friend of mine asked on the Maven IRC channel yesterday and was told syncing over HTTP “just works”, though this isn’t evident by the details in the guide, the issues being reported nor the list of automatically synced repositories.

For those interested, these instructions were gleaned from looking at other projects that deploy Maven artefacts, including GWT-Maven and the Struts 2 Maven Plugin repository.

Written by Tom Adams

August 11th, 2008 at 8:25 am

Posted in Functional,Instinct,Java

Tagged with , ,

Instinct 0.1.9 Release

without comments

I’m happy to announce the release of Instinct 0.1.9. This is a maintenance release that also allowed me to introduce some new features such as custom lifecycles. Other tidbits include multiple specification failures reported, Scala examples, additional Functional Java matchers (List, Either, Option), Maven bundle and a seperate source JAR.

Downloads are available from the project site.

Here’s the full list of changes:

  • Core Features
    • Custom specification lifecycles can now be plugged in using the @Context annotation. This would allow for example, a Spring application context to be initialised and wired into each specification method allowing integration-style specifications using a Spring beans.
    • Multiple errors are now displayed when a specification fails with more than one exception.
  • Expectation API
    • Added fj.data.List matchers: expect.that(List.nil().cons(1)).isEqualTo(List.nil().cons(1))
    • Added fj.data.Option matchers: expect.that(Some(1)).isSome(1), expect.that(foo.bar()).isNone().
    • Added fj.data.Either matchers: expect.that(error()).isLeft(runtimeException), expect.that(foo.bar()).isRight(42).
  • Scala integration
    • Added Scala stack example, showing the use of Instinct under Scala to spec out Scala code.
  • Maven integration
    • Added Maven bundle to allow uploading to central Maven repository.
  • Packaging
    • Moved source code out of main jar into instinct-<version>-sources.jar (for Maven).
    • Moved javadoc into jar instead of zip instinct-<version>-javadoc.jar (for Maven).
  • Infrastructure
    • Upgraded to Functional Java 2.10.
    • Downgraded (again) to CGLib 2.1.3 (for Maven integration).
  • Bugs
    • (Issue 21) Expected exception try-catch should only wrap specs, not before and after.
    • (Issue 19) Throwing exception out of mocked code confused expected exception handling.
    • (Issue 40) Formatting of multiple exception report could be nicer.

Written by Tom Adams

August 8th, 2008 at 4:10 pm

Posted in BDD,Instinct,Java,Scala

Error handling with Either, or, Why Either rocks!

with 3 comments

Instinct, like most xUnit-like frameworks provides the ability to run methods, and have the status of those methods reported. In xUnit frameworks these methods are called tests, in Instinct they’re called specifications.

Specifications are ordinary instance methods that are marked in a way (naming convention or annotation) that tells Instinct to run them. Each specification has a lifecycle associated with it, where both the creator of the method (the developer specifying code) and the framework itself performs pre- and post-specification steps (Instinct tries to take away and simplify a lot of the drudgery involved in traditional testing).

Specifications have the following lifecycle (the default implementation can be overridden):

  1. The context class for the specification is created.
  2. The mockery is reset, all mocks now contain no expectations.
  3. Specification actors are auto-wired.
  4. Before specification methods are run.
  5. The specification is run.
  6. After specification methods are run.
  7. Mock expectations are verified.

Any step of this lifecycle can throw exceptions causing the specification to fail. For example a before specification method may throw a NullPointerException, or a specification may pass while a mock used in it may not have an expectation met.

The framework needs flexibility in choosing which parts of the lifecycle to run, which parts are important when executing the specification, what failures constitute stopping the run of a specification, etc.

Here’s the code we’re starting with:

private SpecificationResult runSpecification(final SpecificationMethod specificationMethod) {
  final long startTime = clock.getCurrentTime();
  try {
    final Class<?> contextClass = specificationMethod.getContextClass();
    final Object instance = invokeConstructor(contextClass);
    runSpecificationLifecycle(instance, specificationMethod);
    return createSpecResult(specificationMethod, SPECIFICATION_SUCCESS, startTime);
  } catch (Throwable exceptionThrown) {
    final SpecificationRunStatus status = new SpecificationRunFailureStatus(exceptionSanitiser.sanitise(exceptionThrown));
    return createSpecResult(specificationMethod, status, startTime);
  }
}

private void run(final Object contextInstance, final SpecificationMethod specificationMethod) {
  Mocker.reset();
  actorAutoWirer.autoWireFields(contextInstance);
  try {
    runMethods(contextInstance, specificationMethod.getBeforeSpecificationMethods());
    runSpecificationMethod(contextInstance, specificationMethod);
  } finally {
    try {
      runMethods(contextInstance, specificationMethod.getAfterSpecificationMethods());
    } finally {
      Mocker.verify();
    }
  }
}

This implementation of of the specification runner is overly simplistic. It runs everything within a large try-catch block, which means there’s no way to tell which part of the specification failed (before, spec, after, etc.). It also cannot collect up errors, so if an error occurs in a specification and a mock fails to verify, only the verification error is propagated. These are currently two of the highest priority user reported issues on Instinct.

Here’s my first attempt at isolating which part of the specification failed, each of the constants passed to fail define the location of the failure.

public SpecificationResult run(final SpecificationMethod specificationMethod) {
  try {
    final Class<?> contextClass = specificationMethod.getContextClass();
    final Object instance = invokeConstructor(contextClass);
    Mocker.reset();
    try {
      actorAutoWirer.autoWireFields(instance);
      try {
        runMethods(instance, specificationMethod.getBeforeSpecificationMethods());
        try {
          runSpecificationMethod(instance, specificationMethod);
          return result(specificationMethod, SPECIFICATION_SUCCESS);
        } catch (Throwable t) {
          return fail(specificationMethod, t, SPECIFICATION);
        } finally {
          try {
            try {
              runMethods(instance, specificationMethod.getAfterSpecificationMethods());
            } catch (Throwable t) {
              return fail(specificationMethod, t, AFTER_SPECIFICATION);
            }
          } finally {
            try {
              Mocker.verify();
            } catch (Throwable t) {
              return fail(specificationMethod, t, MOCK_VERIFICATION);
            }
          }
        }
      } catch (Throwable t) {
        return fail(specificationMethod, t, BEFORE_SPECIFICATION);
      }
    } catch (Throwable t) {
      return fail(specificationMethod, t, AUTO_WIRING);
    }
  } catch (Throwable t) {
      return fail(specificationMethod, t, CLASS_INITIALISATION);
  }
}

Obviously, this is very ugly, it’s also hard to reason about. But, as we now have the location of the failure we can make decisions as to whether we fail the specification, or not, so we’ve solved our first issue. But we haven’t made our second task any easier, we aren’t generally able to keep processing (we still validate mocks in the above code upon specification failure) and we don’t collect all the errors that occur.

And at about this time enters Either (in Scala):

The Either type represents a value of one of two possible types (a disjoint union). The data constructors; Left and Right represent the two possible values. The Either type is often used as an alternative to Option where Left represents failure (by convention) and Right is akin to Some.

Either can be used in place of conventional exception handling in Java, or, to wrap APIs that use conventional exception handling (a more thorough treatment of this issue is given in Lazy Error Handling in Java, Part 3: Throwing Away Throws). Here’s an example of the latter, using both Either and Option (discussed later).

public Either<Throwable, List<Field>> wireActors(final Object contextInstance) {
  try {
    return right(actorAutoWirer.autoWireFields(contextInstance));
  } catch (Throwable t) {
    return left(t);
  }
}

...

public Option<Throwable> verifyMocks() {
  try {
    Mocker.verify();
    return none();
  } catch (Throwable t) {
    return some(t);
  }
}

At a high level, the good thing about using Either is that your methods no longer lie; they don’t declare that they’ll return an Int, or, maybe, they’ll throw an exception, they come right out and say it: I’ll return either an exception or an Int. This is akin to conventional checked exceptions in Java (which Scala does away with), where a checked exception is used to represent a recoverable failure (enforced by the compiler) and an unchecked exception to represent an unrecoverable failure (not compiler enforced). Scala takes the correct approach here, it uses unchecked exceptions to represent the bottom value in non-terminating functions, and Either to represent recoverable failure.

Either is also much more flexible than exceptions, you can map across it, convert it into an option, add them into a container, and generally treat them like any other data structure [1].

So armed with this new knowledge, here’s the new specification lifecycle broken out from the runner itself (note, there are eleven steps in the lifecycle, including validation, however only these are exposed).

interface SpecificationLifecycle {
  <T> Either<Throwable, ContextClass> createContext(Class<T> contextClass);
  Option<Throwable> resetMockery();
  Either<Throwable, List<Field>> wireActors(Object contextInstance);
  Option<Throwable> runBeforeSpecificationMethods(
      Object contextInstance, List<LifecycleMethod> beforeSpecificationMethods);
  Option<Throwable> runSpecification(
      Object contextInstance, SpecificationMethod specificationMethod);
  Option<Throwable> runAfterSpecificationMethods(
      Object contextInstance, List<LifecycleMethod> afterSpecificationMethods);
  Option<Throwable> verifyMocks();
}

Now we need to make use of this in the specification runner, one step of which is determining the overall result, from the sequence of steps. Here’s my first attempt at this, using Functional Java’s Either to represent the result of each of the steps.

public <T extends Throwable> Either<List<T>, SpecificationResult> determineLifecycleResult(
    final Either<T, Unit> createContextResult,
    final Either<T, Unit> restMockeryResult,
    final Either<T, Unit> wireActorsResult,
    final Either<T, Unit> runBeforeSpecificationMethodsResult,
    final Either<T, SpecificationResult> runSpecificationResult,
    final Either<T, Unit> runAfterSpecificationMethodsResult,
    final Either<T, Unit> verifyMocksResult) {
  List<T> errors = List.nil();
  if (createContextResult.isLeft()) {
    errors = errors.cons(createContextResult.left().value());
  }
  if (restMockeryResult.isLeft()) {
    errors = errors.cons(restMockeryResult.left().value());
  }
  if (wireActorsResult.isLeft()) {
    errors = errors.cons(wireActorsResult.left().value());
  }
  if (runBeforeSpecificationMethodsResult.isLeft()) {
    errors = errors.cons(runBeforeSpecificationMethodsResult.left().value());
  }
  if (runSpecificationResult.isLeft()) {
    errors = errors.cons(runSpecificationResult.left().value());
  }
  if (runAfterSpecificationMethodsResult.isLeft()) {
    errors = errors.cons(runAfterSpecificationMethodsResult.left().value());
  }
  if (verifyMocksResult.isLeft()) {
    errors = errors.cons(verifyMocksResult.left().value());
  }
  return errors.isNotEmpty() ? Either.<List<T>, SpecificationResult>left(errors)
      : Either.<List<T>, SpecificationResult>right(runSpecificationResult.right().value());
}

All those ifs are a bit ugly (what happens when we have more?), and we’ve got a mutable list, surely we can do better? We’ve spotted a pattern here, and we could clean this up by folding across a list of results, pulling out the left of each Either, however Either does this for us, using Either.lefts() (it performs the fold for you).

Here’s the next cut, making use of a list of results and Either.left():

public <T extends Throwable> Either<List<Unit>, Unit> determineLifecycleResult(
    final List<Either<T, Unit>> allResults, final Either<T, Unit> specificationResult) {
  final List<T> errors = lefts(allResults);
  return errors.isEmpty() ?
      Either.<List<T>, Unit>right(specificationResult.right().value()) :
      Either.<List<T>, Unit>left(errors);
}

So what’s this doing? It takes a list of results and goes through each of the lefts (the errors) returning them as a list. As Either is a disjunction (we’ll have an error or a result, but not both), if any of the results contain an error on the left, our list will be non-empty, meaning our specification failed to run. In this case we return the errors on the left. If we have no errors (i.e. the list is empty) we return the real result on the right.

This code can be simplified further by using Option instead of Either. Option would allow us to place any exception into the some data constructor, the Unit we’re placing into Either becomes the none (we’re used to thinking of void as nothing in Java anyway). The only hassle comes if we want to treat the Option as an Either (say in the lefts call above), in that case we’d need to lift the Option into an Either.

Option<Throwable> option = ...
Either<Throwable, Unit> either = option.toEither(unit()).swap();

Option also allows use to pull each some out of a list of Options, in a similar way to how we pulled the lefts out of a list of Eithers.

List<Option<Throwable>> results = ...
List<Throwable> errors = somes(results);
Option<Throwable> overall = errors.isEmpty() ?
    Option.<Throwable>none() :
    some((Throwable) new AggregatingException(errors));

Given that we’ve now decoupled the lifecycle from the runner and we know have a better way of handling errors, here’s the pattern of the new runner code:

private SpecificationResult runLifecycle(final long startTime,
    final SpecificationLifecycle lifecycle, final SpecificationMethod specificationMethod) {
  ...
  List<Option<Throwable>> lifecycleStepErrors = nil();
  final Either<Throwable, ContextClass> createContextResult =
      lifecycle.createContext(specificationMethod.getContextClass());
  lifecycleStepErrors = lifecycleStepErrors.cons(createContextResult.left().toOption());
  if (createContextResult.isLeft()) {
    return fail(...);
  } else {
    final ContextClass contextClass = createContextResult.right().value();
    ...
    lifecycleStepErrors = lifecycleStepErrors.cons(contextValidationResult.left().toOption());
    if (contextValidationResult.isSome()) {
      return fail(...);
    } else {
      ...
      lifecycleStepErrors = lifecycleStepErrors.cons(...left().toOption());
      if (...isSome()) {
        return fail(...);
      } else {
        ...
        if (...isSome()) {
          return fail(...);
        } else {
          ...
          if (...isSome()) {
            return fail(...);
          } else {
            ...
            if (...isSome()) {
              return fail(...);
            } else {
              ...
              return determineResult(..., lifecycleStepErrors);
            }
          }
        }
      }
    }
  }
}

See the pattern there? Let’s see it in slow motion. Assume each of the lifecycle results is called a, b, c, etc.

if (a.isLeft()) {
  return fail()
} else {
  if (b.isLeft()) {
    return fail()
  } else {
    if (c.isLeft() {
    } else {
      ...
    }
  }
}

What we’re doing is binding through each lifecycle result, if we get an error, we fail fast, if we don’t, we execute the next step. There’s some other muck going on here too, we’re destructively updating the list of errors (lifecycleStepErrors), and the last few steps (run the specification, run after methods, verify mocks) are always executed, regardless of whether any fail. So how do we clean the code up? We anonymously bind through Either on the right, and sequence through the rest accumulating errors. What???

Here’s a simple example that contains eleven steps representative of running a specification. For the first eight (a through h), each step’s predecessor must succeed (i.e. we have at most one error). For the last three (i through k), we execute all of them regardless of whether they fail and accumulate the errors. We make use of the new Validation class in Functional Java (in version 2.9) to perform the last three steps (full source; this example has been further refined in the trunk).

class X {
  // The first sequence of steps...
  Either<Throwable, Unit> a;
  Either<Throwable, Unit> b;
  Either<Throwable, Unit> c;
  Either<Throwable, Unit> d;
  Either<Throwable, Unit> e;
  Either<Throwable, Unit> f;
  Either<Throwable, Unit> g;
  Either<Throwable, Unit> h;
  // The second sequence of steps...
  Either<Throwable, Unit> i;
  Either<Throwable, Unit> j;
  Either<Throwable, Unit> k;

  // Execute the first sequence of steps, fail on the first error.
  Either<Throwable, Unit> t1() {
    return a.left()
        .sequence(b).right()
        .sequence(c).right()
        .sequence(d).right()
        .sequence(e).right()
        .sequence(f).right()
        .sequence(g).right()
        .sequence(h);
  }

  // Execute the second sequence of steps, accumulate the errors.
  Option<NonEmptyList<Throwable>> t2() {
    return validation(t1()).nel().accumulate(
        Semigroup.<Throwable>nonEmptyListSemigroup(),
        Validation.<Throwable, Unit>validation(g).nel(),
        Validation.<Throwable, Unit>validation(h).nel(),
        Validation.<Throwable, Unit>validation(i).nel());
  }
}

Each of the fields in the above represents the result of executing a step in the specification lifecycle (including validation, which is beyond the SpecificationLifecycle itself), t1 represents the first eight steps, t2 the last three steps. t1 sequences through (anonymous bind) the result of each step, failing if any individual step fails. t2 executes [2] each step, continuing execution of the remaining steps if any step fails, and accumulates the errors.

Remember that this is what t1 looked like originally:

if (a.isLeft()) {
  return fail()
} else {
  if (b.isLeft()) {
    return fail()
  } else {
    if (c.isLeft() {
    } else {
      ...
    }
  }
}

Some simpler examples may make the binding clearer; consider Scala’s Option (used here for brevity). We can bind through Option using orElse:

scala> Some(7).orElse(Some(8))
res0: Option[Int] = Some(7)

Here we execute Some(7), if that fails (i.e. returns none), we execute Some(8). As we see, the result is Some(7). Let’s take a failure case:

scala> None.orElse(Some(8))
res1: Option[Int] = Some(8)

We execute None, if that fails (i.e. returns none), which it does, we execute Some(8). As we see, the result is Some(8).

Taking it back to our simple Java example, we evaluate the result of step a [2], if it fails, we return the failure, if it succeeds, we evaluate step b, and so on. This is the same logic we saw in the nested if-else blocks earlier. If any of the first eight steps fail, we get back either one error (from t1), if any of the last three steps fail, we get back at most 3 errors (from t2)

If we apply this pattern to our specification runner code, we get the following:

private SpecificationResult runLifecycle(final long startTime, final SpecificationLifecycle lifecycle,
    final SpecificationMethod specificationMethod) {
  final Either<Throwable, ContextClass> createContext = lifecycle.createContext(specificationMethod.getContextClass());
  if (createContext.isLeft()) {
    return fail(startTime, specificationMethod, createContext.left().value(), false);
  } else {
    final ContextClass contextClass = createContext.right().value();
    final Either<Throwable, Unit> validation = validateSpecification(contextClass, specificationMethod);
    if (validation.isLeft()) {
      return fail(startTime, specificationMethod, validation.left().value(), false);
    } else {
      return runSpecification(startTime, lifecycle, contextClass, specificationMethod);
    }
  }
}

That looks bit better, but where’s the complexity gone? OK, here it is…

private SpecificationResult runSpecification(final long startTime, final SpecificationLifecycle lifecycle, final ContextClass contextClass,
    final SpecificationMethod specificationMethod) {
  final Object contextInstance = constructorInvoker.invokeNullaryConstructor(contextClass.getType());
  final Validation<Throwable, Unit> preSpecificationSteps =
      validate(resetMocks().f(lifecycle)).sequence(validation(wireActors().f(lifecycle, contextInstance)))
          .sequence(validate(befores().f(lifecycle, contextInstance, contextClass.getBeforeSpecificationMethods())));
  if (preSpecificationSteps.isFail()) {
    return fail(startTime, specificationMethod, preSpecificationSteps.fail(), Option.<Throwable>none());
  } else {
    final Option<Throwable> specification = specification().f(lifecycle, contextInstance, specificationMethod);
    final Option<NonEmptyList<Throwable>> result = preSpecificationSteps.nel().accumulate(throwables(), validate(specification).nel(),
        validate(afters().f(lifecycle, contextInstance, contextClass.getAfterSpecificationMethods())).nel(),
        validate(verifyMocks().f(lifecycle)).nel());
    if (result.isSome()) {
      return fail(startTime, specificationMethod, result.some().toList(), specification);
    } else {
      return success(startTime, specificationMethod);
    }
  }
}

Here’s the complete old and new versions of the code if you’re so inclined…

This code combined with the extracted lifecycle class is functionally equivalent to the first snippet of code I presented above. It may look verbose (it would be much simpler in Scala for example), but an interesting thing came out of it; it made explicit a bunch of places where I wasn’t handling exceptions correctly. It forced me to make a decision as to what to do in each case, so I got a much finer grained exception handling mechanism. Of course, I could get the same using try-catch (arguably more verbose), and I can choose to ignore left results (errors) if I want. The other thing it highlights is Java’s woeful generics implementation [3].

When I started down this path to error handling I had two objectives (two reported issues to resolve); to allow the runner of a specification to know which parts failed (this gives the flexibility to allow before specs to error and not be reported as expected exceptions) and to return all the errors resulting from running a specification. I didn’t initially intend to go down this path, however after talking in the office, decided that there was a better way to handle this than nested try-catch blocks. The resulting code is smaller (even using Java), simpler and much more flexible than the traditional Java method of exception handling. A win all round. There are some downsides however, firstly, the verbosity of Java’s typing leads to a mess of Either<Throwable, Unit> and this method of error handling will be foreign to a lot of Java developers today.

Epilogue

The concept of sequencing while accumulating errors has been generalised in Functional Java (from version 2.9) as validation, here is an example of it in action. Scalaz contains a similar concept, though this uses applicative functors over higher kinds (something which Java’s type system does not support), here’s a small example of its use.

Footnotes

  1. There’s no reason why you couldn’t catch an exception and wrap it up in a list, or your own data structure instead of using Either, but most of the work is already done for us (i.e. useful functions across Either have been defined) and it’s a useful convention.
  2. Java is strict, so we don’t get the benefit of lazy evaluation in this case, but could emulate it with a function.
  3. The problems I’ve encountered mainly have to do with covariance in type parameters and differences between the way methods returning types are treated vs. local variables (see the bottom of StandardSpecificationLifecycle for details).

Written by Tom Adams

August 6th, 2008 at 2:27 pm

Testing in Scala using a Java tool

without comments

My first post on Graceless Failures, a blog about adventures on the path to learning Scala:

Scala, like a lot of other languages these days, ships with a unit testing framework – SUnit – built in. Many other Scala specific “testing” frameworks have sprung up in recent times that contain similar or vastly different feature sets to the traditional xUnit tools. These include Reductio, ScalaCheck, Specs, ScalaTest, and SUnit (built into the Scala distribution).

And as Scala is “just Java” you can also use Java frameworks such as JUnit and TestNG. Having only used Reductio, I can’t vouch for any others, though ScalaTest is getting good airplay on Artima and Specs seems to have the Scala BDD mindshare.

These tools can be loosely categorised as traditional unit testing tools, ala xUnit, or automated specification testing tools, ala QuickCheck. Reductio and ScalaCheck are incarnations of automated specification testing, while Specs, ScalaTest and SUnit are more your traditional xUnit frameworks.

However, I’m not to write about any of these frameworks, instead, I’m going to write about Instinct, a Java BDD framework that I’ve been developing for around 18 months, and for which I’ve recently started to add specific support for Scala into the codebase. Good fodder for blog posts!

Continue reading Testing in Scala using a Java tool.

Written by Tom Adams

July 29th, 2008 at 9:43 am

Instinct 0.1.8 Release

without comments

I’m happy to announce the release of Instinct 0.1.8. This is a maintenance release resolving some minor bugs and packaging issues from the 0.1.7 release.

Chris has been spiking some Spring integration and needed a way to turn off detection of specs based on naming conventions and annotations so this is in this release also. Currently it’s only available via the @Context annotation, so you’ll have to annotate each class you want to override the default behaviour on. If there’s enough interest in this, I’ll add a way to change this globally.

Downloads are available from the project site.

Here’s the full list of changes:

  • Core Features
    • Custom specification, before specification and after specification annotations and naming conventions can be provided via the Context annotation. This can also be used to turn off naming convention-based detection (using NoNamingConvention).
  • Infrastructure
    • Upgraded to: Functional Java 2.8.
  • Bugs
    • (Issue 36) Abstract classes are being run as a result of specification runner refactoring.
    • (Issue 37) Expected exceptions are being printed to console in Ant runner even though spec passes.
    • (Issue 38) Functional java jar is not included in release zip.

Written by Tom Adams

July 26th, 2008 at 2:22 pm

Posted in BDD,Instinct,Java

Nobody *ever* uses map, et. al… (example 3)

without comments

Continuing my nobody *ever* uses series, here’s a Java example.

Consider the following horrible piece of Java code taken from Instinct which finds specifications to run:

private Collection<LifecycleMethod> findMethods(final MarkingScheme markingScheme) {
    final Collection<LifecycleMethod> lifecycleMethodSet = new HashSet<LifecycleMethod>();
    final Collection<Method> methods = methodLocator.locateAll(contextType, markingScheme);
    for (final Method method : methods) {
        lifecycleMethodSet.add(new LifecycleMethodImpl(method, contextType));
    }
    return lifecycleMethodSet;
}

Here’s the same method rewritten using Functional Java, with the conversion function pulled out for clarity:

private Collection<LifecycleMethod> findMethods(final MarkingScheme markingScheme) {
    final F<Method, LifecycleMethod> conversion = new F<Method, LifecycleMethod>() {
        public LifecycleMethod f(final Method a) {
            return new LifecycleMethodImpl(a, contextType);
        }
    };
    final Collection<Method> methods = methodLocator.locateAll(contextType, markingScheme);
    return toFjList(methods).map(conversion).toCollection();
}

The good thing about the above is it expresses succinctly what we’re doing; find the methods we’re interested in and map across them to convert them into another type. The unfortunate things are that we have to convert from a Java collection to a FJ collection (toFjList) and we need to use the clunky Java anonymous inner class as a pseudo first class function.

For comparison, here’s a rough stab (i.e. probably won’t compile) at what it’d look like in Scala, notice that implicits will take care of the conversion for us:

def findMethods(markingScheme: MarkingScheme) =
  methodLocator.locateAll(contextType, markingScheme).map(new LifecycleMethodImpl(_, contextType))

Now isn’t that better?

Written by Tom Adams

July 21st, 2008 at 2:11 pm

iPhone sites launch

with 4 comments

Well it’s finally official, and public. After two solid months of development, today marks the launch of the three iPhone sites we’ve been working on:

Screenshots of the sites courtesy of Pete.

These sites may not work on a regular browser, you may be pushed to the regular sites. Tweak your user agent to pretend to be an iPhone for all the goodness.

They’re all developed in Rails, with TrueLocal running under Tomcat using JRuby.

Bring on the 11th!

Written by Tom Adams

July 10th, 2008 at 4:53 pm

Posted in Java,Ruby,Technology

Microsoft release Pex

without comments

My googlebot just informed me that Microsoft has released Pex, which “generates Unit Tests from hand-written Parameterized Unit Tests through Automated Exploratory Testing based on Dynamic Symbolic Execution” (what a mouthful!). At first glance, it looks very similar to JUnit’s Theories and Tony’s Reductio. The difference appears to be that Pex generates the tests statically (don’t know how long they hang around) rather than dynamically like Reductio and JUnit. Ah, QuickCheck for Microsofties…

Written by Tom Adams

June 3rd, 2008 at 2:25 pm