Technology roundup

I'm a Java programmer, so I use Java all day, right?

Not really. I got to thinking about all the moving parts in what I do, and I decided to write it down to see how long the list is.

• Java of course
• JUnit for test-driving Java
• Eclipse for editing and building
• Eclipse plugin frameworks for the Eclipse plugins
• Ant for even more building
• Bash scripts for deploying and our famous Continuous Delivery and sundry tasks
• StringTemplate for making web pages
• HTML for making web pages with content
• CSS for making them pretty
• JavaScript for making web pages come alive
• YUI for even more dynamic web pages
• Prototype for enabling simpler JavaScript
• Scriptaculous for nifty screen effects and client-side drag-and-drop
• SQL for storing stuff about things and people
• Hibernate for ORM'ing said stuff
• C/C++ for client-side exercise critique stuff. And eLearning content
• Visual Studio for editing and building C# code and the occasional foray into JavaScript debugging for IE
• ReSharper for making Visual Studio useful
• C# for our ReSharper plugin as well as eLearning content, of course
• AdvancedInstaller for making the installer for the ReSharper plugin
• NUnit for test-driving C#
• Selenium Grid and RC for storytests (acceptance tests)
• Hudson for Continuous Integration build and test, and for Continuous Release
• Tomcat in a cluster for seamless deploys and other things
• RHEL and all its intricacies
• Yum for keeping RHEL happy

And this doesn't even include sysadmin and administrative stuff.

No wonder I'm tired!

Class: Robot Programming with leJOS

I am teaching a class on robot programming with leJOS as a follow-up to my "Programming with Scratch" class. LeJOS is an embedded JVM for LEGO Mindstorms. I started with a 30 minute tutorial on Object Oriented programming (what's a class? what are instances, methods, inheritance and polymorphism?) Yes, 30 minutes.

Then we jumped straight into test-driven development with JUnit and my own library that sits on top of the leJOS API. (I had to make this layer because leJOS objects are untestable.) Predictably, this was challenging, but not impossible! However, since we're dealing with fake (testable) motors and light sensors and so on, the production code we write doesn't cause physical wheels to turn and so on. That makes this all very abstract. It's a healthy stretch for a 12 year old, but they're doing it.

We took 15 minutes out to begin assembling the robot, and wrapped up the session with a brief retrospective.

Next week we'll download some of our code and watch the robot move (assuming we can build the damned thing!) Then we'll continue test-driving more behavior.

This is cool!

Test-driven development with leJOS in Eclipse

I have a working setup and have been happily test-driving development of an embedded leJOS app all day!

I created one project for the app itself (CUT - code under test). It has only the leJOS classes.jar (and no JDK/JRE classes) and uses the Eclipse Java Builder as would a standard Java app.

I created a second project for the tests. It's a plain Java project. It has the CUT as a project dependency.

Now for the code that made this possible: For several of the classes (Motor, LightSensor, etc) I had to make special decorators so that I could mock them.

For example, I have an Eye class that has-a LightSensor and which implements setLightValue() in addition to the pass-through getLightValue(). In my tests, I have a TestableEye subclass of Eye that simply remembers the value passed in to setLightValue() but does not call the underlying LightSensor at all.

I did the same thing with Motor. There's a decorator with a "Testable" subclass.

Now my unit tests (when we teach this stuff, we use the term "microtest" because it's less confusing to the QA folks) I simply instantiate a robot with a bunch of TestableMotors and TestableEyes. The microtest then exercises all the hairy logic required to get data from the eyes and drive the motors.

This is a pretty strong proof of concept. I expect to build up a few of these decorators to work around the issues with the leJOS design philosophy, specifically that work is often done in the no-args contructor. That makes them extremely difficult to subclass for testing purposes (that is, to mock.)

Epilogue
I was told this in the leJOS forums:

Note, that on the NXT, only the classes in the file classes.jar are available, and that this contains only a limited subset of the JDK APIs. For example, reflection is not available. Ergo, JUnit will not work.

There's something about proving people wrong when they say "it cannot be done."

TDD with LeJOS

My "Programming with Scratch" class has come to an end. The 5 girls had fun with it, and voted to move on to Java programming rather than continue in the world of Scratch. Specifically, I promised them LEGO Mindstorms with Java, or to be precise, LeJOS.

I am a diehard TDDer, so of course I shall be teaching Java programming test-driven. But test-driving with LeJOS is challenging. Because LeJOS is an embedded system, it uses a pared-down version of the Java class libraries and a custom JVM. The LeJOS system does not support reflection, which is required by straightforward JUnit. So how will we approach TDD with LeJOS?

One option is to build our LeJOS apps using the normal JDK – after all it's all syntactically correct Java. We can then test-drive all aspects of our logic while mocking things like motors and sensors. Of course, this only gets us so far. The real snag is that we will be test-driving in one environment and running in another. Any time you have a mismatch between dev and production environments, you end up with a problem, and in this case, the difference is quite significant: Not only are the JVMs quite different, but the languages themselves (in the broader definition) are dissimilar.

I'll be looking into this issue and posting more as I learn more.

Notes on making a simple auth app using Wicket

Follow the instructions here and add the junk to your pom. You'll need to add a repositories element in which to put you repository. I put mine near the end, after the build

Run mvn install

I prefer JUnit 4 to 3.8, so I removed the generated "variable" ref from my class path and added Eclipse's built-in library for JUnit 4.

To keep things consistent, I modified the pom to refer to JUnit 4.. Maybe I should have let the Eclipse build reference the Maven repo rather than Eclipse for its JUnit. Hmm.

Added "variable" ref for both swarm and wasp (sheesh - why two? This is so confusing) and changed my application to extend SwarmWebApplication as described in the swarm getting started doc.

Of course, Eclipse helped me out with the stubs of the overrides of the abstract methods, so on to filling them in.

So, starting with a bare-bones app, start adding security. Hmm. Not working.

OK, so start with the kitchen-sink security demo app and begin deleting unneeded code. Hard, because I don't have a project and have to rely on reconstructing one from the war.

Aaargh! Give up on swarm. The guy's dead anyway, so who's maintaining it?

So auth-roles is the next.

Generate a project using mvn. Modify the pom to include auth-roles:

<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-auth-roles</artifactId>
<version>1.4-rc4</version>
</dependency>

Then use mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true to make the eclipse project. Import into eclipse.

Copy gibblies from the "wicket-auth-roles-example" available on Sourceforge. (It's in maven too if you know how to get it. I don't.) It has a weird structure, not like the autogenerated one with the embedded Jelly, so that's why we have to copy/paste.

A couple little syntax errors: Stuff that's presumably changed in more recent releases. An overridden constructor needs to be deleted, since the thing it overrides no longer exists. Stuff like that.

And the source needs to be reformatted. What's with all this C#-like source formatting anyway? Sheesh, this is Java, guys.

Bootstrapping Java on Fedora

Every time I have to bootstrap a new box with Java, it's a learning experience. I sure wish it was easy. The hardest part this time 'round was finding Java. Sun sure goes to a lot of trouble to hide it.
Found this:

[How To Install JDK 6 / Java SE 6 (+ Tomcat) in Fedora Core 6 / Fedora 7 in 5 Minutes]

There is a secret directory where you can download JDKs and just install. The end.

Wicket

So I started by using the QuickStart tool to generate the correct Maven incantation. (I'm already hating Maven) and following the instructions there in my Eclipse workspace. This generated a whole tree of stuff: my new project. The artifactId I specified ended up naming the root of the tree.

Next – still following the instructions – I went into the project and did this: mvn eclipse:eclipse -DdownloadSources=true

After that, into Eclipse and tell it to create a new Java project. I gave it the name I'd already chosen (the ArtifactId) and Eclipse noticed that there was a project there to import. Cool.

Unfortunately, it uses an eclipse "variable" called M2_REPO in the classpath, so this need to be resolved. I opened my Eclipse preferences and navigated to Java > Build Path > Classpath variables and set M2_REPO to point to $HOME/.m2/repository

But Eclipse still wasn't aware that it needs to copy the .html files to the output ("target") directory. That's because the generated .project explicitly includes only .java files in the source path. To remedy, open the project properties and go to the Source tab in Java Build Path. Add an inclusion pattern for "**/*.html" to the one already there for the .java files.

Great! A working wiket app. That wasn't too bad.

So what about Wicket Bench, the Eclipse plugin? Out of date. It points to its own internal Wicket distro, which is old. Nice idea otherwise, and TDD-friendly.

Next, I tried one of the apps included in the Wicket download. There's a pom.xml there, so maybe I can get to run as a standalone app? Too many missing pieces.

At this point I am miffed. What a mess! Every other thing I touch is out of date, but as a n00b, I have no way of knowing which is the new and which is the old.

So, my latest strategy is this: Use the archetype stuff from the QuickStart tool, and then copy needed pieces in from other parts of the distro – in my case, the auth & auth sample code. We'll see how that goes.

Selenium-RC

My "Mastering Selenium" tutorial has been approved at Agile2008. I am very excited about that.

Meanwhile, in preparation, I have been putting Selenium-Grid through its paces at Industrial Logic. We are formulating a high level lingua franca to use in describing executable Storytests. This is a big deal. It was the promise of FIT, but for one reason or another (and theories abound!) Customers stayed away from FIT in droves.

Our old Selenium tests had been constructed using a wiki language and template processor. This provided a powerful toolset with which to construct Stroytests, but we ran into 3 major issues:

1. Selenium Core is slow
2. With the HTML Selenium tests, there is no guaranteed tear-down as there is in xUnit
3. Our toolset, while providing a fair bit of power and reasonable readability, did not come close to the expressiveness of a language like Python, Java, or Ruby in Selenium-RC.

So we went full out for Selenium-RC, using JUnit to drive the tests against a local Selenium-server.

To get the ball rolling, we grabbed the HTML from an old Selenium test by simply firing it up in Selenium Core and grabbing the code from the browser. It's a trivial matter to translate it into one of the Selenium-RC languages - we're using Java - with the Selenium IDE. Some deft text editing massages the Java file into shape so that it fits into it's place in the class hierarchy. More about the hierarchy later. We found reworking these legacy Stroytests to be an illuminating process. It meant bottom-up - starting with a very long list of Selenese commands, and doing extract method on clumps of code. The original source file with its template language was found to be lacking in many cases. The tests had grown over time and had not been refactored. We found some very interesting issues:

• Code that verified conditions that had absolutely nothing to do with the story at hand. These were presumably checks that had been copied/pasted from some other test where those checks were relevant.
• Some tests attempted to do too much and veered off on a tangent.
• Once we put in place some decent setup and (dependable!) tear-down, the test body was short and to the point.

So what does the class hierarchy look like?

The immediate superclass of each Selenium test class contains domain-specific helper methods. Its superclass is essentially a proxy for the Selenium server (or a hub, which is in turn a proxy for the runners.) For that class, we started with the SeleneseTestCase that is provided in the download, but decided that we didn't like it for a variety of reasons. In particular, we needed to do some magic so that a new Selenium session (and hence a new browser) is only launched if none already exists for the current thread. That's because we exploit the good ol' JUnit 3.x suite and test decorators to run allow for parallelization of tests. Sure, we could have used testng but its seemed a little haphazard to us and that business of generating an intermediate xml file just doesn't seem right.

The next chunk of learning relates to just what is needed to get tests to run in parallel so that they don't stomp all over each other.

TDE - test-driven exploration

So I was geeking out last night, as Mike would say. I was writing some code to fix some corruption in my calendar. It's in iCalendar format, so I found an ical4j framework on SourceForge and fired up Eclipse.

The first thing I did was to copy some snippets of sample code from the ical4j docs into a JUnit test and see what happens. I did this for 2 reasons: (a) the docs for FOSS are often wrong, and (ii) I needed to get all the config set up (e.g. to make a test calendar with events in it containing the properties I was interested in.)

As I worked, my "production code" kept evolving inside my test class. When I had successfully TDD'ed a step, I extracted the code into a real production class. I could move code around with Eclipse's refactoring tools and I had solid tests to save my butt.

Nothing startling in any of this. The interesting thing for me was that I was exploring ical4j. It was something of a spike - I didn't know ahead of time if the framework would be up to the job - yet each success immediately resulted in moving the code to the production classes.

So what do you think of this approach of evolving the code in the test and moving it as soon as it's green?

Redux: Keith Ray says: It's good. I think Kent Beck or someone called these "Learning Tests"when learning an API via TDD.

Very disappointing: Java + AS hosed

Well, I might have known.

The framework to invoke AppleScript from within a Java app is hosed. There's a thread on the O'Reilly network about the problem, but no word of the solution.

For some obscure reason, executing a compiled script takes forever - minutes for a trivial tell application "Finder" to ...

That means that my grand plan is completely foiled. Very disappointing.

IDE

Your choice of free IDE realistically comes down to two players: Eclipse and NetBeans. Which you choose depends on your process mainly.

Eclipse is a little slower than NetBeans overall, but it shines in a number of crucial areas:

1. It manages incremental compilation completely seamlessly. Simply saving a Java file compiles it. References to other classes are immediately available, complete with JavaDoc - no messy regeneration of metadata is needed since it's all done incrementally.
2. Eclipse has refactoring tools built in - and good ones at that. Since class metadata is constantly available, the refactoring tools can find callers or a method without searching. NetBeans has the RefactorIT third-party plug-in which mostly sucks and cannot possibly offer the power of Eclipse's built-in tools.
3. JUnit is integrated so that there is no need to create Suites. (NetBeans now has support for unit testing, though I have not tried it.)

But NetBeans stands head-and-shoulders above Eclipse in one significant area: JSP editing and debugging. With tag completion in JSP's, editing is a breeze. With integrated Tomcat, running and debugging JSPs inside the IDEs is practical.

Since both tools are free, it makes sense to use Eclipse for the majority of development, and then fire up NetBeans for editng and debugging JSPs.

Compiling JLint

I periodically get all enthusiastic about static analysis tools. I hadn't tried JLint for a while, because of compilation issues. Then I remembered that gcc can be "reverted" on Mac OS X. Use the gcc_select command (type gcc_select -h for help) to switch between gcc versions.

By editing only one file (field_desc.hh - make sure the "using namespace std;" at the top is not conditional) and reverting to gcc 3.1 all built without incident.

I think I'll recommend JLint as a candidate for fink.

Findbugs

I recently discovered FindBugs, and was pleased to note that there's an Eclipse plugin. Small problem with installation instructions: Any FindBugs problems encountered are not shown until you explicitly allow Eclipse 3.0's filter in the Problems view to pass them.

• On the "Problems" view, click on the "Filters ..." button (top right corner)
• Check "FindBugs Problem" on the "Show items of type" list

I also noticed that the FindBugs site links to JLint. I haven't had much useful info from JLint in the past, but it's worth another look. And JLint also has an Eclipse plugin.

Enum Values

When a variable can contain one of several constants (really an enum - O hurry, Java 1.5!) then the values should be chosen carefully. In particular, never chose zero as one of your enum values. Java initializes int variables to zero, and if you choose one of your values as zero, there is no way to distinguish an uninitialized variable.

This is why in real life, we use String constants, rather than int. Since the compiler does constant pooling, and if the simulated enum constants are declared final you can use the == operator to compare them. That addresses that performance objection you were about to raise. I haven't done benchmarks, but I am willing to bet that comparing object references performs as well as comparing integers - or at least within a tolerable delta.

"Command Line" args to Eclipse on Mac OS X

Actually, this applies to pretty much any Java app on mac OS X - and really any app at all.

To configure the "command line" args (the stuff that gets passed in to main()) for Eclipse (like the location of your workspace directory), edit the Info.plist file. You'll see an array of things that you'd normally put on the command line. The one we're interested here is "-data /My/special/workspace/dir".

Since the properties list is just an XML file, you can edit it in BBEdit (or even Eclipse if you like) but it's handy to navigate to it in a Terminal window and then type "open Info.plist" to have it open in Apple's Property List Editor.

Formatting of Java source

There are several things I have observed that are contributing to a slower process, higher defect rate, and increased maintenance costs. Less tangibly, there are factors that raise the frustration level among developers.

I am going to be posting a series of rants about some especially bad practices I've observed recently. These are obvious to most people who have been programming seriously for a while, and the excuses for the bad practices are legion. But I am going to list the sins here so we can all be reminded.

The first is Formatting of Java source.

Many Holy Wars have been fought among developers regarding "coding standards." The best way out of this trap is to adopt an external standard. The canonical standard is that used by Sun. It's how most Java programmers do things. (I personally don't like C-style formatting in Java. It smacks of reactionary intransigence, and it consumes vertical space unnecessarily.)

Provide tools like Jalopy so that mavericks can see the source the way they want it, but enforce an automated reformatting to the standard upon checkin. Of course, none of that is necessary if everyone just adopt the standard and adhere to it.

Exporting a Schema from Ant with Hibernate

After trying to run the SchemaExport application from within ant (silly me - I believed the dox) I thought there just has to be a better way. I was about to start writing an ant task to do the trick when I thought I'd look and see if one was perhaps squirreled away in the jar somewhere. Sure enough, there it was.

Lesson One: With open source code, always read the source because the docs are occasionally out of date.

Getting it to work was a feat. I was getting weird error messages, so I thought I'd add some debugging code to the Ant task and regenerate hibernate2.jar. I'm running ant inside eclipse, and I thought just using the provided build.xml would be easier than trying to set up a whole new Eclipse project for Hibernate. So that's what I did. Without quitting eclipse, I simple ran ant against the build.xml in the hibernate directory. Back in Eclipse, the weird behavior would not go away and my debugging code was not being executed.

It seems that all of this is using one giant ClassLoader so I needed to quit and relaunch Eclipse in order for my new hibernate2.jar classes to be loaded. Ouch. I should have known that.

Lesson Two: Never forget that Java will not dynamically reload a class unless you take pains to do so programatically. You must quit the VM to force the class to be reloaded from a rebuilt jar.

The bottom line is now I have a working ant task for publishing my schema automagically as part of my build.

Uninitialized variable in Java

public class Foo {
    private int foo;
    public Foo(int bar) {
        this.foo = foo; // supposed to say bar
    }
}

This code runs, but foo is never initialized.

It's become common to use names that match ivars as constructor arguments. This illustrates one of the dangers.