I labeled myself a Software Craftsman many years ago, after reading Pete McBreen's eponymous book. While the book had a couple of weak spots as I recall, I obviously believed then that Craftsmanship is a Good Thing™.
I also consider myself an Agilist. I work with the best XPers, and have learned from true leaders in Agile and Lean.
Then the other day, I saw the abstract of a talk for the Ágiles 2010 conference in Lima. Surprisingly, it seemed to me that Lean processes were being contrasted with Craftsmanship!
So what's going on here? Have I latched on to an antiquated term that has acquired connotations that are antithetical to Agilist identity? By calling myself a Craftsman, am I lumping myself in with waterfall reactionaries?
I prefer to think that Craftsmanship is bigger than the subject of the talk. I prefer to think that my adoption of the Craftsman moniker is a true statement of genuine Agile principles.
I am inclined to thumb my nose at the (real or imagined) detractors and say that a true Craftsman is Agile — indeed, on the forefront of emerging Agile practices. A true Craftsman adheres to a Bushido-like code. A true Craftsman knows that the underside of the tabletop does not need french polishing.
A brief google yielded this Manifesto for Software Craftsmanship. I think I'll sign it. There's nothing there that comes across as anti-Agile.
We finally switched this blog over to Google's new template mechanism which uses xml rather than those funky <$ foo $> tags that caused editors the world over to choke.
We're taking the opportunity to clean up some older posts (re-introduce lost formatting from some port long, long ago) and add the occasionally missing tag/label/keyword.
It's not ready. It doesn't look right. But it's a step in the right direction.
A baby step.
Before I knew what National Institutes of Health was, before I moved to the US, NIH stood for "Not Invented Here."
The reappearance of this phenomenon on my radar is coincident with my deepening involvement in the Agile/XP world. Coincidence? I sure hope so. So step forward all you XPers and agilistas and tell me that you are at least as allergic to NIH as I am.
In the last month I have spent hours (with a pair) figuring out why http session management is broken. I have spent days (with various pairs) trying to outwit a half-baked Hibernate session management infrastructure. Today I spent several hours (with a pair) getting some file manipulation utilities to work.
Even back in the days of Struts 0.5, session management was pretty damned solid. And the Spring-like management of Hibernate sessions by tying them to a request/response cycle (in the simplest case) was hashed out years ago. And the Apache commons I/O library is pretty mature at this point. Hell, it's already on our classpath!
So how did this mess come to be? The answers I got in the past were vague hand-waves along the lines of "well we needed something simple and lightweight so we built our own."
Yeah, yeah, "the simplest thing that could possibly work." But surely the definition of "work" also includes "does not require constant repair and debugging"?
Maybe it's just innocence. Maybe those people really believe that an app framework is "too big" and so building out a Struts-lookalike from scratch is excusable. Maybe they really believe that they just know they will never need to manage Hibernate sessions with a broader scope than a single call. Maybe they really believe the file manipulation is just a quickie, undeserving of a library.
To me the simplest thing that could possibly work is to use code that someone else has spent years refining and is known to work.
Please tell me all this NIH is not hubris.
FileMaker Pro (FMP) has the ability to access a real RDBMS via ODBC dynamically. An "external" table can be accessed in the same way as a native FMP table. FMP calculations, normally associated with a native FMP table, can be added to an external table as "supplementary fields".
So all this sounds terribly glorious. Surely one can migrate one's FMP db to MySQL (or other) so that the data is stored in an industrial-strength RDBMS. That, naturally, allows all kinds of other enterprise apps to hit the same data as the FMP. In fact, one can think of the FMP db as a front end to the enterprise db.
I know folks who have made this work. But migrating is another story. There is no programatic interface to the FMP schema - the definitions of the tables and "table occurrences" (really aliases that allow for the definition of associations that become joins at run time). And FMP's use internally of ids rather than names for fields, scripts, tables, etc – one of its key benefits that allows transparent renaming of objects – means that once a relationship is defined using one table, the only way to switch to another (external) table is to delete and recreate it.
That spells a ton of manual labor. I'll outsource it if I can find someone.
We'll still be left with all the presentation intelligence (FMP layouts) as well as the business logic (scripts, calculated fields, portals) firmly in the FMP world. As we build a parallel implementation that does not rely on FMP, we'll be rebuilding all that stuff. Only this time we'll do it in a transparent, modifiable, refactorable way, using code that can be edited using industry-standard tools like (ooh, gasp) a text editor.
So if you're thinking of building a real enterprise app in FMP, don't. It will bite you.
And to answer my own questions from a few months ago regarding FMP in an XP world: The more I bang my head against the closed, constrained FMP model, the more I realize that its only use is as a spiking tool.
All of my on-stage experience to date has been unplugged. That is, I have never dealt with amplification of any sort. I have sung the lead in operettas and musicals of all sorts as well as numerous musical roles, of course. All of it unamplified. But I have never – until tonight – had to sing onstage with a full band, had to deal with mic technique, had to try to follow exactly where the hell we are in the song with a monitor blaring the bass line alone...
The "Music Masti" at Agile2008 was my wired debut. Damn, it's hard to figure out where you are, to keep track, to be able to hear un-monitored instruments when you are singing in a band.
Now I have designed sound for stage productions, even mic'ing actors where needed. But now I know what it's like to be on the performing end. Who'd a' thunk it was so hard? This is what happens when you let popular culture pass you by.
Sigh. There's a lesson in this. There is no substitute for doing, even when it seems that "it" is a natural extension of what one already knows. Maybe it's the geek mystique: "How hard could it be?" But when you pick up that mic and you realize that it's actually not that self-evident,.. it's no longer an extrapolation from the known the assumed, but genuinely into the unknown, it's a bit of shock.
Regardless, it was unbelievable fun.
Rock on!
I caught a bit of this morning's Forum on KQED (Download (MP3)) which was mostly about some civil servant in SF who sabotaged the City's computer systems, and what that means for employers and "health and security of the workplace."
The interesting bit for me was a Towers Perrin survey of 90,000 employees across 18 countries indicating "widespread disengagement".
- 21% "engaged with their work"
- 38% "partly or fully disengaged"
Also cited was a Gallup survey saying that
- 19% of employees are "actively disengaged" i.e. trying to sabotage the workforce
I bring this up because as XP evangelists we run into people who we want to see succeed and excel, but who might be dealing with the own demons. And a much-cited antidote mentioned on the show was "job rotation" and "employees training each other". So pair programming is not just good for the code, good for the dev, good for the team, good for the enterprise, but it might just stop you from going postal. ;-)
Those numbers also tell us that there is a vast amount of waste. Large numbers of people punch clocks and little else. Again, as XPers, we talk about productivity, but what does that really mean for someone who is comfortably numb?
When you have a FileMaker "solution" in place, you have a large investment in UI design and usability. You might, depending on the quality of the original design, also have an investment in the database schema. Porting to another environment - say Java + Spring + Hibernate - is daunting, mostly because re-implementing the UI is such a vast undertaking.
As an XPer, I need to be able to test-drive development. The only reasonable way to do that is by using external AppleScript (or Python + appscript and maybe PyUnit too) tests. I don't know about you, but that sounds like far more trouble than it's worth. Just navigating the maze of what works as expected and what requires arcane tricks in the AppleScript world gives me a headache just thinking about it.
XPers also refactor. Not surprisingly, there are no tools at all for refactoring. Worse, the structure and syntax of FileMaker's scripting language actively discourages refactoring. For example, while it's possible to pass multiple arguments into a sub-script, they have to be packed on the caller side and unpacked by the callee - multiple lines of script. And having to drag commands into place rather than just type like a normal human...
Then there is basic source control. A FileMaker file consists of several elements, the most important being
- Tables and relationships
- Scripts
- Layouts
I need to be able to see what's changed from one version of my FMP file to another. Since all the above elements are stored in a single binary file with a proprietary format, it's impossible to do a diff. Sure, one could export a DDR XML file and diff that, but diffing XML?!
And there are indeed tools that purport to do this. Unfortunately, they are universally hard to use and slow, if you cam get them to work at all. (Of the 3 main contenders, I have only managed to get the demo version of one of them to work consistently. The others are so obsessed with license keys that they fail to work at all.)
My "Mastering Selenium" tutorial has been approved at Agile2008. I am very excited about that.
Meanwhile, in preparation, I have been putting Selenium-Grid through its paces at Industrial Logic. We are formulating a high level lingua franca to use in describing executable Storytests. This is a big deal. It was the promise of FIT, but for one reason or another (and theories abound!) Customers stayed away from FIT in droves.
Our old Selenium tests had been constructed using a wiki language and template processor. This provided a powerful toolset with which to construct Stroytests, but we ran into 3 major issues:
- Selenium Core is slow
- With the HTML Selenium tests, there is no guaranteed tear-down as there is in xUnit
- Our toolset, while providing a fair bit of power and reasonable readability, did not come close to the expressiveness of a language like Python, Java, or Ruby in Selenium-RC.
So we went full out for Selenium-RC, using JUnit to drive the tests against a local Selenium-server.
To get the ball rolling, we grabbed the HTML from an old Selenium test by simply firing it up in Selenium Core and grabbing the code from the browser. It's a trivial matter to translate it into one of the Selenium-RC languages - we're using Java - with the Selenium IDE. Some deft text editing massages the Java file into shape so that it fits into it's place in the class hierarchy. More about the hierarchy later. We found reworking these legacy Stroytests to be an illuminating process. It meant bottom-up - starting with a very long list of Selenese commands, and doing extract method on clumps of code. The original source file with its template language was found to be lacking in many cases. The tests had grown over time and had not been refactored. We found some very interesting issues:
- Code that verified conditions that had absolutely nothing to do with the story at hand. These were presumably checks that had been copied/pasted from some other test where those checks were relevant.
- Some tests attempted to do too much and veered off on a tangent.
- Once we put in place some decent setup and (dependable!) tear-down, the test body was short and to the point.
So what does the class hierarchy look like?
The immediate superclass of each Selenium test class contains domain-specific helper methods. Its superclass is essentially a proxy for the Selenium server (or a hub, which is in turn a proxy for the runners.) For that class, we started with the SeleneseTestCase that is provided in the download, but decided that we didn't like it for a variety of reasons. In particular, we needed to do some magic so that a new Selenium session (and hence a new browser) is only launched if none already exists for the current thread. That's because we exploit the good ol' JUnit 3.x suite and test decorators to run allow for parallelization of tests. Sure, we could have used testng but its seemed a little haphazard to us and that business of generating an intermediate xml file just doesn't seem right.
The next chunk of learning relates to just what is needed to get tests to run in parallel so that they don't stomp all over each other.
[From Macworld | Mac OS X Hints | Add more power to 10.5's screen sharing]
I live by VNC, since we are a distributed XP team. So I was thrilled when I found this MacWorld article on tweaking the build-in VNC client on Leopard.
Now if only it would get over some of its temperamentality so I can use it reliably over an SSH tunnel or to a non-Mac machine.
So I was geeking out last night, as Mike would say. I was writing some code to fix some corruption in my calendar. It's in iCalendar format, so I found an ical4j framework on SourceForge and fired up Eclipse.
The first thing I did was to copy some snippets of sample code from the ical4j docs into a JUnit test and see what happens. I did this for 2 reasons: (a) the docs for FOSS are often wrong, and (ii) I needed to get all the config set up (e.g. to make a test calendar with events in it containing the properties I was interested in.)
As I worked, my "production code" kept evolving inside my test class. When I had successfully TDD'ed a step, I extracted the code into a real production class. I could move code around with Eclipse's refactoring tools and I had solid tests to save my butt.
Nothing startling in any of this. The interesting thing for me was that I was exploring ical4j. It was something of a spike - I didn't know ahead of time if the framework would be up to the job - yet each success immediately resulted in moving the code to the production classes.
So what do you think of this approach of evolving the code in the test and moving it as soon as it's green?
Redux: Keith Ray says: It's good. I think Kent Beck or someone called these "Learning Tests"when learning an API via TDD.
 One normally thinks of a "launch" as a discrete event, an atomic occurrence, something which starts and ends almost instantaneously. A product launch takes days to get out the door. It's more like childbirth. Today, Josh sent out the press releases to Dr Dobb's and other publications, email to numerous mailing lists, and the mass mailings. The launch continues.
I have mentioned a few times that I have been using Python to talk in an AppleScript-like manner to Mac apps, via the appscript module. It is really exceedingly cool. It provides very tight control over when one does the “get” (i.e. the AppleEvent that actually hits up the app for some data) allowing one to manipulate references instead. This has the great benefit of a dramatic performance improvement over the brute-force approach one might use with AppleScript.
Correspondingly, of course, there is the fact that I have learned a great deal about how AppleScript works, and, should it be necessary, I'll be able to write much cleaner and more efficient AppleScript.
“More efficient?”
Well, yes.
And no, I am not violating the “don't optimize prematurely” maxim of XP, because AppleScript is universally, painfully slow. I pretty much know that anything I do in AppleScript is going to take forever at run time, so any optimizing rubrics I come across of are worth remembering.
And while we're talking about OSA... I have not had much occasion to use JavaScript OSA. I bet it's as nice to deal with as Python from the perspective of scripting scriptable apps. Of course, JavaScript is implemented as a “component,” meaning that it must be executed in an OSA-aware script editor (or via the osascript command) but there is no interactive interpreter available as there is for Python. Not exactly convenient.
The main reason I wanted to use Python (other than the fact that it's a nifty language and I know it quite well) was that I wanted to interact with a MySQL db. Easy in Python. From a purely OSA language like AppleScript or the JavaScript component, I'd have to have a scriptable app act as go-between. How would I do that? And can you imaging how slow that'd be?!
One more thing to try, however, might be to use the old MRJ technology to make a scriptable Java app. Then the Java app can use Hibernate to get at the db. Since the goal of this whole exercise is to integrate with a Java web app with Hibernate backend, the code reuse is obvious, and having to maintain two persistent layers is no longer an issue.
There are standards. They might not have existed when you wrote your code, but they exist now. So, is it worth migrating from your proprietary tool or framework to a standard?
Well, it depends. I like to apply refactoring to all things: tools, subsystems and processes and not just code. So that means that if you find yourself having to tweak or maintain something that smells - the build system in my prior example, or a ponderous process, or the use of standards in this example - then it's always worth investing time to make it better. Do it scientifically: Weigh the pros and cons, just as you would with any refactoring. Is it worth abandoning my proprietary, homemade, hand-built app server to embrace a J2EE-based server? Probably. Is it worth throwing out an arcase and convoluted ORM tool that relies on an unsupported tool? Almost certainly.
The definition of "broke" (as in "if it ain't broke") must always include pain and suffering. Martin Fowler's term is "smell" and it has a less tangible connotation than merely "broke." Choose wisely. Be brave.
Not everyone is going to like what I say about process: it's time to leave the waterfall model of development behind and "embrace change."
Sure, there are times when committments to outside customers require than you define to some level of detail what it is you're going to build. Sure, internal customers, like QA and Tech. Pubs., might have a genuine and legitimate need for a firmer spec. than an Extreme process can supply. And yes, sometimes a project is so large that an agile process really won't cut it. But face it, folks, building an enterprise web app (what we do for a living) is a perfect candidate for XP. We can handle it. Why big the whole thing down just because of some small challenges?
So what about documentation? People tell me that arcane, proprietary architectures and frameworks need to be carefully documented. I disagree. By pairing neophytes with veterans, and by appropriate use of unit tests (as "living" documentation) written documentation is unnecessary. Indeed, documentation is seldom adequately maintained, and if we're talking about a shop where "no one has time to document" then they surely are not going to have time to maintain documentation, even if it was handed to them on a platter.
An argument I keep hearing is that we "don't have the luxury" of pairing a veteran with a neophyte. Hmph. So how do you justify having talented Engineers reverse engineer gnarley corners of system lore, when a few minutes with an expert will save everyone time in the long run? Maybe my counter-argument should be that we need physical proximity in an open work area, and let the hard-core pairing wait for another day. Baby steps.
Everyone loves CVS. But let's face it: It sucks. It lacks "atomic commits," whereby you can check in a whole bunch of files as a unit. You can't check in directories. Branching is a pain and not very good. Tools for merging at checkin time are terrible. Of course, everyone knows CVS; it's well documented, well debugged and so on. It takes seconds to set up and is easy to back up. Best of all, it's free.
Commercial alternatives cost a lot but address some of CVS's shortcomings in a serious way. In particular, Perforce does a fine job of atomic commits and handles branches well. Sometimes that could just be worth the cost. (Support, mentioned repeatedly on their web site, is really a non-issue in real life.) Perforce, however, enforces exclusive checkout, meaning that no one else can edit a file if you have it checked out. While this eliminates the need for merging, no one works that way in real life.
Although I have not used Subversion , it sounds fabulous. It's a "better CVS" that addresses some of the worst shortcomings. It still retains the CVS model of non-exclusive checkouts but allows atomic checkouts and so on.
Incremental builds are not a luxury, they're a necessity. If your build system does not allow you to recompile just the files you've changed, then you need to throw it away and use Ant. Your IDE should allow this too.
Once files are compiled and your unit tests all pass, you need to be able to fire up your app and see the code in action. A recent client has their app as the root context in WebLogic, requiring that the app server be bounced for each code change! They had WebLogic configured so that startup time was in excess of two and a half minutes. That cannot be tolerated - the round trip time to see a code change in action should be minimal. If you're using Ant and Tomcat, there are Ant tasks to manage deployment and reloading of specific contexts.
A warning is supposed to quicken your pulse, cause sweaty palms, perhaps even a mild headache, shortness of breath or dizziness. Warnings are not benign. Be afraid of warnings. If your build (or other lengthy) process routinely emits warnings that can "safely" be ignored, you're looking for trouble. How long before the words "Wolf! Wolf!" no longer mean anything to you? Besides, those so-called "benign" warnings are spew, and you do have a rule against spew, right?
If your compiler is being hysterical about something that cannot be changed, find a way to encourage the compiler not to emit the warning - just that one warning. You don't want to turn off warnings altogether, since they warn you of important stuff you need to know. If your compiler won't let you suppress a specific warning on a specific line of code (the Java compiler doesn't) then document this particular warning in the console output. This is a last resort.
The correct way to deal with warnings is to fix the cause.
Those doing the work must own the schedule. It's that simple.
Estimating is hard, but we learn - we're brave. We calculate our initial velocity based on the phase of the moon and then correct it. Within a few interations, we're spot on. But where is the incentive when others own the schedule?
Don't spew. A tool that does a lot of stuff, like a build tool, must tell you succinctly and precisely if it succeeded, and if not, exactly what failed. Spewing lots of information to the console makes troubleshooting extremely difficult. Sure, if you feel compelled, spew to a log file, but the console is sacrosanct and must be used sparingly. The above hall-of-shamer spews 30,471 lines to the console when it succeeds! Try finding an iffy warning or important message in that. Worse, if the build process believes that it has succeeded, but some important message has been emitted that would help with troubleshooting, having to wade through thousands of lines of output makes it much, much harder.
Use Ant. Or Maven. Or even Make if your heart yearns for the Old Days When It Was Dangerous. But for goodness sake, don't use a mish-mash of homegrown tools consisting of some DOS .bat files, some korne shell scripts, some bash scripts, some perl scripts, a handful of make files, and ant script or two, ... (Yes, it's true, an organization I know uses just such a setup to set off a 30 minute build!)
Why?...
When your build involves arcane but clearly necessary steps, Ant can still handle it, even if you have to write an ant tool or (less optimally) a shell script invoked from ant to do it. Ant is well-understood in the industry - everyone knows it. It's well supported. It's documented. It's reliable and well debugged. It's a little weird in spots, but with a short trip to any number of books or Google, you can get the answer to almost any question. It's integrated into modern IDEs so those arcane steps I mentioned which cannot be executed by the IDE's incremental compilation can still be run with ease.
|
Facebook
|
