Wabi Sabi Software

Scala version of Log4JFugue, plus revamped website

2012-01-27T08:14:00.000-08:00

Over the recent holidays I decided to create a Scala version of my Log4JFugue open source music project. I was able to reduce the 2500 lines of Java to just over 250 lines of Scala!

Checkout the new www.log4jfugue.org as well as:
Scala sources

I'll be giving a talk on the experience of converting a small Java project to Scala at the March meeting of New England Java User's Group.

new article in December PragProg magazine

2011-12-07T10:07:00.000-08:00

I just had an article published in the December Pragmatic Programmer magazine on
a new approach to logging

Writing on a Tablet

2011-11-18T08:47:00.000-08:00

I do a lot of writing these days, some on my high end Windows laptop with dual 20 inch monitors and some on my 10 inch tablet with Bluetooth keyboard.

I have surprisingly discovered that I am far more efficient on the tablet. This is surprising because on the tablet I have much smaller screen (less than 100 square inches versus more than 700 square inches), I don’t have a mouse, and switching from my writing app to email or the web is more involved.

That last item proves to be the key item. Because the tablet is primarily not a multitasking device I tend to focus on writing when I’m writing. The lure of the emails lurking in my system tray is gone. The other browser windows in my giant multimonitor display do not call to me. The fact that my tablet can really only do one thing at time is not a limitation it is a benefit.

While at a certain level this is obvious it’s also interesting given the current debates about multitasking operating systems. At the same time that some OSs are claiming to be better because of multitasking some of the mainstream OSs are adding features to discourage multitasking! For example various flavors of Linux have the option to automatically dim all windows other than the active one. The idea is to reduce the lure of context switching.

All kinds of current research is showing that humans cannot really multitask, yet we want to pretend that we can. At the same time, we know that the key to writing is to…well…write. And as much as we love to write we also love anything that can distract us from writing. So using a “limited” system to do our writing on is really to our advantage.

A few minutes with the Kindle Fire

2011-11-16T09:15:00.000-08:00

I got to spend a few minutes with the Kindle Fire yesterday. Full disclosure: we have two kindles, an iPad, a smartphone and assorted laptops at home.

The iPad has almost completely replaced the laptop at home, and with a bluetooth keyboard I do most of my writing on it. It comes with me to the numerous evening Meetups I attend.

My kindle comes with me everywhere during the day, because it fits in my pocket which lets me squeeze in lots of spare minutes reading.

The smartphone has been relegated to making phone calls and checking gmail when out of wifi.

The KF is agreeably small and light and I could imagine it being with me everywhere. The screen is clean and crisp. Some reviewers have complained about slow screen response but I did not encounter that. Page turns while reading were very responsive and selecting items from the various menus was easy.

The touch screen made selecting and highlighting text much simpler than on the legacy kindle, so I might pick the KF for technical reading where I do lots of note taking.

One problem I did encounter was in getting to the menu screen. Most apps are full screen so to get to the menu and the Home button you have to touch the bottom of the screen. The location to touch was hard to find. It often took me lots of touches to bring up the menu resulting in lots of unwanted page turns. Presumably this would be something one would figure out over time.

Movies look nice on the KF, though obviously a smaller image than on the iPad. My kindle reading is largely opportunistic, a few minutes here and there. While I'm willing to read books a page at time I don't know that I'd want to watch a movie that way. When I know I might have blocks of time I bring my iPad. So movies wouldn't be a KF draw for me though they might be for my iPad-less wife.

Magazines look much better on the KF than on the legacy kindle.

In the end buying a KF or not depends on what devices you already have and what you want to do.
If you just have a legacy kindle or a smartphone and don't want to do content creation the KF makes sense. If you just have a iPad but no book reader you might get the KF for its extra portability.

For my wife who has a kindle plus dumb phone and wants some internet access we have a couple of choices. We could get a $300 smart phone plus two years of a $30/month add-on data plan; two year cost $1020. Or a $200 KF, possibly adding on the $79/year Amazon Prime membership. That seems like an easy choice for us. Your mileage may vary.

New article published at Pragmatic Programmers

2011-11-03T06:29:00.000-07:00

I just had an article published in the Pragmatic Programmer magazine on
the long history of root cause analysis

Kindle Fire replaces SmartPhone not tablet

2011-09-30T08:08:00.000-07:00

Just had a conversation with my wife about the Kindle-Fire (KF) and had an epiphany.

KF isn't a replacement for the full tablet, but it is a replacement for the smart phone!

KF costs $200 plus zero for service contract
smart phone costs $50-$200 plus at least $30 a month for data contract.

If you mostly use the 'smart' features of the phone for things like email/calendar/games, then it may be very compelling to ditch the cellular data plan, ditch the smart phone, get a dumb phone for making dare-I-say-it phone calls and the KF for email/calendar/etc.

I think we're going to do that.

OTOH, if you want a tablet for creating rather than consuming content, you still need a full size tablet, preferably with a keyboard.

Publication Summary

2011-09-14T06:21:00.000-07:00

I've been asked a few times for a summary of my publications over the last few years so here it is.

Sept 2011 Scala Traits

August 2011 Code As Dialog

May 2011 Catch the Pig!

November 2010 The value of a personal bug log"

September 2010 The Key to Better Software

June 2010 Better Tools and Training

June 2010 Debugging MySQL Stored Procedures

May 2010 The Floating Finish Line

April 2010 Medicine Making Music

March 2010 Testing As Contract

November 2009 Interrupt Mood

June 2009 ...And Your Bugs Can Sing

Review of Venkat Subramaniam's new book on Concurrency

2011-08-25T06:45:00.000-07:00

Programming Concurrency on the JVM: Mastering Synchronization, STM and Actors
by Venkat Subramaniam

This book should be considered part of a two book series along with Venkat’s 2009 book “Programming Scala: Tackle Multi-core Complexity on the JVM”. The earlier book was a relatively short (250 pages versus 852 pages for Odersky’s Scala book), perhaps too short, overview of Scala. It’s 12 chapters skip over lots of the details in Odersky’s 18 chapter book, and contains only a single chapter on Concurrent Programming.

By comparison the new book is all about Concurrent Programming models and attempts to be agnostic about the JVM language to be used. Most examples are first given in Java, then a simpler way is shown to do it in Scala, and then often a short section saying how the example could be done in Ruby.

The book describes the difficulties of concurrent programming in JVM languages especially given the rise of multi-core processors. It then describes three “solutions”: the naïve Java approach, Software Transactional Memory and Actors. It should be noted that the Actors chapter is based on Akka, so if you don’t care for Akka you may not like this book.

The Power and Perils of Concurrency – This should be an unsurprising recap of well known problems; if you’re reading this book you will almost certainly have run into the problems listed here already.

Strategies for Concurrency – Talks about ways to divide problems into parts that can be computed independently

Modern Java/JDK Concurrency – This talks in detail about the changes made to the Java libraries to better support concurrency, specifically things like using ExecutorService rather than “new Thread()” and using Lock vs. synchronized. Interestingly though the chapter fails to discuss the little known but serious differences between synchronized blocks and methods involving fairness and barging.
Taming Shared Mutability – This chapter begins to offer the Scala kool-aid of mutability is bad. Reading this chapter is made easier if you’ve already read Goetz’s “Programming Concurrency”.

Software Transactional Memory – This chapter talks about Clojure’s STM model and how to use it in Java via Akka. Its an interesting chapter but one that will feel odd if you’re a programmer also coming to terms with non-relational databases. As databases move away from the transitional transactional model here comes STM bringing transactions to mutable variables.

Actor-based Concurrency – Actors are basically a message passing schema where immutable messages are passed between lightweight actors (thread-like). Just as you can use thread pools to reduce the cost of threads there are what amount to actor pools which lighten their cost. It does require some rethinking to embrace the actor approach.

Overall this is a fine book that brings quite a lot to the table. My only reservation would be that if you are a seasoned Java programmer you might take the first four sections as a given and jump right to the Actors approach. And if you’re not a seasoned Java programmer you probably don’t realize how badly you need this book.

Books on Scala

2011-08-24T06:49:00.000-07:00

While the best way to learn a new language is to write in it there is still a great value to reading books on the language. When I was a C/C++ coder (a long time ago) I made it a point to read every book published on the language. The benefit of picking up even a single tip on how to use the language I spent 10 or more hours a day in was simply too large to ignore.

So, I decided to look at the current set of books on Scala. If you don't know Scala it is a hybrid OO/Functional JVM language.

Here are the books I found that are either in print or soon to be in print.

Title	Author	Length	Publication Date or Amazon Rating
Programming In Scala	Odersky	852 pages	4.5 stars
Programming Scala	Wampler	448 pages	4.5 stars
Programming Scala	Subramanian	250 pages	4.0 stars
Beginning Scala	Pollak	320 pages	3.5 stars
Scala In Action	Raychaudhuri	525 pages	(fall 2011 9 of 14 chapters available)
Scala In Depth	Suereth	225 pages	(Winter 2011 8 of 11 chapters available)

While book length is not a good predictor of quality its hard to see how you can cover a language like Scala in a short book. In particular "Scala In Depth" looks to be the shortest book!

Personally my approach to learning Scala was to put Subramanian's book on my right, Odersky's book on my right, and a Scala console in the middle. I then read the same chapter in each book, working through each and every example in the Scala console. This way I got the benefit of learning the same concept via the different wording of two top authors...and I forced myself to actually write some Scala.

For my next Scala reading I'll try Scala In Action simply based on past enjoyment of the "X In Action" books from Manning Press. None of them have ever let me down.

I should also mention two other related books:

"Seven Languages In Seven Weeks" by Bruce Tate (full disclosure: I'm interviewed in the book's chapter on Prolog)
and
"Programming Concurrency on the JVM" by Subramanian. While not strictly a Scala book it is still very interesting as it introduces Software Transactional Memory and Actors and shows how to use them in various jvm languages.

The most basic failure

2010-12-19T07:07:00.000-08:00

In this holiday buying season the Kindle and the Nook continue to be very popular, and the battle still ranges over the viability of dedicated e-readers versus general purpose devices like the IPad.

Poor Sony.

I own one of the Sony E-Readers and quite like it. It has many advantages over the Kindle, Nook or IPad and yet its largely forgotten. I have a theory as to why.

Sony forgot to give its product a name! Quick test: what is the name of Sony's e-reader? Its "E-Reader". Since thats also the generic term for the device it amounts to not having a name.

Sometimes having the generic name for the device is a good thing. Xerox has the generic name for the photocopier. The important thing however is the timing.

Its a Good Thing to have your product name become the generic name. Its a Bad Thing however to simply use an existing generic name.

Imagine a device making introducing an MP3 player and calling it "MP3-Player". Rather than subsuming all the good will associated with the general product this new device would feel like a cut-rate generic device.

I think thats what happened to Sony's device. You can't even talk about their reader. I can't tell you how many times people have asked me: "is that a Kindle?". I say "no, its Sony's Kindle", which is certainly not an answer to warm the hearts of Sony's marketing department.

I don't know what surprises me more, the fact that Sony forgot to name their device, or that no one seems to have noticed this failure.

Guest Blog over at RubyLearning.com

2010-10-25T06:12:00.000-07:00

I am the guest blogger over at RubyLearning.com this week. My article on the value of maintaining a personal bug log is at: http://rubylearning.com/blog/2010/10/25/the-value-of-a-personal-bug-log/.

Although I'm not a Ruby programmer the RubyLearning blog run occasional articles on learning in general.

Profiled for JavaOne

2010-09-09T09:07:00.000-07:00

I was recently profiled for the upcoming Oracle JavaOne. This was because of my open source project Log4JFugue.

The link is: http://www.oracle.com/us/javaonedevelop/future-of-java-168616.html#tarbox

It was interesting to sit back and think about what makes using Java interesting. It also reminded me of the freedom that comes from having an open source project. If I want to explore using Groovy I don't need to get the approval of a committee I can just do it.

Where do you get your Intellectual Stimulation?

2010-08-01T08:28:00.000-07:00

Most of us work in jobs that primarily focused on solving an immediate need for existing customer, who needs a quick solution today. This certainly pays the bills but generally isn't the kind of venue to leads to discussions about to create "software that sings". Indeed when I used that phrase in a previous article I got comments that even thinking about writing such software was beyond the pale for most people.

So, where do people find an outlet for that part of the mind that goes beyond solving the problem for today? Having just come back from the Atlassian Summit 2010 I can tell you how recharging it was to spend a few (long) days surrounded by people whose mission it was to be better. One of Atlassian's commitments this year is for their tools to save every developer 15 minutes a day. It was fun to spend time thinking, arguing, brainstorming about how to change tools and processes to squeeze out an incremental amount of productivity. And then I went back to work. where the focus is (as it should be) on the short term deliverable. I suspect I'm not alone in this regard and wonder how people recharge their batteries.

Some years back I visited The Orchard tea room in Cambridge, England. This humble tea room hosted Virginia Wolf, Bertrand Russell, Ludwig Wittgenstein and others. Its as if Edward Tufte, Richard Feynman and Claude Shannon hung out together at the local Starbucks. Imagine sitting at the next table from that gathering every week! Without putting myself in such company I wondered if such places existed today where motivated people could gather and discuss their craft.

Our conferences and User Group meetings try fill this role but to a very limited degree. Its tough to have an in depth discussion at a 15,000 person, $5000 JavaOne for example. User Space conferences may be better for this but they're still discrete events rather than something ongoing. I started wondering about other models such as the JavaPosse. That's a group of four friends who meet once a week or so and basically spend an hour discussing whatever they find interesting. While the Apple bashing/praising they engage in has gotten a bit tiring it’s still a very interesting model. The parts that stand out for me are:

- its a regular meeting rather than an occasional event
- it happens (mostly) in person. The four members actually sit together and talk
- it's outside the work venue and not paid for by my employer so I'm not obligated to focus on near term problems/solutions

I've decided to start a similar group myself, not with the intention of podcasting it but simply to spend some regular time talking with smart people about the craft. We currently have three members and are carefully thinking about a couple more people. We're also trying to determine what kind of meeting place we should have.

I wonder if anyone else has tried something like this or if anyone has other ideas for on-going mental stimulation?

Follow The Sun versus Chase The Sun

2010-06-21T16:04:00.000-07:00

Follow The Sun is the management concept that by spreading your development team across the world you always have someone who is awake and can work on a problem. In the imagined perfect world Team A works on a problem for their eight hour day and then hands off the partially developed feature to the next team arriving on the next continent. The idea is based on the notion that developers and software are both fungible assets. (Fungibility is the property of a good or a commodity whose individual units are capable of mutual substitution. Examples of highly fungible commodities are crude oil, wheat, orange juice, precious metals, and currencies. Wikipedia)

Since developer A is equivalent to developer B it just make sense for them to hand off bits of partially developed code to one another. (sic)

In the real world this often becomes Chase The Sun rather than Follow The Sun. There are two variations of Chase the Sun.

In the first variation all developers are basically expected to be available for questions and support 24 hours a day. Between Skype and text messages and laptops or smart phones we often see developers basically in perpetual on-call mode. This can actually be effective...until the team burns out.

In the other variation team A makes a change that breaks the system or changes it significantly, and then goes to bed. Team B comes to work to find a broken or "different' system and spends their day doing archeology on team A's changes. Successful or not they try to warn Team C about the state of the system and the changes that they've made in an attempt to resolve the issue.

Each team leaves emails that will be read 18 hours later by the team prior to them in the earth's rotation. Besides being inefficient this variation leads to predictably negative team interactions. Its just too easy to explain each day's problems as being caused by "them".

Chase the Sun can be changed back into Follow The Sun by following a few simple rules

a) developers are not fungible

b) there is enough code that the code also isn't fungible, so not everyone will understand everything

c) geographically dispersed teams can assist each other investigations or in remote pair programming where one team develops and the other tests

d) asking geographically dispersed teams to work on the same code or features is often a recipe for disaster

April issue of PragPub is out, see my article

2010-04-07T07:37:00.000-07:00

The April issue of PragPub, the magazine by the Pragmatic Programmers is out, and I have another article in it. The link to my article is http://www.pragprog.com/magazines/2010-04/medicine-making-music and the link to the whole issue is http://www.pragprog.com/magazines/2010-04/content

My article is about an NPR report I heard where some ER's are using music is a way that seems similar to my open source project Log4JFugue.

Check it out!

A trick for managing workspaces within Eclipse

2010-03-19T08:42:00.000-07:00

There are lots of ways to deal with the issue of multiple workspaces within Eclipse. The way I approach it is to have all of my workspaces live under /opt/workspaces and have them all available at the same time. This allows me to switch back and forth between projects easily. It does have a drawback however, but I've found a way around it and thats the topic of this post.

If you multiple workspaces open and you use the Cntrl-Shift-T class finder to navigate between classes you will be shown a list of with search results from all of the workspaces. Chances are you are only actually working in a single workspace at a time. Wouldn't it be great to only see results from the workspace you are actively working in?

There is a drop down arrow in the upper right corner of the Cntrl-Shift-T dialog. That brings up a "Select Working Set" dialog. Press "new" and then "Resource" to select just the project you are working on. From that point on your class searches will be limited to that project.

The one other step to do is in the Project Explorer select the project you are working on and from the right-click popup menu select "go into".

These two tricks let you have the best of both worlds, context that is limited to your current project along with trivial switching between projects.

Groovy XML parser for ITunes Music Library

2009-12-27T09:41:00.000-08:00

I work in the Video On Demand field and so am very interested in the distribution of content selection. In other words, which content gets played and how often.

A few years ago the standard thinking was that most people were watching the small set of popular content. This was known as the 80/20 rule, that 80% of people were watching 20% of the content. Then the notion of Long Tail came along which said that although some content would be popular the set of content that got at least some plays (i.e. the long tail of graph) was quite large.

Being a bit of geek I decided to write a Groovy based program to analyze the data in my ITunes library (even though this week I switched to the Motorola Droid).

ITunes keeps all of its information about your music, the play counts and so on in a file called library.xml. This file can get quite large so I decided to go with a SAX XML parser approach to minimize my memory consumption. The program reads the libary.xml file, extracts the Artist, Song title and play count and emits a csv file that can later be read by Excel.

The program is shown below:

import javax.xml.parsers.SAXParserFactory
import org.xml.sax.*
import org.xml.sax.helpers.DefaultHandler

class MyHandler extends DefaultHandler {
def tempVal
boolean expectingArtist = false
boolean expectingSong = false
boolean expectingPlayCount = false
String artist
String song
Integer playCount
def outFile = new File('\\musicPlays')

void endElement(String namespace, String localName, String qName) {
if(expectingSong) {
song = tempVal
expectingSong = false
}

if(expectingArtist) {
artist = tempVal
expectingArtist = false
}

if(expectingPlayCount) {
playCount = Integer.parseInt(tempVal)
expectingPlayCount = false
String thisLine = artist + '; ' + song + '; ' + playCount + "\n"
println(thisLine)
outFile.append(thisLine)
}

if(tempVal.equalsIgnoreCase('Artist')) {
expectingArtist = true
}
if(tempVal.equalsIgnoreCase('Name')) {
expectingSong = true
}
if(tempVal.equalsIgnoreCase('Play Count')) {
expectingPlayCount = true
}
}

public void characters(char[] ch, int start, int length) throws SAXException {
tempVal = new String(ch,start,length)
}
}

def handler = new MyHandler()
def reader = SAXParserFactory.newInstance().newSAXParser().xMLReader
reader.contentHandler = handlerdef inputStream = new FileInputStream('\\library.xml')reader.parse(new InputSource(inputStream))
inputStream.close()

This program let me analyze my listening preferences. Turns out I have about 8000 songs in my library, of which I've listened to about 3200 of them at least once.

I've listened to about 260 songs at least ten times and about 1000 songs at least five times.

The Killer App for E-Book Readers is Smell

2009-12-09T07:40:00.000-08:00

After resisting the Kindle for a long time because of DRM fears (I don't like to be locked in to a single vendor) I got a Sony EReader a few months ago. I love it, and I'm reading a lot more because its such a nice reading experience.

Many people have said to me that they are reluctant to go to E-Readers because they miss the smell of books.

So, I propose E-Book-Smell Strips! These would be strips containing the smell of physical books that you would stick to your E-Reader.

You could even charge more for "Hard Cover Smell"!

This month's issue of Pragmatic Programmers Magazine

2009-11-04T04:55:00.000-08:00

This month's issue of the Pragmatic Programmer's magazine is out and as always it is interesting, and in an unexpected way. This magazine continues to look at things a bit differently. For example, there is an article on hand writing a letter...remember when people did that? Its interesting to look at what has been gained and lost by the transition.

I have an article in this month's issue on interruptions, specifically interruptions of people who think for a living. I explore the idea that most current attempts to limit interruptions are doomed to failure because they place the cost of the interruption on the wrong person. Go read the article to learn more!

Mocks Run Amuck

2009-10-28T17:45:00.000-07:00

I just spent 2 hours debugging a problem that turned out to be tied to a lazy use of mocks.

In our core code we have a number of Mock objects defined for the key objects in our system. These are moderately functional mocks in that they support a fair bit of state. They are Good. They should be used.

I ran into an issue where I believed I was setting some state on a mock, and some later code appeared to be making the wrong decision based on the state of the mock.

It turns out that someone had needed a mock and didn't bother to see that a really good mock class already existed. So they created their own completely non-functional mock class, and they made it an inner class of the test class to boot. So, when I set the state on the mock I was actually calling a no-op method of the bad mock.

It took me a long time to come to grips with the fact that "my" mock object was misbehaving and not remembering state. I didn't think of this as an option because I had written that mock class and the state preserving code was trivial. So trivial that it could not be failing...unless I was actually using someone else' mock object.

Before writing a class, even a test related class you ought to look around a bit and see if the class already exists. In this case that's especially easy since the file will be named MockThing.java (or .groovy), and especially since Eclipse can find your class so easily.

Come to think of it, this new no-op mock class was probably created by Eclipse. Normally I'm a fan of tools making it easier to do things, but not when its the wrong thing to do!

Recharging Yourself; getting an E-Reader or a netbook

2009-09-09T17:13:00.000-07:00

Having written several blogs about why I don't like the pricing model of the Kindle book reader I've been very interested to read about Sony's new e-book readers, especially as you can download books to it from your local library. I've played with the $300 Touch model and quite like it. The only real competition to it for me is a netbook. If you're not familar with netbooks, they are basically really small (8 or so inch) laptops with limited disk and memory; designed to view documents and web pages but not really to do authoring.

The standard discussion goes like this: e-readers are single purpose devices, netbooks are general purpose devices. E-readers are better for actually reading books but can't do much else. Netbooks are less suited for reading but can do more. For techies like us the argument that no one would want to look at an LCD screen for hours at a time is kind of funny since that's what we do all day, every day.

I was about to get a netbook, especially since much of what I read are PDF versions of technical books from Manning Press or The Pragmatic Programmers, when a senior collegue made the following remark:

"Brian, we are thought workers, designers, creative people. We have an obligation to read non-technical material so that we can keep approaching problems from unexpected directions and keep bringing creativity to tasks. You should get the device that makes reading fiction easier".

That really struck me. So, after posting this entry I'm driving to the Sony store to buy the Touch E-Reader!

Classes = Structs plus Closures.

2009-09-06T05:39:00.000-07:00

I was leading our Groovy book club this week and had an insight. We have Java programmers, C/C++ programmers and script writers so we try to use a wide range of examples to illustrate points.

We were discussing closures, which can be challenging for people unfamiliar with them. I was making that point that Closures are Objects which means they are instances of classes. Particularly, they are basically classes with a (generally) single method and no data members.

That made me think about Structs, which are basically classes without methods.
So, I had the thought that Classes = Structs plus Closures.

I'm not sure if this is a helpful insight yet, but anytime I can think of things in a new way I like to explore it a bit. I think the C/C++ programmers in the book club found it helpful, perhaps because they're already used to thinking about different types of objects such as structs. So, thinking about Closures as another kind of class was not such a stretch for them.

New issue of Pragmatic Programmer's magazine

2009-09-03T18:53:00.000-07:00

The third issue of the new magazine PragPub from the Pragmatic Programmers is now available at http://www.pragprog.com/magazines.

In the interest of full disclosure I'll say that I have an article in this issue, so I encourage you to go read the issue just for that.

If you're a programmer you probably already know about the Pragmatic Programers and likely have some of their books, but you may not know about the magazine. With authors like Kent Beck, Dave Thomas, Andy Hunt (and me!) it's a worthwhile investment of your time.

The unexpected thing is that many non-programmers will find a lot of value here as well. One of the regular columns is called Get A Life and is about the various ways to find balance in life and recharge. I'm an apprentis instructor at a Tai Chi school and am passing this article around to the staff because it's so relevant.

Take a few minutes and check out the magazine, especially my article (with Dave Koelle) titled: "And Your Bugs Can Sing"

Thinking about levels of Risk and Releases

2009-08-27T03:26:00.000-07:00

One aspect of releases in the real world that seems to get little mention is the reality of working on more than one of them at a time. Many of us work in shops with a main branch and then multiple named releases hanging from various places off of the main branch. This is standard.

So, the inconvenient truth is that when I make a fix in release 2.3 it probably needs to also go in release 2.3.1, release 2.4 release 3.0 and "main" (whatever you call your root branch). I tend to think of those as pass through integrations. Someone made a fix in a past release, likely for a real customer (so it was done quickly) and then wanted to make sure it got propagated to all of the other releases that might correspond to any existing or future customers.

Now, the awkward question is: do you think the developer ran a full regression test on every release they checked the code into? Officially we all probably say yes, but in reality the answer is "I made sure it compiles" if you're lucky.

Depending on your branching strategy and the level of automation of your regression tests you may not need all builds deployable at all times or you may simply not be able to afford it even if you want it. If your tests are not automated it may simply be too expensive for each checkin to cost a developer-day of regression testing.

So, the two key questions are: how much risk do you have and how much can you afford?

A proposed approach to dealing with these questions is as follows. Each time a developer makes a check in to a branch the risk that the branch has destabilized has increased.by a quantum. Each time QA runs the regression suite the risk is reduced. Make the invalid but simplifying assumptions that all checkins add a constant risk and the regression suite reduces the risk to zero.

Imagine a graphical tool that shows a graph of the accumulated risk associated with each branch. This would allow you to answer the question of how much risk you have with each branch. Put another way, it lets you know how much deferred work would be required should any particular branch need to be released.

The Atlassian tool suite might already do this but I suspect that any development shop with a code management system and decent script writer could create a system for building such graphs.

Once you have the ability to measure your risk you can make an informed decision about how much to accept. You might decide that some branches should be regression tested everytime they’ve accumulated x checkins, while other branches only need testing every y checkins. The point is that you get to decide.

Spark: The revolutionary new science of exercise and the Brain. John Ratey, MD

2009-08-18T20:03:00.000-07:00

This book is strongly recommended reading for anyone who uses their brain.

This book starts with the assertion that conditioning the body is just a side effect of exercise, and the real benefit is the changes that exercise causes in and on the brain. Many people have made the case that our current lifestyle doesn't match the conditions we evolved for, but the assumption is that too many caleries and not enough running only effects the heart and lungs. Ratey argues that part of what we evolved for was hunting and gathering that required speed, stamina, cunning and fine motor control. We lose all of that when we're sedentary.

In a Chicago High School gym class was transformed into exercise class, with emphasis on effort as measured by heart rate monitors. This allowed even unfit kids to succeed, by working enough to elevate their heart rate. This school's obesity rate is now 3% compared to the national average of 30%. They are also setting records for academic performance.

This book has a lot of chemical names in it and one of the most important is brain-derived neurotrophic factor (BDNF). BDNF causes an increase in synaptic connections which is how we learn. This molecule is produced in the hippocampus and studies show the exercise causes an increase in its production. Ratey calls BDNF "miracle grow" for the brain. BDNF doesn't make you smarter but it creates an environment conducive to forming new connections. So, if exercise and then learn something its easier for the brain to encode the learning.

We used to think that we were born with all the brain cells we'd ever get, but we now know the neurogenisis occurs all the time. New neurons are born and then have 28 days to get connected into a network. If they don't they die. Exercise increases both the rate of nerurogenisis, and the ability of new neurons to make the connections they need to survive.

Ratey goes on to say that the benefit can be enhanced by combining aerobic exercise with complex activity such as Tai Chi. "The more complex the motions, the more complex the synaptic connections". Although these neurons are associated with exercise they can be recruited for other tasks (such as learning)

When under stress we produce cortisol which tells the hippocampus to selectively process data and memories (so as to focus on the stressor). While beneficial in the short term, in the long run it actually causes the non-stressor related nerves to degenerate and lose connections. This makes it harder for the non-stressor memories to be accessed, which can lead to more perceived stress. Exercise on the other hand causes a reduction in the production of cortisol, which can break the cycle.

The book does get somewhat repetitive as it explains the positive effect exercise has on the neurochemistry of anxiety, depression, ADHD, addition and aging. On the other hand you really can’t fault the author for providing so much good news. Each of these conditions represents a deviation from normal brain chemistry and exercise is a strong force for bringing the brain back into its normal condition of plasticity. Which is to say, that our brains were designed to be adaptable and exercise creates the right chemical soup for the brain to swim in to enhance the flexibility.

Another class of Bugs that Are Hard to Unit Test

2009-08-05T06:56:00.000-07:00

Imagine the set of classes listed at the bottom of this post. The point is that we're performing a calculation using an expensive test, but when using a derived class we can make use of an inexpensive test. Both tests will give the same result but clearly its better to use the inexpensive test if we can. How can we write a unit test that verifies that we've used the inexpensive test (and used it correctly)?

Several possibilities come to mind. One option is to write a unit test that times the call to the method. This is pretty clearly not a good idea as it leaves the results hostage to the load on test system.

Other approach is to embed a flag in the class that gets raised when the expensiveTest method is called. That's not a terrible idea but it does put test code in the actual production code. A slightly better idea is to create a test class that extends BaseThing and implements the flag in its own version of expensiveTest.

@Test testQuickReject() {

DerivedClass foo = new DerivedClass () {

boolean flag = false;

boolean expensiveTest() {

flag = true;

return super.expensiveTest();

}

};

foo.pickBestThing();

assert(flag == true);

}

This leads to an observation about the effect of testing on code design. The approach being taken only works if the expensive test is a separate and thus overridable method. If the code representing the expensive test was just a bunch of inline code then there would be nothing for the test class to override. One could certainly go back and to a Refactor:ExtractMethod on the code, but if the class had been designed with testing in mind from the start that would not be necessary.

class BaseThing {

Thing pickBestThing(List thingList) {

Thing bestThing = null;

for(Thing thing : thingList) {

if(quickReject(thing)

continue;

if(expensiveTest(thing))

bestThing = thing;

}

return(thing);

}

boolean quickReject(Thing thing) {

return false; // no-op method

}

class DerivedClass extends BaseThing {

boolean quickReject(Thing thing) {

// some real test that only applies in the derived class case

}

Sharpening Your Sword

2009-07-30T06:35:00.000-07:00

We’re recently reinstituted our technical book club, even though we’re probably now busier than ever before. I sent an email asking our technical staff if they thought there would ever be a time in the future where we would be less busy. If not, were they willing to permanently suspend their own technical growth? Interestingly enough that email generated a lot of interest in our book club!

Choosing which book to study is an exercise in learning about balancing your needs with the needs of the company and the other developers. Having recently attended JavaOne I can think of lots of interesting topics: alternate JVM languages (Groovy, Scala, Ruby), frameworks: (Spring Roo, Spring DM, OSGI, Jigsaw), gui languages (JavaFX, Flex), tools (Maven, Eclipse goodies), general (“Concurrency In Practice”, “Pragmatic Thinking and Learning”).

To some degree you need to build consensus around the book choice but remember that your choice doesn’t have to be perfect. You may also find as I did that most people are busy enough to defer the choice to you.

Having selected the book we’re taking a slightly different approach to studying it than we have previously. In the past we all read the book chapters offline prior to the meeting and then discussed the chapter during the meeting itself. When questions would arise we’d sometimes open up a laptop and try something but it was largely a text centric discussion. Since our current book is discussing a language that’s new to most attendees we’re going to be much more laptop and code-centric. Our first meeting in fact will be devoted to getting the language and associated tools installed on everyone’s laptops so that we can all type along with the examples in the book. The goal is to make this more than an academic exercise. We’ve all attended trainings that were interesting and yet covered technologies that we never touched again. We’re aiming with the code-centric approach to fairly quickly add this new language to the tool box of our developers and testers.

Some of the practical steps in that direction include:

Installing the IDE plug-in for the new language
Modifying our main system build to compile classes written in the new language
Adding a HelloWorld class in the new language to the source tree to ensure that the modified build scripts actually
work.

And probably most useful, pick an existing but non-critical bug/feature request in our product to fix by adding a new class in the language. I think that this set of steps will make the new language real for people.

As a side comment I say that I like asking interview candidates what books they’re recently read. It can be fairly revealing about the ongoing educational habits of the candidate. It can also reveal how they react to an unexpected question, and one that they might have answered “wrong”.

Learning from your unit testing mistakes

2009-07-30T06:33:00.000-07:00

By now one hopes that we don't have to convince developers of the need to write unit tests, and to use a tool like Cobertura to enforce some level of code coverage. And yet, there are other steps that really ought to be taken, steps that I don't see many development shops taking.

Even with fairly high levels of code coverage (e.g. 80% or greater), we still find defects in our code. The galling thing is that we find defects even in the code that Cobertura tells us we tested. How is that possible?

The first step in this process is a painful and often time consuming step, but one that can be very revealing. The next time a defect gets reported against 'tested' code I suggest you stop and write a unit test that finds that exact defect. (Now, many shops require a unit test for all defects so this step isn't new for them, but lots of other shops don't require this step).

The next step is the most interesting: reflect honestly on why you did not write that unit test already. Write down the reason in your engineering notebook (you do keep an engineering notebook, right?) Over time you may detect patterns in the types of tests you do and don't tend to write.

A fairly common case is where mainMethod() ought to have a call to subMethod() but doesn't. The defect is that subMethod() is not called. Even if you have individual tests that execute every line of mainMethod and subMethod you will not detect the missing call unless you go further. Presumably subMethod accomplished something useful, something that your tests of mainMethod could detect.

The question really morphs from line coverage to results analysis. As sad as it may seem many developers do not really understand that your unit tests are useless until you test the results. The following test increases your coverage numbers but doesn't actually test anything

@Test
public void myTest() {
int results = mainMethod();
}

There are static code analysis tools such as PMD that will flag unit tests without asserts. Try adding one of these to your build…but don’t be surprised if many of your unit tests turn out not to be tests at all.

A question that has bothered me about at this is what to do about stub modules. We sometimes have to implement an interface (a bad interface) that has dozens and dozens of methods, only a few of which are actually necessary. Your concrete class has to have stub implementations of the unused methods, which leads to a code coverage decision. You can write unit tests for these no-op methods, but since they don’t do anything you have nothing to test (which may be hard to explain to your static analyzer). Or you can skip testing these modules which may reduce your coverage numbers enough to get your coverage enforcement scripts to complain. I’d like to see an annotation of some sort like @STUB that told Cobertura to ignore this method or class for calculating its coverage metrics.

JavaOne Thoughts and Observations

2009-06-04T23:44:00.000-07:00

It's now the second to the last day of JavaOne 2009 and I've had a chance to reflect a bit on what's been going on. First, I have to talk a bit about my experience as a speaker at the conference.

- On Tuesday we got interviewed by Java Talk Radio (who knew there was such a thing!); the interview is available at: www.log4jfugue.org/javaone_interview.mp3.

- I had a conversation with Rod Johnson (inventor of the Spring Framework) and he later tweeted that he thought Log4JFugue was a "brilliant" idea.

- Dave Koelle and I gave our presentation yesterday and it was extremely well received. The room was full, not a single person left during the presentation, we got laughs and applause in the appropriate places, and several people approached us afterwards asking to collaborate with us.

- I got to play my Native American Flute during the presentation to illustrate a musical point...and I really enjoyed performing. I think I should do more of that.

- This morning at breakfast I sat down at a table where some people were already sitting, and I said hello. One of the people looked at me and said "Hey, you're that Log4JFugue guy!". That was fun, and it turns out the person wants to work with me on some extensions to the product.

Switching gears to the conference itself several key ideas stand out.

- cost cutting was painfully present in lots of small ways: the conference backpack was very cheap, the food quality was much reduced from previous years, most of the entertainment was recorded rather than live, and most significantly the conference attendance was way down.

- there didn't seem to be an overall theme to the conference. I think the official theme was ubiquity but it didn't feel pervasive.

- no one would touch the question of whether there would be a JavaOne next year...even Larry Ellison who made a guest appearance ducked the question. Its possible that this is due to a "quiet period" required by the upcoming merger but uncertainly is scary.

- dynamic languages or more generally other languages that run on the JVM continue to gain in popularity. I think this is a really good thing. When I went to school engineers were expected to be fluent in multiple languages (assembler, fortran, pascal, prolog, etc). But after a while it seemed to C/C++ and then Java took over and a generation of programmers grew up thinking that they only had to learn a single language. It feels like things are coming full circle back to the notion of using different tools for different tasks. So Java 7 won't have closures? No worries, just write the method needing the closure in Groovy.

Super Survey - An idea for a new kind of College Class

2009-04-07T09:25:00.000-07:00

Super Survey is an idea generated by my somewhat eclectic college education but one that I think might have broad appeal.

I spent my freshman year at Kenyon College where I expected to be a biology major. I had placed out of Intro Biology by getting a good score on my Advanced Placement Biology test but somehow got shut out of the next biology class and so ended up taking Intro to Philosophy...and discovered that I loved it.

I transfered to Wesleyan University after freshman year and continued to take philosophy classes as well as computer classes. I spent a junior year semester at Goldsmith's College of the University of London. All in all I took classes in 14 different departments:

athletics (fencing)
anthropology
biology
chemistry
economics
english
geography
government
math
music
philosophy
political science
psychology
religion

I got to thinking that most people didn't get the opportunity to take such a broad range of classes, or if they did it happened during senior year. I can't imagine only discovering my passion at the end of my college career.

Super Survey is the simple idea of having each class be the Introductory lecture of a different department. In a 3 day a week class with a 13 week semester you can exposed to 39 or so departments. The class would be pass/fail based on attendance so the effort for the students would be low. Each department only has to give one lecture so the effort for them would be low as well; though they have incentive to make it a good lecture.

The smaller departments have a special incentive to give a good lecture as they can almost always use more students. If a school has more than 39 departments they could ration participation in the class to give preference to the departments with lower enrollment

When I look at a course catalog I'm struck by the departments I never touched (at least undergraduate), at least that list is shorter than the list of departments where I did study:

art history
astronomy
dance
history (don't get me started on how badly we teach history, see * below)
linguistics (took post-grad)
physics (took post-grad)

* In Junior and Senior High School we studied the revolutionary war every year. We never studied any non-US history, and we never studied even US history after the revolution. I can tell you when the War of 1812 happened, and who the Spanish-American war was between, but that's about the limit of my knowledge. For a high end school system (Winchester, MA) that's pathetic.

When I tried to take history in college I found the classes to have a micro-focus such as "The Japanese Imperial Court from 1901-1902" (I'm not making up that title). I was looking for (and am still looking for) what I'd call Meta-History...what Jared Diamond writes about...the large scale sweeping trends of history.

Package Scope, the often forgetten good choice

2009-03-31T15:55:00.000-07:00

Scoping refers to how software programs control which parts of the system can see which other parts. Back in the day, most things were global which meant that everything could see (and touch) everything else. This made writing the code easy, until you had a variable suddendly change out from under you. You then got to spend long hours finguring out who might have changed it...and you had to look everywhere because everyone had access.

Programming languages such as Java provide a number of scopes that can be specified to control who can see a particular variable. Ask the average Java programmer how many types of scope there are and they'll probably say three: public, private and protected. Public scope is like the old global...a public object can be seen and touched by anyone. Private scope is basically the opposite...a private object can only been seen by the class that owns it. Conventional wisdom says that most things should be private. Protected scope is a variation on private. A protected variable can be seen the class that owns it or by any class derived from that class.

Thats the answer you'll get from the average programmer, but its the wrong answer.

The right answer is that there are four scopes in Java. The scope that most people forget about is package scope. An object with package scope can be seen by any other object in the package. This is a surprisingly useful scope and so its a bit surprising that so many programmers ignore it. Part of the reason for that is that while the other three scopes each have keywords, you get package scope by not specifying any scope. Personally I think the language should have included a package_scope keyword so as to make this clear but thats water under the bridge.

Why is package scope so interesting?

Consider a software system with two classes: One and Two. Assume that we want to expose two methods to the outside world "begin()" and "end()". Further assume that each class needed to call a method in the other class. The following is the sort of code that most people would write.

package sample;
public class One {
public begin() {
new Two().internal();
}

public internal1() {
}
}

package sample;
public class Two {
public end() {
new One().internal1();
}

public internal2() {
}
}

Most people would tend to make the two internal methods public so that the two classes could use them. The problem with that is that it makes the two internal methods part of the publically accessible API of the overall system. A user of this package has no particular clue that they should or should not call those internal methods.

A better approach is to remove the "public" in the declaration of the two internal methods. This leaves them at package scope, which is just fine. The two classes are in the same package and so have access to all package scope methods. The two internal methods are now however absent from the publically visible API of the system, which is as it should be.

So, the rule of thumb should be:
default to making classes and methods package scope
make methods only used within a class private
and, only grudgingly make the hopefully small number of truely public methods be declared as public.

Your users and maintainers will thank you.

My JavaOne proposal was accepted!

2009-03-02T13:57:00.000-08:00

My proposal to give a talk at JavaOne with my friend David Koelle was just accepted.
We'll be giving the following presentation in San Francisco in June.

Abstract: Would you like to create Java™ technology-based programs that play or create music but don't know where to begin? Come to this session to learn all about JFugue, an open-souce API that enables you to program music with ease. With its simple but powerful API, new UI components, and cool features, JFugue promotes creative music programming and exploration. For example, what if you could listen to what your application has been trying to say to you? Learn about Log4JFugue, which combines the power of Log4J and JFugue to turn your application's logging into a real-time song. By listening to your application, your pattern-matching brain can detect subtle changes in behavior that would normally be lost in a sea of log messages. The intended audience for this technical session is developers at any level who are interested in writing musical programs or who would like to use more parts of their brain to increase their productivity. In the session • Learn how to get and use JFugue • Learn about some advanced and exciting features of JFugue, including new ones • Learn about Log4JFugue for turning your log files into songs

Re-experiencing content, followup

2009-02-23T09:47:00.000-08:00

I've had several interesting conversations about re-experiencing content since the last post.

One friend asked about the phenomenon of people getting stuck on a particular content and watching / reading it over and over again. While this is usually associated with younger children who can watch the same episode of Thomas the Tank Engine multiple times a day, it sometimes occurs with adults as well. I think these are cases of the person looking for comfort through familiarity. When life seems unpredictable it can be comforting to revisit a content that won't surprise us. For small children most of life is unpredictable. Try to imagine how weird the world must seem before you figure out object permanence! I know that when I'm sick I'll tend to dig out an old movie to watch. This might also be why we watch certain movies (It’s a Wonderful Life) every year at a special time.

I’d distinguish this from watching a movie a few times to hear or memorize more lines, or to experience it again with the ending already known. I’ll confess to having watched some Monty Python movies too many times so as to memorize the lines from a scene or two. Inexplicably this is sometimes viewed as an accomplishment. :-)

Another reader wrote me to say: "These are very interesting matters. I haven't thought about them in an organized way before. I can watch a movie a few times (not multiple times, unless there's a lot of time in between). I've always supposed that the reason I want to listen to music over and over again is that I cannot really retain music--I mean I can remember short songs etc, but I can't unroll a whole symphony in my head, just parts of it, although musicians can replay the whole thing in their heads. I think I don't have the capacity. On the other hand, words stick in my head like glue. I have hundreds, maybe thousands, of word things, in my head but not a whole symphony or sonata. I remember songs, but probably because of the words. Do you think the content thing may have something to do with one's capacity for enjoying a particular medium? Some people are visual, verbal, or whatever? Interesting things to ponder"

Another person said that for them music could be re-experienced many times because they were different each time they heard the song. When it was pointed out that they were also different when they re-read a book they said that experiencing a book or movie was more immersive than listening to a song...there was less room for them. Perhaps listening to music is more of a participative thing while books / movies are more like being an observer? I don't necessarily share this belief but I can understand it. After all, I can listen to my ipod while coding at the office but they tend to frown on people watching movies there!

I wonder how this relates the concepts of Popular vs. Long Tail content? One school of thought says that most people will be watching the same small set of popular contents (the new “House” or “Heros” or “Battlestar”). The other school of thought says that most people will be off in their own corner watching their favorite old episode of “I Love Lucy”, “Hogan Heros”, or “F Troop”. For those of us designing Video Servers the difference matters. I’ve never heard the issue of whether individuals re-watch a given content brought up in the Popular vs. Long Tail discussions. Perhaps it should be.

The Kindle and the cost/payoff of experiencing content again

2009-02-15T18:15:00.000-08:00

In thinking about the Kindle (Amazon's electronic book reader) I got thinking about a larger question about how people experience content. I'm one of those people who can watch the same movie multiple times, and I tend to accumulate tapes and DVDs of my favorite movies. My wife has never understood this, once she's seen a movie she's done with it and has no interest in seeing it again. We've both tended to think the other one was rather peculiar. We were at a brunch today and I tried to enlist support for my way of thinking but found surprising diversity.

Some people viewed movies as one-shot experiences but would read a book multiple times, others viewed both movies and books as one-shots, but most people viewed music as something to experience over and over again.

So I asked people: what's different about experiencing a song from experiencing a movie?

Several people said that while movies are based on a narrative, they did not see music that way. Several said that they did not feel that songs told a story (some of the others of us found that idea shocking...of course a song tells a story...for us). So for these people a movie is boring the second time around because the know the story but a song is just an experience ... so repeating it is still rewarding...perhaps like eating ice cream? The fact that you've eaten ice cream before doesn't diminish your enjoyment of more ice cream.

I wonder if there is a different kind of answer underlying this: limited time. I can listen to a song in 2-3 minutes, watching a movie takes about two hours, and reading book might take 5-10 hours. Who has that kind of time for a repeat?

On the other hand, there is a payoff for getting involved in a long story...you get to know the characters. I tend to dislike short stories because why should I get invest time in a story that won't return my investment? Think of the angst many people feel when a beloved TV series ends (The Sopranos, Friends, Mash, etc). There has been an investment and the end of the story brings an end to the return on that investment. For some of us, re-watching a movie gives us some extra return on the time we spent on the content in the first place.

The link back to the Kindle for me is that for many people the fact that Kindle does not contain your existing library of books is simply not an issue. Many people view their existing library of books as more or less dead storage; perhaps to be rarely accessed at some time but certainly not in need of ready access. From this point of view loading the Kindle with only new material is just fine. I suspect that this sort of person would also be happy with a Kindle-like device with very limited storage, perhaps only enough to hold the books you were currently reading.

How Digital Cameras remind me of the old B&W days

2009-01-24T04:40:00.000-08:00

Recently my Tai Chi instructor asked me to take pictures of my class for the school's web site. His digital camera has some shutter lag which makes photographing people in motion a hit or miss proposition. So, I took a hundred or more shots, knowing that I only needed a couple of good shoots.

As I was doing this I was reminded of my old days learning to shoot black and white photos, (which happened long before there was such a thing as digital photography) and I was struck by the similarities.

I learned photography from an old friend, long since passed away named Christopher Resnik. Chris taught photography from the ground up. We bought 100 foot rolls of 35mm film and made our own film cartridges. Cameras back then were fully manual and Chris wouldn't even let us use a light meter...preferring to train our eyes to read the light and determine the exposure. To this day I can look around and know that such and such a day requires an F8, 1/250 second exposure with 400 ASA film. Today most people probably don't even know what those terms mean.

Since we bought film in bulk and did all of our own darkroom work it was quite inexpensive and so we shoot a lot. Chris used to say that is you got one or two good shots out of a 36 exposure roll you were doing well. When we found an interesting subject we'd take a number of shots of us...from various angles, with various exposures and with various perspectives. We might take whole roll of film on a single subject. Taking 100 Tai Chi shots reminded me of that.

Back in the days before digital cameras a roll of color film might cost $3-$5 to buy and then perhaps $10-$12 to process. With that price model you were careful how you spent your photos and a roll of film might sit in the camera for months or a year.

We've now come full circle in a way. Photos are basically free, especially since many people never make prints but instead view their photos on a digital picture frame or online via Snapfish or KodakGallery.

A missing element for most digital photographers however is the culling process. Back in the day, after processing a roll we'd stay in the darkroom evaluating our photos and would throw out 90% of them. We shot 36 pictures of that flower so that we could find the best one, but then we left the other 35 shots on the darkroom floor. Today most digital photographers take the 36 photos, post them all to Snapfish and go on their way. After a few months they have 100s or 1000s of pictures, most of which are awful and have no idea where the dozen or so gems are.
I'm sure there's a metaphor in here for the pace of modern life but it escapes me. I do however like when life comes full circle.

For my own pictures, (and I have a 3 year son so I take a lot of pictures), I've developed a culling process. When I plug my digital camera or my IPhone into the computer it creates a new directory where all the pictures go. I then create a subdirectory called so_so_shots. I immediately look at each picture and if it isn't great it gets moved to the so_so_shots directory. Since I'm just moving them rather than deleting them I can be quick and ruthless in the decision. This lets me easily manage a much smaller set of much better pictures.

Why I don't want a Kindle yet

2009-01-20T12:17:00.001-08:00

The Kindle is Amazon's book reader and its an amazing device, but one that I don't want yet.

The Kindle has an astonishingly readable screen (using electronic ink rather than a display tube or LCD). This means that once its painted a page there is no electricity needed to maintain the image so its always on, and the battery life is very good.

So why don't I want one of these cool devices that would let me take all/most of my books with me? Because Amazon wants me to pay for all of those books again.

As I look at my book shelves I see hundreds of books, most which were purchased from Amazon. They know this. And yet, when I buy a Kindle I have to turn around and repurchase all of those books again if I want to put them on the device. And I have to pay what amounts to full price.

When you buy a Kindle Amazon should give you and electronic copy of all they books you've purchased from them, say in the last couple of years. Until they do something like that the advertising line that you can take all your books with you simply isn't meaningful.

A Tutorial for Adding Groovy to a Java Project

2009-01-08T09:45:00.000-08:00

I program primarily in Java and have done so for close to ten years. Before that I programed in C++, C, Pascal, Prolog (yes, I had a real job using Prolog!), and even Fortran and assembler. So I know that languages wax and wane in their importance. So, I decided to add a class or two written in the Groovy language to one of my side projects.

For those not familiar with Groovy its one of a surprisingly large group of languages whose source code is different from Java, and yet runs on the Java runtime. Java source code is compiled into Java bytecode which is executed by the Java Virtual Machine (JVM) on your computer. Groovy source code is also compiled into Java bytecode which then runs on the very same JVM. This allows the two languages to interoperate in interesting ways.

As a starting point for this tutorial I assume you have an existing Java based project, use Ant to build your system and use Eclipse as your development environment. None of that is required for Groovy but it is a fairly standard setup and its what I use, both at home and at work.

The steps are basically
a) download and install the Groovy GDK
b) add a how-to-compile-groovy task to your Ant build.xml (the task is included in the GDK)
c) add the groovy-all.jar file to your library path
d) add the groovy-plugin to your Eclipse project
e) enable groovy-nature in your Eclipse project
f) write a Groovy class and use it.

Overall this should take no more than twenty minutes or so. One of the really cool things is that once you have a Groovy class you can use it from your Java classes just like any other class...your Java code has no idea its using a class written in Groovy.
Now lets look at the steps listed above in cookbook manner.

a) go to http://groovy.codehaus.org/Download and select the latest stable release (currently 1.5.7). Click on the download link and follow the very simple instructions.

b) add the following task to your build.xml to define how to compile groovy code: You then need to actually compile your groovy source files. A standard build.xml will have a compile target that invokes the javac task compile your java source files. I changed that so the my compile task was empty but depended on java_compile and groovy_compile targets. See the full definition of the groovyc task options here: http://groovy.codehaus.org/The+groovyc+Ant+Task

c) the Groovy GDK includes a groovy-1.5.7-all.jar file. Add this to your library and execute paths

d) From Eclipse, select Help, Software Updates, Find and Install, Search for new features to install, next.Now select New Remote Site. Enter Groovy as the Name and http://dist.codehaus.org/groovy/distributions/update as the URL.Press Ok and you should find Groovy added to the list of Sites to include in search. Press Finish.It should find Groovy, check the features you want and press Next.Accept the license terms and press Finish. At this point the plugin will be downloaded and installed.

e) From Eclipse select your proejct, right click and select Groovy->Add Groovy Nature. Eclipse will now understand files with the .groovy extension.

f) Remember that any Java file is also a Groovy file, so if you want to start with baby steps just rename one of your java files to have a groovy extension. Eclipse and Ant will recognize it and compile it to bytecode and then allow you to use it.

You could then take another small step and remove most of the semi-colons, because Groovy doesn't require them.If your class has import statements from various java packages you can probably remove them as well since Groovy automatically imports them.

Clearly you then want to write a class that actually uses Groovy features but that's beyond the scope of HelloGroovyInAJavaProjectWorld.

Enjoy.

Tai Chi, Software Design and simplicity

2009-01-05T18:26:00.000-08:00

Its always interesting when different aspects of life intersect. In this case Tai Chi, Software Design and simplicity. One of the lessons I've been learning in my Tai Chi studies is the value of simplicity and minimalism. Most of the flaws in my technique end up being attempts to do to much. My motions tend to be too large and often too complex. Such large and complex motions are relatively easy to defeat.

In software as well there is the truism that the code you don't write doesn't contain any bugs. There are many possible refactorings that can increase the quality or robustness or performance of a program, but code that no longer exists takes no time to run, doesn't need to be learned and doesn't obscure the structure of a program.

The trick of course is that simplicity is hard. My Tai Chi Shifu (instructor) gives me simple instructions from which I somehow deduce complexity. Only after continued study am I sometimes able to get the flash of "oh, that's all I have to do?" as I see the simple and powerful move hiding inside my attempts.

Even though I know this to be the case I do not seem to be able apply to new moves I learn. There seems to be need to for the learning to be a process. The simple move doesn't seem to be obvious so I flounder about adding unnecessary nuances.

This makes more sense to me in software design, perhaps because I've been doing it for so long. In software I do have the ability to move fairly quickly to a simple design, and to spot the extraneous complexity in most designs. In software the simple code path usually seems to leap out of the spaghetti I often encounter.

One way of looking at this might be via the Dreyfus model of learning (see Pragmatic Programmers). The Dreyfus model describes five stages of skill:
Novice, Advanced Beginner, Competent, Proficient and Expert.

Novices need to be given a cookbook description of how to perform a task, while Experts simply do the task and might not even be able to describe how or why they are doing it. The inability to describe why can be a problem. I've had bosses ask me why I think design X is better than design Y. My answer that Design X "sings" or is "beautiful" while Design Y "feels" clunky does not tend to convince them.

I remember watching a documentary about an about-to-retire Cotton Inspector trying to teach his apprentice how to select the best cotton. The expert said to rub the cotton between your fingers and feel when it was "good". The apprentice had no idea what that meant but it was the best answer the expert could give him.

I think that most people stay in their comfort zone and thus stay within whatever skill level they've achieved. While I might claim to be in the expert group in software design I'm certainly no higher than advanced beginner or competent in Tai Chi. Hopefully this makes me a better instructor as I can relate to the students just starting to study Tai Chi.

Whether all of this eases my learning curve with Groovy or Scala (new computer languages) remains to be seen :-) It does however remind me of the quote by Blaise Pascal: "I have made this letter longer than usual because I lack the time to make it shorter".

software haiku

2009-01-03T05:45:00.001-08:00

I wrote this haiku a couple of years ago to describe the software I was working on and it seems pretty appropriate to Wabi Sabi. For non-software developers spaghetti is an adjective used to denigrate code whose logic seems to go in all directions at once with no hope of untangling it.

Code in Haste no Pattern
Spaghetti's not food for the brain
The mind recoils, I weep

A Cost Model for Anticipatory Design

2008-12-31T11:24:00.000-08:00

Anticipatory design, as coined by Ron Jeffries, describes the extra effort a software developer might do to anticipate new or changing requirements. It’s interesting to attempt to model the cost of doing such work, particularly in light of XP or Agile methodologies. The goal is to come up with a rigorous basis for deciding when to do “extra” design up front.

A key variable in Jeffries model is R, the cost of designing a feature only when it is explicitly needed. Jeffries explores the cases where R ranges from 1 (implying no extra cost) to 5 (i.e. doing the design later is 5 times more expensive).

We extend the treatment of R in two directions: scale and variability. We believe that 5 is by no means the upper bounds on the cost of late design. We also believe that R comes in at least three variations.

We believe that there are actually three distinct class of R: R-normal, R-catastrophic, and R-XP.

R-normal is the R described by Jeffries in which doing the design later incurs a small but noticeable cost.

R-catastrophic is the case in which doing the design later incurs a huge cost.

R-XP is a class surprisingly not described by Jefferies in which doing the design later actually decreases the cost. This would occur in cases where it turn out that a needed component had already been created by someone else (within the group or as a third party purchase/download).

The question becames how to quantify the relative frequency of the three classes R and to see how that would change the overall cost equation. We began by creating a model in which

R-overall = (R-normal * percent occurrence) +
(R-catastrophic * percent occurrence) +
(R-XP * percent occurrence);

We set R-normal to 5 (as per Jeffries), R-XP is set to -5 (i.e. a saving) and R-catastrophic to a huge value such as 1000 (that is the definition of catastrophe after all).

Given these conditions the cost of catastrophic changes almost always overwhelms the effect of the R-normal and R-XP components. Only when the frequency of catastrophic change becomes extremely low or the definition of ‘catastrophe’ is set extremely mild do the other components have a real influence on the overall cost.

The question then becomes: does the practice of delayed design, or XP as a whole, itself influence the occurrence rate of catastrophic design failure? This would seem to have to be true in order to explain the successes being reported for projects using XP techniques. It also seems to be implied by Martin Fowler’s seminal work on refactoring. When discussing design Fowler advocates doing just enough design so that there is confidence that the resulting code would be refactorable. In other words, do enough design to leave the code resilient to change and thus resistant to catastrophe.

The conclusion to draw from this analysis is somewhat against the conventional wisdom. Our analysis indicates that the benefit of XP is not so much that it is faster but rather that it is safer! XP is often considered a high risk strategy and yet our analysis shows the exact opposite. Done correctly, XP leads to just enough design to avoid the castrophes that delay or destroy so much projects.

(portions taken from an unpublished paper written in collaboration with Morgan Creighton, 2001)

one more way to make cell phone vibrate mode more usable

2008-12-31T07:44:00.001-08:00

I promise this will be my last post about cell phone vibrate mode :-)

The biggest problem with vibrate mode is that people don't use it.
The second biggest problem, for me, is setting it and then forgetting to unset it which can result in missing calls.

Currently setting your phone to vibrate mode is binary, you do it or you don't. Wouldn't it make more sense to tell your phone to go into vibrate mode for the next hour?

Imagine a configurable go-to-vibrate-mode-for-n-hours option. Set it for 1, 2 or 3 hours and from then on when you went into vibrate mode it would return to normal mode when the time expired.

Fewer missed calls...and maybe fewer lost phones. Ever misplace your phone and then try to call it so you can find the phone by listening for the ring? This doesn't work if your phone is in vibrate mode (which Murphy tells us will always be the mode its in when you lose it!). With auto-vibrate-reset this problem goes away.

Ok, I'll get back to discussing software, cognition and philosophy now. :-)

Is Ruby the new "C"? Decoration vs. Terseness

2008-12-31T06:00:00.000-08:00

Hopefully the title of this post sounds odd. Asking if Ruby is the new Java would be more expected, but I want to look at a different aspect of language evolution than just how Ruby compares to Java.

Way back when I started playing with computers we had lots of languages to choose from, Pascal, Fortran, Lisp, Prolog (my personal favorite for some tasks). At that time is was standard and expected that an engineer would be fluent in multiple languages.

One of the aspects of C that is striking is how terse the language can be. Complex statements such as z = (++x > y) ? m : n; can be written with what Tufte might call a high data to ink ratio (http://www.infovis-wiki.net/index.php?title=Tufte%2C_Edward). To put this in the modern vernacular: C has low ceremony, most everything you write matters, there is a low degree of extra tokens needed just to keep the compiler happy.

As we moved to a Java based world where many/most engineers seemed to think they only needed to know one language, we found ourselves with a language with a bit more ceremony. To create a new object I need to say:

MyClass myObject = new MyClass();

Thats a lot of tokens just to get a new object.

One of the stated advantages of some of the newer languages such as Ruby is that they are low on ceremony. Sometimes writing fewer token makes the code simpler but sometimes it simply makes it terse.

I wonder if somehow we're coming full circle back to the asthetic of C.

Another way to automate cell phone etiquette

2008-12-31T05:48:00.000-08:00

One approach that some people have proposed to eliminate intrusive cell phone ringing is to jam or shield an area from the cell phone signal. Its actually surprisingly easy, but possibly illegal, to block the signal. A group in Japan has experimented with adding metal oxides to paint, a wall painted in this way appears to block cell phone reception.

I think this is the wrong approach, and not just for the standard "what about the doctor who much always be reachable" argument.

A better approach would be to standardise on a signal / text message / whatever that could be broadcast to a cell phone to request that the phone turn itself into vibrate mode. Movie theatres, resturants, etc. could simply broadcast this signal and compliant phones would become well behaved. Considering the venom we have for people who interrupt us with inappropriate cell phone ringing non-compliant phones might soon find themselves at a competitive disadvantage.

Alternatively, location aware phone like the IPhone could take a pro-active approach an notice their own location. Imagine being able to configure your IPhone so that whenever you were "near" a location on your Be-Polite-List it automatically went into silent mode.

Bad cell phone etiquette is the cell phone's fault

2008-12-31T05:23:00.000-08:00

Though its hard to remember life without a cell phone it also hard to imagine not having to complain about bad cell phone etiquette. As fun as it to complain about people forgetting to set their phones to vibrate in the movies, at dinner, etc., I propose that the solution to this problem is to fix the phone, not the user.

The cell phone is a small computer, and we should offload standard tasks to that computer, and setting your phone to vibrate in certain well known conditions is a standard task.

My IPhone for example has a build in Calandar, and is well setup to know about my Google or Microsoft calendar as well. I think my phone should notice when I'm in a calendar event and automatically set my phone to vibrate mode. How hard is that? Imagine your phone being smart enough to know when you're in a situation where a silent ring is more appropriate than an audible ring.

This approach would have the additional benefit of automating the return to audible ringing. I suspect I'm not the only one who sets their phone to vibrate before going into the movies and then forgets to turn the ringer back on at the end of the movie. I've missed a lot of calls that way. Noticing the end of an event and taking appropriate action, turning back on the ringer, is another task we should hand off to the phone.