Retrospective Velocity - Part 1 of 3

So how long is this going to take?  This is one of the most common questions to ask or be asked in a traditional software development environment, and one of the most difficult to answer.  The agile methodologies, and Scrum in particular address this problem with the notion of velocity.  Whatever your team accomplished in the last time period (sprint) is likely similar to what they’ll be able to accomplish in the next one.  Scrum’s use of burn down charts, standard definitions of “done” and retrospectives allows for the discovery of a team’s average velocity.  One can argue about hours versus story points and many similar details but almost any implementation of Scrum allows for tracking how much work was done per unit time.

Sadly, many projects are still using “traditional” methods (which seems to be one of the new names for waterfall).  One of the drawbacks of this approach is that the time scales are relatively quite large compared to Scrum, which also means there are fewer time periods to measure.  There is simply less data available is you have two six month buckets versus 12 one month buckets.  This is one reason that “traditional” projects do not typically end up producing a team velocity.  This in turns make it substantially challenging to estimate how long the next project will take.

There is however other data that can be mined so as to uncover this more fine grained time measurements we are looking for.  The data in question lives in your requirement tracking system, bug database and source code control system.  While these systems are often, shall we say imprecise, they do contain useful data.

Using QuickSort to Explore Scala Collection Methods

In my continuing effort to play with my new favorite language Scala I decided to look at one of the Standard Algorithms in the field: quick sort. Quick Sort is a divide and conquer algorithm that picks a pivot point in a list and then calls itself to sort the two halves.


StackOverflow.com demonstrated a very simple Scala quick sort shown below. Let’s examine it as a learning vehicle. The complex looking first line defines a function that takes a List of things T where T is any class that supports "Ordered". It also returns such a list.

 
 
 
 
 
 
 
As with almost all recursive algorithms quick sort has two cases: the empty list and the non-empty list. Scala's "list match" sets up those two cases. In the case of the Nil list we return Nil which is how the recursive algorithm completes. Otherwise we match against "cdr::cons". That expression means "first item in the list followed by the rest of the list". Cons and cdr are the standard names that the Lisp language uses for those terms. This may seem like an odd way to view a list but in the functional world it is a very common idiom.


So given a first element and a rest-of-the-list what do we do? We use another of Scala powerful collections function: partition. "cons partition (_ < cdr)" says "create two lists: a list of the items in the list that do match the condition and another list of the items that do not match the condition. These two lists are returned into the anonymous tuple “val (before after)”. This shows how Tuples are first class citizens in Scala, allowing us to return multiple values from a function call.

Searching the web for Java implementations of quick sort shows the following code as quite typical for the partition portion of the algorithm:

I don’t know about you but Listing 2 looks prime for off-by-one errors and is nothing I’d expect to get right the first time. Compare that to: val (before,after) = cons partition (_ < cdr). The beauty of that line is that its close to the English description: create lists of the items before and after the specified item by partitioning the list based on a test. I’m beginning to feel that Scala feels hard not because it is hard but because it’s so different from Java in that its easy!

Lastly we have the recursive part of the function. We call ourselves on the before list and the after list, and build a new list of the results of those two calls plus the cdr value (because it’s not in either list). This implementation works and has the advantage of being a tiny bit of code. The drawback is that it always picks the first item in each list as the pivot point. It is known to be sub-optimal especially in the case where the function is called on an already sorted list.

This next version of the program picks a better pivot point, in this case the middle entry of the list.  We accomplish this by changing the "case" to "theList : List" which just  means any list.  We manually find the pivot point and then perform two for-comprehensions to find the list items larger and smaller than the pivot entry.  This gains performance via the better pivot point but trades that off against having to scan the main list twice for the two for-comprehensions.  Further testing revealed that this implementation also had a defect in that if an item was present in the list more than once it would only appear once in the  sorted output.  Another reason for robust testing!

Our last version (for now) tackles the duplicate issue and is also a bit more efficient. We go back to using partition to generate our two lists, but now we post-process the second list.  We call partition again on the second list (which contains items not-less-than the pivot) into a list of matching items and not matching items.

Summary: This article is not meant to create the best possible Scala implementation of  Quick Sort, but to give you a vehicle for playing with Scala list manipulation functions.

Scala version of Log4JFugue, plus revamped website

Over the recent holidays I decided to create a Scala version of my Log4JFugue open source music project. I was able to reduce the 2500 lines of Java to just over 250 lines of Scala!

Checkout the new www.log4jfugue.org as well as:
Scala sources

I'll be giving a talk on the experience of converting a small Java project to Scala at the March meeting of New England Java User's Group.

new article in December PragProg magazine

I just had an article published in the December Pragmatic Programmer magazine on
a new approach to logging

Writing on a Tablet

I do a lot of writing these days, some on my high end Windows laptop with dual 20 inch monitors and some on my 10 inch tablet with Bluetooth keyboard.

I have surprisingly discovered that I am far more efficient on the tablet. This is surprising because on the tablet I have much smaller screen (less than 100 square inches versus more than 700 square inches), I don’t have a mouse, and switching from my writing app to email or the web is more involved.

That last item proves to be the key item. Because the tablet is primarily not a multitasking device I tend to focus on writing when I’m writing. The lure of the emails lurking in my system tray is gone. The other browser windows in my giant multimonitor display do not call to me. The fact that my tablet can really only do one thing at time is not a limitation it is a benefit.

While at a certain level this is obvious it’s also interesting given the current debates about multitasking operating systems. At the same time that some OSs are claiming to be better because of multitasking some of the mainstream OSs are adding features to discourage multitasking! For example various flavors of Linux have the option to automatically dim all windows other than the active one. The idea is to reduce the lure of context switching.

All kinds of current research is showing that humans cannot really multitask, yet we want to pretend that we can. At the same time, we know that the key to writing is to…well…write. And as much as we love to write we also love anything that can distract us from writing. So using a “limited” system to do our writing on is really to our advantage.

A few minutes with the Kindle Fire

I got to spend a few minutes with the Kindle Fire yesterday. Full disclosure: we have two kindles, an iPad, a smartphone and assorted laptops at home.

The iPad has almost completely replaced the laptop at home, and with a bluetooth keyboard I do most of my writing on it. It comes with me to the numerous evening Meetups I attend.

My kindle comes with me everywhere during the day, because it fits in my pocket which lets me squeeze in lots of spare minutes reading.

The smartphone has been relegated to making phone calls and checking gmail when out of wifi.

The KF is agreeably small and light and I could imagine it being with me everywhere. The screen is clean and crisp. Some reviewers have complained about slow screen response but I did not encounter that. Page turns while reading were very responsive and selecting items from the various menus was easy.

The touch screen made selecting and highlighting text much simpler than on the legacy kindle, so I might pick the KF for technical reading where I do lots of note taking.

One problem I did encounter was in getting to the menu screen. Most apps are full screen so to get to the menu and the Home button you have to touch the bottom of the screen. The location to touch was hard to find. It often took me lots of touches to bring up the menu resulting in lots of unwanted page turns. Presumably this would be something one would figure out over time.

Movies look nice on the KF, though obviously a smaller image than on the iPad. My kindle reading is largely opportunistic, a few minutes here and there. While I'm willing to read books a page at time I don't know that I'd want to watch a movie that way. When I know I might have blocks of time I bring my iPad. So movies wouldn't be a KF draw for me though they might be for my iPad-less wife.

Magazines look much better on the KF than on the legacy kindle.

In the end buying a KF or not depends on what devices you already have and what you want to do.
If you just have a legacy kindle or a smartphone and don't want to do content creation the KF makes sense. If you just have a iPad but no book reader you might get the KF for its extra portability.

For my wife who has a kindle plus dumb phone and wants some internet access we have a couple of choices. We could get a $300 smart phone plus two years of a $30/month add-on data plan; two year cost $1020. Or a $200 KF, possibly adding on the $79/year Amazon Prime membership. That seems like an easy choice for us. Your mileage may vary.

New article published at Pragmatic Programmers

I just had an article published in the Pragmatic Programmer magazine on
the long history of root cause analysis