Tuesday, March 26, 2013

Is It Coding in Scala or Coding With Courage And Humility That Matters?


Scala is all the rage these days, replacing its legacy cousin Java in the hearts and minds of all the cool kids.  Admitting that you still code in Java is the geek’s equivalent to saying you drive a mini-van and don’t have a twitter handle.  We’ve all heard the proclamations of how much more powerful and succinct Scala is..and I’ll admit to having done some of that myself.

While not dismissing those statements,  because I do in fact find Scala to be a better language,  I think there is another cluster of factors at work here as well.  Those factors are courage and humility.

There is a correlation between Java and large, often distributed teams.   This may be largely in part due to the relative age of the language.  It is established and “mature” to use a kind word.  If you have a 50-100 person development shop with offices in the US as well as in some subset of China, India,  Russia you are very likely to be a Java shop.  If you have maintenance or sustaining teams as well as a development team you are likely to be a Java shop.  If you spend a fair bit of your time on “process” you are likely to be a Java shop.

Notice that I said “likely”.  There are lots of counter examples.  My own company uses Java and we have only three programmers.  None the less, I think the preceeding statements are generally true.

So what?  Well, my assertion is that along with large, process oriented and distributed comes the notion of lowest common denomenator coding.  As the size and geographic distribution of your team grows so does concern about “those other programmers” being able to understand and maintain your code.  That leads us to want to standardize and simplify the code.  We want to make the code clear to that possibly junior coder who may be new to the project, who may never have absorbed the designs, and who many not be familiar with the code base.

This leads us to write code like:

public int calculateTheValue(long someInput, long someOtherInput) {
    long intermediateOne = someInput * getSomethingElse();
   intermediateOne += someOtherInput;
   intermediateOne = someMethod();
     ….
   return intermediateOne;
}

There is nothing wrong with this code and to a newbie its certainly more accessible than:

public int calculateTheValue(long someInput, long someOtherInput) {
     return somethingElse(someInput, someMethod(someOtherInput));
}

The problem is that 10 lines of code versus 3 for every method in your system results in a sea of code where you literally can’t see the forest for the trees.  I can tell what each individual line of code does but I have no idea why because I can never see more than 0.01% of the code on my screen at a time.
Some readers might protest at this point that this is all just formatting and Eclipse or Emacs could in principal convert between these two representations.  To which I say: not so much.

The functional approach is all about the composition of a method from a collection of existing functions.  In this approach it is clear that the new method is “just” using the existing methods.  The new method has no logic per se other than using the output of other…presumably well named and tested functions.

In the more familiar Java approach each method is a new creation created out of whole cloth.  It might do any old thing it wants.   In this case freedom and creativity are to be considered bad things.  Each method must be examined line by line to see what it might be doing.  Lets look at a bit of open source code I’m actually currently debugging:

int to = readTimeout - clientCnxnSocket.getIdleRecv();
int timeToNextPing = (readTimeout / 2)  - clientCnxnSocket.getIdleSend();                       
if (timeToNextPing <= 0) {
            sendPing();
            clientCnxnSocket.updateLastSend();
 } else {
           if (timeToNextPing < to)
                     to = timeToNextPing;
}
clientCnxnSocket .doTransport(to);

After some period of study we can see that this code has two variables related to time outs: “to” and “readTimeOut”.  Based on the results of two “getIdle” calls we might send a ping, and then we mutate “to” in a couple of possible ways and then use it as a parameter to a socket call.  Further investigation reveals that “to” is the length of time the socket method may spend in a blocking “select” call.  Thus, “to” is related to how long we can block before sending another ping.

I’ve spent the last couple of hours trying to track down a bug in this system and the problem is that every single line is ontologically at the same level.  By that I mean that any of them could have or be a side effect, any could do something other than whats expected and the gestalt of what this code fragment intends can only be gleaned by close study.

I’ll assert that the following code does not suffer from those flaws:

if(timeToPing()) sendPing();
clientCnxnSocket(safeTimeToWaitForRead());

And that brings us to the humility side of the equation.  Its ok to write little one line functions that just do the one thing that their name implies.  TimeToPing is not a function you will put on your resume.  You will not proudly show it to your coworkers.   You will not tell your husband/wife/partner about the amazing bit of code you wrote today.  This one line function will sit there quietly, unnoticed…working.
If we have the humility to write simple functions and then have the courage to combine them into composite functions without extraneous scafolding and temporary mutable variables then we have a chance to achieve greatness…even in a legacy language.

To be sure, there are things that are trivial in Scala that simply can not be done in Java.  I know of no way to annotate a method to indicate that it does or doesn’t ever return null.   We recently changed such a method to never return null.   There is no way however to  find all the code that’s now unncessary.  Or assuming we had made the opposite change…to find the code that was not an NPE timebomb. [1]   

In Scala of course if your function might return a Foo but might return nothing you return an Option(Foo)…and function’s callers must deal with the Option(Foo) or they will not compile.  This isn’t fixed in Java 7, nor will it be fixed inJava’s 8, 9 or 10.  Null is just baked into the language.

Java isn’t going to be “replaced” by any of the newer languages.  It will continue to lose market share but will command a large segment of the market for the foreseeable future.  Its also clear that Java will continue to evolve and will over time gain missing features such as lamda expressions and better package structure.  Other things like null and the Generics system are likely to be with us to the bitter end.    For good or bad erasures and generics are part of the language now and forever.  Java 8 sprinkles a bit of syntactic sugar allowing the second repeat of the type to be omitted as in:
HashMap myHashMap = new HashMap<>();
but that’s a fairly trivial improvement in this age of modern type inference languages.

What this means is that engineers working with Java need to do the best they can with a 15 year old language.  Courage and Humility can help with that. 



[1] Yes, we could use PMD but that just points out that there is no support for such things in the language itself.

Wednesday, January 23, 2013

Nothing but better hiring will fix things

Recently in Slashdot there have been a series of posts about the quality level of the software at various companies.  They've had titles like "How can I make my team write better code?"  or "How can we improve our code quality?"  

I also subscribe to numerous blogs and MeetUps and User Groups where titles like "Do 'x' to improve" are fairly standard.

I might just be getting my jaded but I'm coming to the conclusion that there is one and only one thing that can improve the quality of the software you/your team produces.  That one thing is better hiring.  To quote Joel On Software..the only two answers after an interview are "Hell yes!" or "No".

By and large people either care passionately about the code the write ... or they don't.  In over thirty years of doing this I can't recall a case where a "bad" coder read a book, went to a class, attended a conference or had a talk with management and suddenly started caring.

That's a bit depressing because I speak at conference and have published lots of papers!  On the other hand, my papers and talks have been aimed at the subset of our field that already care and are just looking to hone their skills.  Trying to convince someone that unit testing, proper naming, or coherent design was a good thing is just a waste of time.  People either "got it" a long time ago or they're not going to get it.  Sorry.

So, work to get your team to hire better people or go someplace that already does.  I just don't see another option.

Wednesday, December 5, 2012

New Article in Pragmatic Programmer

Just a quick note to say that I had another article published at the Pragmatic Programmer's:
http://pragprog.com/magazines/2012-12/agile-in-the-small

Wednesday, October 24, 2012

Picking the next toys just got harder

My VerizonWireless contract is up in three weeks, my Kindle just died and Apple just introduced the iPad Mini so I've got some deciding to do!

I current travel with my 13" MacBook Air, a 2 year old Android phone and often my iPad2 as well.  The MacBook Air is my primary laptop and serves as my coding and general content creation platform.  The Android phone is going into the crusher as soon as possible, to be replaced with an iPhone.  Apple isn't perfect but they support their devices for longer than 20 minutes so it "never again" for Android.

So now we've dealt with phone calls, navigation, quick email checks, quick web browsing, coding and content creation.  What's left?

The iPad has been getting pretty lonely since I got the Air.  They're both big enough to need a carry bag so if I have the iPad I almost certainly have the Air as well.  This leaves time on the train commuting to work as about the only time I'd use the iPad because there's really not room in our crowded train for the Air.  So I use the iPad for games and some reading.

Do I really need either a Kindle or an iPad Mini?  (And when I say "need" I mean it in the geeky sense as in can I develop a rational for it).

The only thing I can think of using an iPad Mini for rather than my existing iPad is reading; lets face it the iPad gets heavy after a while.  But I could get a Kindle Paper White for about a third the cost or a base Kindle for less than a quarter of the cost of the Mini.  I'd rather get the Paper White but they're back ordered for 4-6 weeks so that's not an option.

Put another way: I can get an iPhone and a base Kindle for less than the cost of the Mini.

So, for folks who don't already own an iPad the iPad Mini might be a great idea but I don't see many people getting both.

Saturday, July 28, 2012

Finding your voice

In the past six weeks I've attended a week long Writer's Conference at Wesleyan University, a week long Native American and World Flute Conference at University of Wisconsin and changed jobs (going from a 40,000 person company to a 6 person startup).

I like to look for common threads and the thread here is finding one's voice.

Voice is sometimes thought of as the same thing as "style" which may help explain the idea of programmers having a voice but I think it goes far beyond that.

At the writer's conference we did close readings ("inspections") of various works, looking for themes and patterns and voice.  Voice here meant a way of saying things that was engaging, illuminating, understandable and among other things predictable.  We talked about various contracts a writer can implicitly establish with a reader such as mentioning a shotgun in a foreshadowing paragraph implies a later use of that gun in some future scene.

At the flute festival we attended Master Classes on topics such as use of various scales (major, minor, mixolydian), how to add embellishments or ornaments to ones playing and how to tell a story by use of tempo and volume and such.  While the standard orchestral flute has about 20 keys the various native flutes have six or four keys...and the didgeridoo has no keys.  This can be seen as limited but a smaller set of notes to choose from just means one has to use other ways to produce a full and interesting story. Thus voice.

In my new job I am the second programmer on a medium sized system...meaning that the system was entirely written by a single person.  It definitely has style because the person who wrote it has strong feelings about the myriad of design choices he's made along the way.  He's also free about the choices that were made without a strong sense of one choice being clearly better than another.

His voice is of course different from mine.  We have very different backgrounds and life experiences, but the point is that I can see his voice in the code and I quite like it.  I'm sure that over time our two voices will influence each other, but I suspect that they will remain distinct.

In the past I've been a strong advocate of static analysis tools such as CheckStyle. I don't think I'll be lobbying so much for that tool here.  I'm beginning to think that such tools are most applicable for shops where the engineers have not yet found their voice.

Friday, June 22, 2012

Thoughts about e-Books after attending a Writer's Conference

I just returned from attending the 2012 Wesleyan Writer's Conference in Middletown, Conn.  It was a week jam packed with close reading of poetry, thematic explorations of fiction, exploration of dialog in non-fiction and of course speculation on the future of books, e-books and publishing.

A couple of ideas came to me during the week that I'd like to share.  Let me preface by saying that for me reading means reading books or PDFs on my Kindle or via a Kindle Reading app on my my iPad, MacBook Air or HP laptop.  Last year I read 42 books and only two of them were of the dead tree variety.

One of the great features of Kindle reading (and presumably also for Nook reading though I can't say from experience) is the ability to trivial get the dictionary definition of works in the text via a single click.  I like to pride myself on an extensive vocabulary but there are lots of words that I "mostly know what they mean but could not give you a coherent definition".  I make it a point to click on these words and get a real definition.  That's all well and good but we should be able to do better.

Many books with extensive sets of characters contain a Dramatis personae or list of characters at the beginning of the book.  This can be especially important when reading Science Fiction where some authors show how alien their characters are by giving them unpronounceable names.  Imagine being able to click on a name in your book and get the quick description of who that person is?


A further enhancement to this idea would be to generate what we know about the person at that point in the story.


Some people might view these not as enhancements but as crutches that remove the need to immerse oneself in the book.  I would respectively disagree and say they provide a way to become more immersed in the book, but it doesn't matter.  They are mostly just examples of simple things we can do to play with the book reading experience.


Sadly, while Amazon provides an API to create applications like games on the Kindle they do not (as far as I've been able to tell), provide a way to modify/extend the actual book reading experience.  I don't blame them as most "enhancements" would like be misguided but its still a shame that we can't currently play with the idea.

Thursday, May 3, 2012

Scala Static Analysis

I am finally getting around to working on my next pet project which is a Static Analysis tool for Scala.

Java has a number of quite good Static Analysis tools such as:
  checkstyle - enforces a large set of user configurable rules about things like
                       max-line-length, max-parameters, max-if-nesting,
                       max Cyclomatic complexity, etc
  findbugs    - looks for common bug patterns

Scala, for all of its power currently lacks such tools.  There is even debate within the community if such tools matter for functional languages.  Still, it seems like a very interesting problem to me so I decided to give it a go.

In talking with people at Boston Scala Days 2012 the consensus was that a compiler plugin was the best way to approach this problem.  So, I started thinking about a two pronged approach:

  1 - develop a compiler plugin to get access to the Abstract Syntax Tree of a program

 2 - start a discussion about what kind of rules or metrics might make sense for a language like Scala.

The plug-in itself splits into at least two parts: the skeleton that hooks itself into the compilation process and the part that accesses the AST itself.

There is a great "how to" article on the compiler plug-in skeleton at http://www.scala-lang.org/node/140  This article shows how to build a trivial plug-in that looks for divide by zero errors.  The guts are in the "apply" method...and that's where things got spooky for me!

      def apply(unit: CompilationUnit) {
        for ( tree @ Apply(Select(rcvr, nme.DIV), List(Literal(Constant(0)))) <- unit.body;
             if rcvr.tpe <:< definitions.IntClass.tpe)
          {
            unit.error(tree.pos, "definitely division by zero")
          }
      }
I know that this method iterates over the unit.body tree looking for items that are division, with a literal zero and an integer.  That doesn't mean however that I fully grok the Tree hierarchy and its various flavors of Apply and Select methods!

I will say I'm getting some very good help on this via a question I asked on StackOverflow:
http://stackoverflow.com/questions/10419101/how-can-i-find-the-statements-in-a-scala-program-from-within-a-compiler-plugin

More as I get smarter about Abstract Syntax Trees!  If anyone wants to help on this project please contact me as I clearly could use help.  I'll put the project up on GitHub soon.