Designing command-line interfaces

I like the posting Designing command-line interfaces. Being a CLI type developer I have implemented most CLIs with Anders' recommendations in my tools. Points I would like to emphases are

1) For long running commands make them verbose by default. For short running commands, like mv and cp, make them quiet by default.

2) While I like autoconf's use of "no" to turn off an option, eg --no-foos, and "=" to provide an option's optional value, eg --debug on default port or --debug=port for a specific port, it is not used enough elsewhere and so is too unexpected. Don't use them.

3) Always use a non-zero exit if you find an unknown option or an unknown positional parameter.

4) Use one-dash for one-letter options and two-dashes for multiple-letter options. Eg, -v and --version, -h and --help. There are many libraries available that help parse options. I tend to only use two-dash options and so hand code the parsing.

5) If you don't use one-dash options make sure to reject all positional parameters that start with a dash as this is mostly a user error. If you want to allow positional parameters that start with a dash then use the common "--" parameter to indicate that the remaining parameters are all positional parameters.

6) An option can have multiple arguments. For example, --database url user password.

7) Always have the options --help/-h, --version/-v, --quiet/-q, and --verbose.

8) If a command can run without options make sure the command's results are harmless. Nothing worse than incorrectly using a command and having it destroy data.

I will, perhaps, add other points for emphases at a later date. For now, do read Anders' posting.

FYI: Kanban for Software Development

The posting Kanban for Software Development is one of the clearest introductory explanations of kanban that I have read. For example,

This is where the somewhat brilliant key idea behind KSDM comes to play. The completed job does NOT free the person for working on another job, until that job is pulled into the following phase and work is started there. If work is piling up at a particular phase, those people are NOT ALLOWED to work ahead. As Taiichi Ohno makes so clear, that working ahead is waste. Instead of working ahead, they can look around to see what is wrong.

To put this in concrete terms, consider a process which involves (1) detailed design, (2) coding, (3) testing, and (4) documenting. Each of these stages you place a limit on the number of jobs, and for the sake of example lets say that limit is 4. Say for example that the coders have finished coding on their four job units, and are ready to take a new one. But the testing is backed up for some reason, still working on the last four job units, and are not ready to take a new job. The developers are not allowed to pull in a fifth job unit. There is no point in coding up more features when the earlier features are not getting tested or documented. It is also possible that because the developers are not pulling jobs from design, that the design phase becomes filled up with completed tasks.

When work backs up in this way, one should go and figure out why testing is stuck. Maybe the real problem is that the documentation is blocking test. Whatever it is, the primary job of the entire team is to identify the problem with the flow, and fix it. Do not simply work ahead accumulating a huge pile of work for “someone else”. Instead, focus on the big goal, which is to get features completed to a customer ready state as quickly as possible.

The Problem with Kanban

Comment on The Problem with Kanban | Javalobby:

The biggest problem with Kanban is that it‘s designed for a world where things go through the line once (e.g. a carmaker).

This got me thinking about whether the problem is generally with Kanban's use in software or instead the kinds of software development Kanban that is more useful in. A service firm builds a single use or single customer tool while a product firm builds a multiple use or multiple customer tool. A service firm tends to have more tasks around the missing parts of the software while a product firm tends to more tasks around the bugs in the software. For a service firm the factory is the assembling of the parts. For a product firm the factory is perfecting the assembly. I am mostly thinking aloud here but I think it is a useful train of thought as to why Kanban works for some firms better than for others.

Long Now seminars

The whole Long Now seminar series is very good. If you have iTunes and an iPod just add their pod cast to your iTunes library. Note that the lecture's audio is free but video requires membership. Here are specific lectures I keep coming back to

Mary Catherine Bateson
Live Longer, Think Longer

Jesse Schell
Visions of the Gamepocalypse

Geoffrey West

Why Cities Keep on Growing, Corporations Always Die, and Life Gets Faster

"What's with the moribund twitter bio?"

Eliel asked "What's with the moribund twitter bio?" I am thinking that it must have been a "can't get anything right" kind of a day when I wrote that. But in all honesty, I am not a natural father and so find myself consciously working at it everyday and the same for marriage (perhaps every other day here). And, how many times have I been granted but never exercised stock options? I am a very optimistic guy. While everyone else (ok, many) are grasping for success I am happy working at being a little more whole one day at a time.

Keeping power and maximizing shareholder value

In the 1970s our US government representatives stopped caring about their constituents and started caring only about keeping their own job and its access to power. The corporations didn't initiate this change. They are, however, proficient at using the change. Corporations are just doing what corporations always have done and that is to maximize shareholder value by aiding representatives to keep their jobs. I hate to deliver bad news but y'all are the shareholders -- via pensions and other retirement vehicles -- and so are benefiting.

If you want to help take away the power of money in politics then support (Lawrence Lessig’s) Fix Congress First.

Facebook trapped in MySQL ‘fate worse than death’ — Cloud Computing News

Facebook trapped in MySQL ‘fate worse than death’ — Cloud Computing News

Mr Harris: When you attribute statements like

"Stonebraker said the problem with MySQL and other SQL databases is that they consume too many resources for overhead tasks (e.g., maintaining ACID compliance and handling multithreading) and relatively few on actually finding and serving data."

include references to the hard proof. Pushing logic from the repository to the application does not make anything faster or slower. Cumulative work is just shuffled around. Moving logic from server to client can improve the perceived speed as you have effectively added massively to CPU and RAM capacity. So, if you have a relational data model then use SQL and a relational repository. If your data model is different then use a custom repository. The different NoSQL repositories fill this custom need. Just as, for example, FOCUS fitted the custom need for hierarchical data in the 80s! As to web scale issues, the solutions are all the same no matter how your repository models data.

I need to improve my writing

I need to improve my writing. I have much to say but find myself inarticulate at the keyboard or pad of paper. What I write unfolds from sentence to sentence. There is much backtracking to apply structure afterwards. Time to read some good, short journalism and advocacy pieces and digram them. I will start with Ellen Liberman's The Reporter column.

How to Rethrow an Exception Without Wrapping them - Java Wiki

Nicely said at How to Rethrow an Exception Without Wrapping them

Some do's and don't's with exceptions

1. Unless you have a very good reason to catch an exception, don't catch it.

2. If you can correct the problem implied by the exception. eat the exception, because you fixed it. However there are some exceptions that it is unwise to catch.

3. If you can provide additional information about the exception. catch the exception, and re-throw it as an inner exception with more information. This is a very good reason to catch an exception, but note that we are still re-throwing it.

4. Always try to catch specific exceptions. Avoid catching System.Exception whenever possible.

Comment regards to "An importance of wrapping variables"

The comment was made regards to the blog posting An importance of wrapping variables. A useful feature to add to any logging tool is to allow the formatting of the log message. For example,

File file = ...
...
logger.info( "unable to open file: file-name={0}", file );

Doing this has three key advantages. The first is that the ease of adding relevant data to the message is trivial. This then encourages messages that truly aid monitoring and debugging. The second is that there is a very low performance cost to logging in that if the message is never used (due to the log level) then the only cost to making the call is loading and unloading the stack.

The second advantage becomes significant for debugging messages. For example, the debug logger call

logger.debug("{0} moving towards {1}", a, b );

is a far less costly to ignore than is

logger.debug( a + " moving towards " + b );

And especially so in a tight loop. One can argue that you can always check the debug level before making a debug call. In practice, however, this tends to not to be done and instead the useful debug message is never written.

The third advantage is that it is trivial in the logger implementation to both quote and escape the message's parameters. My implementation passes them through a JSON converter so that I can also see the internals of the parameter's values.

SaaS + Java + Exchange ideas & tools

I currently work for small shop where we run a number of middleware services for publishers. We build services in the traditional style of Java servlets, Spring dependency injection, Tomcat deployments, ActiveMQ message bus, MySql for transient data, and Oracle or permanent data. The engineering of this style is well known to the team. But the consequence are that: Our deployments are manual; Startup times too long; Spring has driven us to a monolithic implementation. We are servicing our customer's needs. But we need to do better. How do we get better?

If you are a SaaS + Java shop in RI, MA, or eastern CT and would like to talk about tools and processes please contact me at andrew@andrewgilmartin.com.

Co-working in South Kingstown, RI

The summer is approaching and the kids will be home. Yet I need to get work done and I suspect you do too. I would like to create a co-working space here in South Kingstown. A co-working space is a space to work from without the distractions that come with home, family, and coffee shops. It has desks, chairs, wireless, lockers, air conditioning and great lighting. And, with luck, a great location. I am looking to keep the membership costs at no more than $100/month.

If you would like to discuss this further please contact me at andrew@andrewgilmartin.com and we can organize a meeting where the first half is informational and the second is planning.

Update: Everyone likes the idea but I was not able to get anyone to commit. When you work for "free" from home, at cafes and at libraries, paying a $100/month is lot less appetizing. It will happen one day. Just not for me this summer.

Apple's Java source and javadoc

Apple's Java installation does not include the source or javadoc api. Apple does have a developer download at X which does contain these files at <http://developer.apple.com/java/download/>. On my machine the package in "javadeveloper_10.6_10m3261.dmg" is installed at

/Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents/Home/

If you are using NetBeans you will need to add this as a new "Java Platform". This posting assumes you called the platform "JDK_1.6". Once you have created the platform you will need to manually fix how javadoc is used. To do this, in the file

~/.netbeans/6.9/config/Services/Platforms/org-netbeans-api-java-Platform/JDK_1.6.xml

edit the //javadoc/resource element so that "!/docs/api/" is appended. For example,

<javadoc>
  <resource>jar:file:/Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents/Home/docs.jar!/docs/api/</resource>
</javadoc>

For some reason, Netbeans emends the source resource with "!/src/" but not the javadoc resource.

Commutative data types and their operations is a powerful idea.

Somewhere this fortnight I read that I should read "CRDTs: Consistency without concurrency control" <http://hal.inria.fr/inria-00397981/en/>. The basic idea is to have a data structure where for the same set of operations applied in any order will lead to the same state. If I have a number of replicants of this data structure then as long as all the operations are applied to each replicant -- in any order -- then the replicants will have the same end state. With concurrent replicants then all replicants will eventually have the same state. Commutative data types and their operations is a powerful idea.

Downgrading my iPhone to iOS 3

I finally took the steps to downgrade my iPhone 3G from iOS 4 to iOS 3. The lose of responsiveness of basic features in iOS 4 is in no way made up by the new features, especially as few of them work with the 3G hardware. I followed the directions on the Lifehacker page at How to Downgrade Your iPhone 3G[S] from iOS 4 to iOS 3.1.3.

One-liner to find all occurrences of property names within a stream of text

One-liner to find all occurrences of property names within a stream of text:

cat * | perl -ne 'while ( /\G.*?([-a-z\.]+\.enabled)/gi ) { print "$1=x\n"; }' | sort -u > /tmp/enabled

This posting is mostly to remind me that 1) print "..." while /.../ does not work and 2) to use \G to anchor the regular expression to the first and/or last match.

Having a Java generic class return a type reference to its specialized class. Aka, using the self-type.

Emulating "self types" using Java Generics to simplify fluent API implementation is a very good example of how to more effectively use Java's generics and inheritance. The "self type" solution is something I would not have imagined. Further, it is motivating to see what other effects can be achieved through Java's generics. Read the article for a compete understanding of self types and a use of them. For those wishing to skip to the conclusion, if you need a generic class instance to return a typed reference to the specialized class then it is coded

public class GenericsTest {
    // We have a generic type here that needs to return from its check method
    // a reference to the specialized type (that is, the generic's sub-class).
    static class Common<SelfType extends Common<SelfType,ElementType>,ElementType> {

        protected SelfType self() {
            return (SelfType) this;
        }

        public SelfType check( ElementType e ) {
             // do something here that is general to all sub-classes
             return self();
        }

        // ...
     }

    static class Foo extends Common<Foo,Integer> {

        public Foo doFoo() {
            // do something here is is specific to Foos
            return this;
        }
    }

    static class Bar extends Common<Bar,String> {
        public Bar doBar() {
            // do something here is is specific to Bars
            return this;
        }
    }

    public static void main( String[] args ) throws Exception {
        Foo foo = new Foo();
        Bar bar = new Bar();
        foo.check(1).doFoo();
        bar.check("a").doBar();
    }
}

// END

Tools needed to support a simple content organization and workflow

I like to use the file system. And I like to use the file system's default explorer: On the Mac this is the Finder, on Windows this is the Windows Explorer, and on Gnome it is/was Nautilus. I much prefer to have other tools integrate into the file system explorer than for the tool to have its own explorer.

When this happens, it is then possible to create your own tools that support your workflows. (Note that in this posting I am concerned with explorer integration and not command line integration.)

Unfortunately, this has not been the norm and so we have iPhoto, iTunes, etc, having there own content organization that is independent of the file system. These content specific explorers are often created due to the weaknesses in extending the system's explorer but mostly it is because creating another explorer is just easier to both imagine and code. The up-shot is that we have organizational silos with little integration between them. And, worse, the inability to create your own specialized tools from the integrations.

This Sunday I spent far too much time tracking down tools that would allow me to use the file system to collect and organize content as follows:

1. Each content collection is a folder. I am comfortable with using the file system hierarchy for organizing and I am aware of the weaknesses of hierarchical organization. However, most collections can be organized into a primary hierarchy and augmented either by search or linking (aliases on the Mac and shortcuts on Windows).

2. Each folder will have, at least, one plain text file of dated notes. Each note is separated by a simple delimiter, for example, a row of dashes. My experience is that this one file of notes is much easier to review and keep updated than to place notes into individual files. With that said, a good tool that can hide the file-ness of the notes would be acceptable too. Until then, one file of notes and using Merlin Man's tip for adding content to it <http://bit.ly/a1kfV6> will do fine.

3. The primary hierarchy needs to be supplemented with full text search. Tagging is not enough and, moreover, tagging can be achieved via search by using specialized tokens: For example, in normal English typography a colon is always followed by a space and so the expression ":foo" distinguishes the tag "foo" from the rest of the text. Spotlight on the Mac is the system's full text searching facility. It is extendable so that other content types can be incorporated. The plugin I am looking for would index the dated notes in a file as collection of "records". (This Spotlight feature was introduced in OS X 10.6.) The Spotlight plugin is the most specific tool that I need. While it is a Mac only indexing solution it is based on plain text content and so could be indexed easily under other operating systems.  Unfortunately, as far as I can tell, and I spent a good amount of time looking, there is no plugin for indexing delimited content as records. There are plugins for content types that somewhat match, for example mailboxes, but there is no documentation on how to use them outside of their intended application. The up-shot that that for this very simple workflow I can not compose a supporting tool from existing parts. Looks like I am going to have to write my own Spotlight plugin: If anyone has skeleton code I can build upon please drop me a note.

Changing Education Paradigms by Sir Ken Robinson

Denis Dutton: A Darwinian theory of beauty