Moving forward

One of my favorite lines from M.A.S.H was Frank Bruns insisting that "we must have a plan even if it is wrong." While this was a line intended to ridicule Frank, it has been my experience that creating a plan, even if it turns out wrong, is a positive step forward.

A miniature wargamer begins

I am enamored with all things miniature wargame. Last year I dove into the world of boardgames (and card games) as my sons were now of an age where we could plays games that were enjoyable to each of us. Boardgaming is not, however, a hobby. A hobby requires hands-on creativity. (Aka, dirty hands.) A hobby is what you do between events. While I am up to the challenge of developing boardgames and card games it is too much of an intellectual challenge only and I have enough of that in my day-job. And further, I don't really want to do it.

As a young teenager I had a detailed model Great Western Railway in N gauge. I loved this hobby. It allowed for the creation of a whole world and one run with British timeliness. (Timeliness, is something that has vanished from the modern UK rail system, I am told.) Modeling railways and railroads are still something I look forward to doing again in my dotage, but not now.

What sparked my interest in miniature wargaming come from an unexpected encounter at the Hobby Bunker. This game store is near to my employer's office and so I went to it too look at their stock of boardgames. What I encountered was not, principally, a boardgame store but instead a miniatures wargaming store. As, as it turns out, the Hobby Bunker is one of the best such stores in the US. And so for two hours I wondered the aisles of shelves stocked with unpainted miniatures in plastic and metal of all historical periods; the selection of paints in acrylic and enamel; the shelves of books with the historical accounts of nations, armies, commanders, and battles; wargamer magazines; and wargame rules. (Hobby Bunker does also have a good collection of boardgames.)

That day I only bought an issue of Wargames Illustrated but the next time I bought some plastic saxons and the paint for them. And, like every good wargamer, they remain unpainted in the box still. But that is not because of waning interest but instead I have dove in the deep-end of the pool and am currently surrounded with magazines, military history books, wargaming books (mostly the foundational one), rule books, wargaming podcasts, and, soon to arrive, armies of vikings and saxons in 6mm.

As one of my sons noted, "you are serious about this dad!" I am.

HAVOC

I had a great time Saturday attending the Boston Battle Groups's Havoc Game Convention (HAVOC). My goal in attending was to see different historical periods and different rule sets in play. In the morning I watched Day of Battle rules payed out by Christopher Parker (the rule set's author), his brother, Jen, and Richard Brian. The game is driven by a unique combination of dice (d6) and a standard deck of cards. I need to see the rules played out in a tournament to make a judgement about them. I want to thank Chris and Dick for sharing their time and knowledge with me.

In the afternoon I watched a Punic wars battle played with the the De Bellis Antiquitatis (DBA) rules ((say what). This rule set seems most approachable. It had the feel of a boardgame but played with miniatures on an open table. Since returning from HAVOC I have spent the most time with this rule set. I want to thank Maureen, Spencer, Mike, and Harrison for a wonderful afternoon.

All in all, I met a wonderful group of people at HAVOC. All most willing to ofter advice and help. And Vic Gregoire invited me to join his "The Usual Gang of Idiots" for some wargames later in the year.

Why is Spring taking a perfectly good Set value and passing along a null?

Why is Spring taking a perfectly good Set value and passing along a null to the property's setter? That is, I had a simple bean declared like the following

<bean name="set" class="java.util.HashSet">
  ...
</bean>

<bean name="x" class="...">
   <property name="y">
      <map>
         <entry key="z" value-ref="set"/>
         ...
      </map>
   </property>
</bean>

Where the property's setter is declared

void setY( Map<String,Object> y )

But every time setY() was called the key z's value was null. The problem's answer was very hard to track down. It comes from how Spring automatically converts objects to type specific property values. Spring has two ways of converting a value for use in a property's setter method. The first is based on converting a textual representation to an object via the java.beans.PropertyEditorSupport API. The second is based on Spring's own org.springframework.core.convert.converter.Converter API. I has used a Converter in an unrelated part of my application's Spring context. This was my application's first use of the Converter API. Turning the Converter API on, however, brought with it other conversions I was not expecting. The most detrimental was the Collection to Object converter. When Spring sees that the property expects an Object it tries to convert any value to an Object. (Even though every value is an object.) The Collection to Object converter converts an empty collection to null. Let me say that again .. with emphasis. The Collection to Object converter converts an empty collection to null. My set was empty. And so, a perfectly good empty set value was converted to a null. The correction is the change the setter for property y to

void setY( Map<String,?> y )

And all works now. p.s. I would never have been able to debug this without Spring having open source. Thanks VWware, Spring Source, and all the contributors.

Need multiple-view calendar

The calendar displays that we have had forever just don't help me anymore. Too much information and/or too much interaction need. I want my calendars to show me what I want without interaction. I want to distinguish what is common and what is unique. Common, for example, are events such as every workday at 9 AM I have a standup meeting; Thursdays I deploy new releases; Every second Monday of the month during a school year I have a PTA meeting. Unique ones are the dentist appointment and the odd professional event at night. I don't really want one-view calendar anymore. I need multiple-view calendar like he one illustrated here.

The top chart contains my unique events over the last 2 and next 8 days. The second chart contains my common events over the last 2 and next 8 days. The bottom left chart is a list of common events for the next several weeks. And the bottom right chart is a list of unique events for the next several weeks.

Since we have separated events that actually occupy the same timeline we need to rejoin them somehow. My initial thought was to add a ghost to each chart that signifies the presents of an event in another chart, but I am not wedded to this.

Does anyone make a calendar like this, either visually or purposefully?

A idea for fixing the flow of money in pharma. deseses

Last year I attended a few lectures and read news reports about the pharmaceutical business where drug researchers and clinicians spoke about how the allocation of money was disproportionally being spent on life-style drugs and existing (often unique) drugs were being discontinued. This got me thinking about how we could continue to allow pharmaceutical companies to make gobs of money (I don't object to profiting from a great product) and also allow clinicians to depend on a consistent supply of drugs. We need a segmenting and deregulation like what was done with electric utilities in the mid-1990s.

The electric utilities in the mid-1990's were split into three segments: generation, transmission, and local distribution. While the customer does not have a choice of local distribution and transmission they should have a choice of generation. Local generation and transmission continued to be regulated. Generation was deregulated to enable competition. This change was fundamental in the motivation to build wind farms, solar farms, etc.

The pharmaceutical companies need to be split into three segments: research and development (R&D), manufacturing, and distribution. No one company can be in more than one segment. Manufacturing and distribution are typically low-risk, low-margin businesses. They are also critical to continuing care and so will be regulated. The R&D segment is typically a high-risk, high-margin business. This will be unregulated (well, more like less regulated). Within R&D there are two means of funding. The first is the self-funding pharmaceutical industry we have today. The second kind of R&D companies are the startups financed by venture capital.

Given these segments how does the money flow to encourage innovation and discourage abandonment of established, low-margin markets. Firstly, by separating manufacturing and distribution these businesses can continue to exist and be profitable as they do not need to directly bare the costs of R&D. People, world over, continue to get sick with the same diseases!

How is R&D paid for? R&D works within a venture capitol environment. Money comes from outside the pharmaceutical industry and money comes from inside the pharmaceutical industry. The money from within the industry comes from the sale of drugs to manufactures and a royalty on retail sales. This money does not go directly to the R&D businesses. This money goes into a venture capital fund. This is one of the many funds that R&D businesses can pitch a therapy to. I do not know how the internal fund is managed. This might be a kind of utility with public oversight.

This segmentation and organization allows for the continuation of the high-risk and high-reward therapy R&D but takes out of R&D's control the manufacturing and distribution. A win for all interested parties -- and we are all interested parties in health.

Lawrence Lessig: How Money Corrupts Congress and a Plan to Stop It - The Long Now

I have mentioned before Lawrence Lessig's work to remove the inequitable distribution of power through money and the dependency corruption that is pervasive in US politics, esp. at the federal level. His book Republic, Lost is a challenging read. He recently gave a talk at the Long Now Foundation that I highly recommend. Please do listen to it. Well worth an hour of your time.

Lawrence Lessig: How Money Corrupts Congress and a Plan to Stop It - The Long Now

Rants (aka Tweets) on resumes

Rants (aka Tweets) on resumes:

A resume should not be a ouija board. Your mother might not need to understand it but I do. Make your work clear.

Your resume should tell me about the teams you were on; team size; team duration; role(s) you played; tools you used. Everything else is BS.

If your resume can't hook me on the first page the following pages are doing nothing more than annoying me.

I hope to god that I never see a 3 page resume again.

Apple had just one customer

“Apple had just one customer. He passed away last year.” - Seth Godin

The realpath command line utility

I seem to use my command line utility realpath all the time. All it does it take a relative path and return its absolute path. It is very useful. So useful that I can't believe all systems don't have one in /usr/bin/. If you are missing this functionality too then here is source

#include <stdio.h>
#include <limits.h>
#include <stdlib.h>

int main( int argc, char** argv ) {

        int i;
        char _realpath[ PATH_MAX ];

        for ( i = 1; i < argc; i++ ) {
                if ( realpath( argv[i], _realpath ) != NULL ) {
                        printf( "%s\n", _realpath );
                }
                else {
                        exit( 1 );
                }
        }
        exit( 0 );
}

Save this to the file realpath.c and then build with make realpath.

Cloning a Subversion repository

If you are following the explanation in Version Control with Subversion, "Repository Replication", page 177, of how to clone a Subversion repository and want a working example then here it is as a Makefile

# NOTE ROOT must be an absolute path
ROOT=/tmp
CLONE=$(ROOT)/clone
SANDBOX=$(ROOT)/sandbox
SOURCE=http://css-boilerplate.googlecode.com/svn/

all: 
    svnadmin create $(CLONE)
    ln -s /usr/bin/true $(CLONE)/hooks/pre-revprop-change
    ln -s /usr/bin/true $(CLONE)/hooks/start-commit
    svnsync init file://$(CLONE) $(SOURCE)
    svn co file://$(CLONE) $(SANDBOX)
    svnsync sync file://$(CLONE)
    ( cd $(SANDBOX) ; svn update ; ls -l )

clean:
    rm -rf $(CLONE) $(SANDBOX)

# END

Replace SOURCE with a value appropriate to you. (Here I am using a convenient, small, public repository.)

A comment on "So which IDE is the best and why?"

My comment to So which IDE is the best and why? Im a beginner and only know of a few I prefer Eclipse : java:

An IDE gives you three very useful tools. Code editing. Code navigation. And code debugging. Each of the commenters, I am sure, have different needs and styles of these tools. You will also.

All the IDEs tend to have very aggressive code editing help. This is great when you are working with new packages but the help gets tiresome as your packages familiarity increases and yet the IDE continues to make suggestions when you know full well what you want. Code navigation in modern IDEs is a godsend. To be able to move easily between classes, up and down the inheritance hierarchy, and around usages. Code debugging is less about the source and more about the data and the threads. The better the tools are at showing data structures -- the plural is important! -- and visually connecting threads w/ stack traces the easier your job becomes.

I have not answered your question because there is no answer. We shape our tools and our tools shape us. It almost does not matter which you pick -- Eclipse, IntelliJ, NetBeans, Emacs w/ JDEE, Vim w/ JDE, .... What matters is that using the IDE becomes second nature to you.

And don't fool yourself that you will fix the IDE's problems. You don't have that much free time.

Code now on GitHub

I have started to move and/or clone my code to GitHub -- like everyone else....

Calliope Sounds has an ISSN

I work for a small publisher intermediary and while working with our data recently I needed a testing ISSN. It got me thinking about the work involved with getting an ISSN for my blog -- an online serial publication. There is one form to complete and that is then mailed to the Library of Congress along with the "front page" of the publication. I did this about two weeks ago and today the Library of Congress, United States, ISSN Center delivered my new ISSN. In all its glory, here it is

ISSN 2165-0861

Using reflection for command line option parsing

Over the last few days I have needed a few command line tools -- mostly for data cleanup. I got tired of manually parsing command line arguments. I wanted something more automated. My first thought was to design a data-structure that expressed the command lines arguments then write an interpreter that would parse the arguments with its guidance. I spent an hour doing this only to realize that preparing the data-structure was almost as much work as manual parsing. Then I remembered that Ant has a simple facility that uses reflection to execute a task expressed in XML against an object. This is what I wanted. So, for example, the command line tool

java ... Sum --multiplier 25 --verbose 1 2 3 4

would be implemented as

public class Sum implements Runnable {

   public void setVerbose() {
      this.verbose = true;
   }

   public void setMultiplier( Long multiplier ) {
      this.multiplier = multiplier;
   }

   public void addPositional( Long number ) {
      this.numbers.add(number);
   }

   public void run() {
      long sum = 0;
      for ( Number number : this.numbers } {
         if ( verbose ) {
            System.out.printf("%d + %d= %d\n", 
               sum, 
               number.longValue(), 
               sum + number.longValue() );
         }
         sum += number.longValue();
      }
      if ( this.multiplier != null ) {
         if ( verbose ) { ... }
         sum *= multiplier;
      }
      System.out.println(sum);
  }

  ...

}

The magic, of course, is reflection. The reflection-based parser sees "--verbose" and matches it against setVerbose(), "--multiplier" matches against setMultiplier() and that it takes one numeric option. The remaining positional arguments are passed to addPositional() as numbers. Once parsed run() is called.

The ReflectiveCommandLineParser is used to handle the magic within the main()

   public static void main( String[] args ) {
      Sum sum = new Sum();
      ReflectiveCommandLineParser.run(sum,args);
   }

See ListFiles implementation for another example.

How MySql loads result sets

I was having a devil of a time yesterday with the simple task of using a Java program to copy records as objects from one MySql database to another [1]. I kept running out of memory. While the root problem had to do with creating 2M objects in memory it did lead to a better understanding of how MySql loads result sets. In short, it loads the whole result set into memory in one shot. So, if you have 1M records at 1K each in the result set you will need at least 1G of memory to hold them. If you then build 1M objects from these records you will need an additional 1M * object size of memory. In other words, a lot of memory. You can have the MySql JDBC driver "stream" the result set, however. That is, read the records row by row from the database. It is less efficient for the driver -- multiple trips back and forth between the server -- but doing so requires far less memory. You can turn on streaming at the statement level or at the datasource level.

Statement Level

To turn on streaming at the statement level you need to use a set of common JDBC settings that, when used together, inform the driver to stream. When you create or prepare a statement you must define how the result will be used and what is the fetch size. For example,

Statement statement = connection.createStatement(
 ResultSet.TYPE_FORWARD_ONLY, 
 ResultSet.CONCUR_READ_ONLY);
statement.setFetchSize(Integer.MIN_VALUE);

and

PreparedStatement statement = connection.prepareStatement(
 "select ... from ... where ...",
 ResultSet.TYPE_FORWARD_ONLY,
 ResultSet.CONCUR_UPDATABLE);
statement.setFetchSize(Integer.MIN_VALUE);

For more information see section Result Set in JDBC API Implementation Notes. DataSource Level
To turn on streaming at the statement level you need to add a property to the JDBC uri. For example, Integer. MIN_VALUE is -2^31 and so use

jdbc:mysql://localhost/?defaultFetchSize=-2147483648

For more information see Driver/Datasource Class Names, URL Syntax and Configuration Properties for Connector/J. [1] I could not use the MySql tools for dumping and loading the table data because I used an auto_increment column in one of the related tables, the target database was active with data, and so could not reset the target's auto_increment column to an appropriate value.

Response to "EasyMX - An alternative to JMX"

A response to EasyMX - An alternative to JMX | Javalobby:

An advantages to using JMX is that the JMX client gets to the service via a different network path than the service's users. When the service is running well the path taken does not matter much. It is often the case, however, that the main service path becomes inaccessible under adverse conditions. Your HTTP requests are not being serviced before timeouts kick in, for example. And, consequently, your monitoring is also inaccessible. JMX clients use RMI or direct socket pathways to connect to the service and so the JMX client can continue to monitor and manage the service.

As Mr Fisher says (first comment to the posting), JMX is one of the "golden parts" of the Java ecosystem. (JBose was built on top of it.) Current JMX coding practices are more sophisticated than in the early days. The "MBean" and "MXBean" interface suffix continue to support quick and dirty monitoring and publishing. And for those with lots of monitoring and management touchpoints we too use sophisticated Java annotations processing to turn existing code into touchpoints.

Revised log levels proposal

@jbarnette: Revised log levels proposal: "fyi," "wtf," and "omg."

In praise of gnuplot's dumb terminal support

I have to say, again, I find gnuplot's dumb terminal support is so useful when you are at the command line and need to see a plot of some data. The plot is very rough but this is usually enough to give you enough insight into the data as to whether or not to continue to exploring it. The script I am using now is

#!/bin/bash

function show_usage {
 echo \
  usage: $(basename $0) \
  [-x label] \
  [-y label] \
  [-t title] \
  [-s width:height] \
  [-f time-format-strftime] \
  timeseries-input ...
}

function swap {
 echo $2 $1
}

function parse_size {
 W=$(expr $1 : "\\([0-9]*\\):[0-9]*")
 H=$(expr $1 : "[0-9]*:\\([0-9]*\\)")
 echo "$W $H"
}

TIMEFMT="%Y-%m-%dT%H:%M:%S"
XLABEL="Time"
YLABEL="Units"
TITLE="Timeseries"
SIZE=$(swap $(stty size))

while getopts "x:y:t:f:s:h" opt
do
 case $opt in
  x) XLABEL=$OPTARG ;;
  y) YLABEL=$OPTARG ;;
  t) TITLE=$OPTARG ;;
  f) TIMEFMT=$OPTARG ;;
  s) SIZE=$(parse_size $OPTARG) ;;
  h) show_usage ; exit 0 ;;
  *) show_usage ; exit 1 ;;
 esac
done
shift $(expr $OPTIND - 1)

for INPUT in $*
do
 if [ "$INPUT" = "-" ]
 then
  INPUT=$(mktemp /tmp/timeplot.XXXXXXXXXX)
  cat > $INPUT
 fi

 gnuplot <<EOH
  set terminal dumb $SIZE
  set autoscale
  set xdata time
  set timefmt "$TIMEFMT"
  set xlabel "$XLABEL"
  set ylabel "$YLABEL"
  set title "$TITLE"
  plot "$INPUT" using 1:2 with lines
EOH
done

# END

For example, if the data in /tmp/data is

2011 34
2012 34
2013 56
2014 24

then this can be quickly plotted using

timeplot -f %Y /tmp/data

and get

Under glass they can have it all

I have been deeply troubled more recently by having been given a name to what I/we see happening. Our world's experiences are rapidly being shared "under glass" [1]. Be it phones, tablets, desktops, etc. We photograph our children and show the image under glass and not on paper. We hear a poet not in person but from under glass. The Call of Duty gets adrenaline rushing with only the risk of spilling your soda. Ever more of our worlds experiences are under glass. For kids under glass has become their first experience. For some it will be there only experience. You can look and listen but you can not touch. Experience as diorama. The outside framed. Technology advances and so we might just be at a low point in the development of experience+technology. I don't think this is the case. The soma of instant gratification to even the artificial is too strong a force. Umberto Eco talked about American's allure of hyper-reality. Under glass they can have it all.

[1] A Brief Rant on the Future of Interaction Design