The Wonder Match

This short rememberance of Bobby Fisher in Sunday's New York Times Magazine (December 28, 2008) is a fantastic piece of writing. Well worth reading both for the story and the writer's skill.

Git and the horror of the extra colon

Never enter the command line git push origin :master when you mean git push origin master. Instead of coping your master branch's local changes to the remote repository you delete the remote repository's master branch. You can't undo this mistake. My initial reaction to the fact that I just deleted the company's source code was not a comfortable one. What I soon found out, however, was that you can just push again your master branch to return it to the remote repository. I am not sure this is the correct fix but it did seem to work.

Update: Here are my git notes at

Routers and manually interpreting a javascript enabled UI

Why do home router/gateways have web administration interfaces that REQUIRE javascript to work? I left home this morning and forgot to forward some ports and now I am stuck trying to figure out how to do so from an ssh shell command line by manually interpreting the javascript. Manufacturers, please include REST services to configure your hardware.

A collaboration on distributed data models and their implementations

I am currently working with Jared Rosoff and Bob Chatham at Wave Syndicate. We are building a tool that needs to operate at internet scale and so I have been thinking alot about distributed architecture and implementation. Constructing a tool for this scale of use is new to me. I had some experience with this while working at Tazz Networks but I am still very much a novice.

On the drive home today I was thinking about
1. data models and using the emerging distributed data tools like Amazon’s SimpleDB and Google’s BigTable and its imitators,
2. the contending variables at play when designing a C++ data model for use with ObjectStore,
3. David Hay’s great data modeling book Data Model Patterns: Conventions of Thought, and
4. a comment made long ago in a forgotten blog posting by a PHP programmer who needed good data models and not data user interfaces.

It would be extremely useful if the data modeling community would create data models and implementations that are factored, designed and tested for each of the distributed data tools. For example, a commentary data model (for use with blogs, wikis, discussion groups, issue trackers, etc) using SimpleDB would be quite different from one using SQL, and another using BigTable. Any of the models might need more than one schema implementation per distributed data tool as each implementation differently trades off consistency, latency, concurrency, etc. I wish I had the skills now to lead something like this. It would be a great boon to both current application developments and, perhaps more so, for developer education. If you know of anything happening along these lines please drop me an email.

A call for a new kind of town hall

Public participation in South Kingstown (RI, USA) town and school governance is negligible. Most public meetings, like this week's joint town council and school committee meeting formally opening consideration of the 2009 & 2010 budget, have only a handful of people present. Too often there is no one present.

A little consideration shows that this is to be expected: The current means of public participation were designed around the needs and wants of a very different time then our own. The town council chamber is itself a monument to these needs and wants. It is a physical community gathering spot. Other community gathering spots include the newspaper (for public notices) and the town hall bulletin board. Even the early evening meeting times accommodate a past when the man of the house could leave his children to the care of his (house)wife in the comfort of his home.

This is not the world I live nor the world of most of the people I know. Both parents work, care for the children, and do housework. Sometimes they work in staggered shifts so that at least one of them is at home with the children. And when the children have gone to bed work picks up again for another hour or two: The "mommy work hours" of 9 AM to 3 PM and then 8 PM to 10 PM.

We also communicate differently. We use e-mail and voice-mail. We chat via IM (instant messaging) or texting on our phones. We discuss every kind of subject in online groups. We share our lives with our friends through our blogs and Facebook "walls." And when we do gather in person we do so by "homing in" on a place, a time, and who will attend via a ever tightening circle of short messages over time.

This pattern is not going away. It is deepening. Just as the town council chamber was build to support participation we need now to build a different and effective mechanism for today.

We need our public documents, meetings, and other artifacts online. We need to be notified about additions. We need to be notified when they have changed. We need to be able to comment upon these online and have this commentary considered. These new tools of participation are not ancillary. They are as primary as the existing ones.

Doing this is not a great technical challenge. The software development industry routinely uses these tools everywhere and everyday. Doing this does not require a great operational cost. The tools are free, the storage and computational costs minuscule, and the support costs reasonable. The most challenging cost is to the school's and town's processes. It is not that more work will be required of officials and staff but that the work is done differently. The difference results in making visible to online tools the workings of the school and the town.

I ask all of you to please contact the school committee and town council and ask that they initiate the changes necessary to enable citizen participation in our time. Now is a very good time for change.

Viewing a man page in

Macintosh OS X command line to format a "man" page and open in the Preview application
groff -man /usr/local/git/man/man1/git-config.1 | open -a -f

Update: Using the man -path argument you can further enhance the command line, eg groff -man $(man --path git-config) | open -a -f

A review of the information graphics in GOOD #13

GOOD has gotten some attention recently with their partnership with Starbucks to develop and distribute the GOOD broadsheets. The sheets coincided with the presidential elections and so have both feed the discussion and feed from the discussion. On the value of the sheets and a look at the GOOD web site ( I subscribed to the GOOD magazine for $10.

Issue #13, my first, arrived a week ago and I have been looking at it on and off since then. The magazine design has the spare and clean lines that I like. Great photography and editorial illustration. And I am attracted by the large number of double-page-spead information graphics. Unfortunately, the actual substance of the written and visualized content is mixed.

The first warning flag is the letters to the editor. The first several are responses to the vulgar language and graphics used in a previous issue. While I did not see the previous issue, the issue I do have also contains vulgar language and graphics. As one letter writer writes, this editorial indulgence lowers the seriousness of the discussion. Much like Sarah Palin’s winks and “heck yeas”. The remainder of the written content is interesting -- especially the “Portraits” section -- but thin. A modern magazine must have links into the web. Sadly, GOOD does not.

What initially drew me to GOOD was information graphics and so I was excited to see several pages devoted to this in the “Transparency” section. Upon close inspect they are utter rubbish. They exhibit a wide number of failings that anyone practicing information visualization should well know and innately avoid.

The “Up in smoke” graphic (no web link) is a stacked bar graph using cigarettes as the bars. Each cigarette presents a state. The length of the cigarette is the population and the length of the filter represents the percentage of smokers. The failure is that the width of the cigarette is also changed so that the relative proportions of the different cigarettes are kept constant. This results in NY’s 18.3% of smokers looking like a significantly smaller number than CA’s 14.9% of smokers. Further, the graph is organized by kinds of smoking bans -- no bans, work bans, bars bans, and restaurant bans -- and so finding your state is a hunting trip. A table would have been much better.

I am still trying to figure out “Seeking Refuge” ( Just what does knowing “refugees per square kilometer” really tell me?

“Burning Fuel” ( is a visual correlation between cars and humans. The energy in a gallon of gas is compared to common foods. While interesting, why is there an information graphic? Information graphics are supposed to enable the reader to make more connections between facts. I am looking to build upon what I know or have just read. And so I expect that when a human’s sticky insides are block-colored the same as the car’s insides that this has meaning. What correspondence am I to draw from the car’s seat cushions, battery, air filter, and some unknown box near the front-wheels's transmission with the human’s heart, liver, and mussels? Isn’t color coding a universal information graphic best-practice? Yes.

Lastly, the graphic “The three-trillion-dollar war” ( is a perfect example of how to present a simple list (as is done on the right side of the graphic) as a complex maze of weak visual analogies. Perhaps I am missing something deeper. I want the visual to have meaning. I am rooting for the illustrator. But, in the end, I don’t think I am missing anything: It has no meaning. It is just a picture.

I am not a practitioner of information graphics. I am a consumer and a fan. I do know how to read this stuff and I know that some visual languages need to be learned before they can be read. I am willing to put the time in to understanding the visual just as much as I am in the understanding the prose. Don’t give me interesting pictures pretending to be information graphics. Please.

Let’s hope the next GOOD is not a DOOG.

Michael Moore has a plan

I just signed Michael Moore's vision to save the Big 3. I don't agree with all of Moore's politics but if there is anyone who knows how incompetent the Big 3 management is and scope of the impact of their failure it is he. This is a great opertunity to use the existing infrastructure and relationships to build a new transit economy in the US.

The Incredible Convenience of Mathematica Image Processing

There are three computational tools that I would like to know well. The tools are SketchUp, ArcGIS, and Mathematica. They successfully map the user's mental model of the problem to the tool's control leavers for a solution. In the hands of a skilled practitioner incredible results are accomplished. The blog posting The Incredible Convenience of Mathematica Image Processing is a reminder of this. This is inspiring as a potential user and humbling as a software developer.

Found via Tufte blog.

Socially acceptable tyrannosaurus rexs had long and muscular forearms

What if dinosaurs were intelligent beings with burial rights that left no trace of their members. What if the dinosaurs we uncover today are the rejected and banished members of their society? Perhaps socially acceptable tyrannosaurus rexs had long and muscular forearms.

"The Story of the Ribbon"

I stumbled upon Jensen Harris's blog and his "The Story of the Ribbon" presentation. This presentation is a wonderful case study of how the Office 2007 ribbon evolved. This long presentation is well worth watching by anyone interested in interface design and/or usability process. The content is great as is the Q&A at the end. (Excuse the "presentation junk".)

Back to

I have decided that is by far the worst email address I have ever had. When I am addressing a message I think "person" and "place". I would guess that everyone does. So, I am keeping as my public address.

Why votes need to be on paper ballots

Techdirt has a video from Video the Vote with a West Virginia official showing how to calibrate the voting machine and it not working. Obama's laywers are going to be busy! Obama really needs our money now.

It seems the only safe way to vote is via paper-based early/absentee voting.

Thank goodness South Kingstown government has the sense to continue to use paper ballets with OCR-based counting. At least here it is POSSIBLE to do a manual recount here.

Callie Shell's Obama photographs

A great set of Obama photographs taken by Callie Shell: "I loved that he cleaned up after himself before leaving an ice cream shop in Wapello, Iowa. He didn't have to. The event was over and the press had left. He is used to taking care of things himself and I think this is one of the qualities that makes Obama different from so many other political candidates I've encountered. Nov. 7, 2007."

Vote to strengthen RI by strengthening RIPTA.

An open letter to Rhode Island's elected officials:

Rhode Island Public Transit Authority (RIPTA) need not be profitable. Its operation is in the public good. At this time, with real decline in wages, a higher percentage of unemployed than elsewhere in the US, and the need to move people around the state to take advantage of employment opportunities, we should be expending RIPTA. RIPTA is not a service: it is an infrastructure. Judge its utility based similarly to other infrastructures. When have the roads and bridges been profitable? They never have been and never will be, but we continue to expand and maintain them because they are in the public good.

If you are paid anywhere near the minimum wage then it takes you more than a full day's pay to buy a tank of gas ( And gas is only part of the operating costs. How to you get to work if you can't afford a car? Without RIPTA then you must walk. Employment opportunities within walking distance are severely limited especially in RI where we have relatively small urban cores.

If RI wants to build the 21st Century Innovation Economy let us start by innovating and supporting public transport. Please vote to strengthen RI by strengthening RIPTA.

Teaching has become automated, measured and homogenized

H&O no longer attend public school. They now attend a small private school.

They attended kindergarten through 3rd grade at the nearby Peace Dale Elementary public school. Kindergarten was great. H&O had a dedicated and effective teacher and teaching assistant. For 1st grade they attend the Kaleidoscope program -- a small school within the larger public school. Kaleidoscope looped 1st and 2nd grade and 3rd and 4th grade so that there were fewer transitions, greater student and teacher understanding, and generally a longer time for teaching and learning the core subjects. First grade was great; they had dedicated and effective teachers. Then the school abruptly canceled the program. Second grade was not great for O; a bad teacher and then a change in teachers. O hated school. Third grade was not great for H; four teacher changes in one year. H hated school.

What Chris and I also saw during this time was teachers losing teaching time to administrative tasks, more and more testing, more and more busy-work homework, and, frankly, a chaotic syllabus, advanced beyond the capabilities of most children, and with prescribed instruction. When the best that you can do is instruct and not teach how long would you stick around? Great teachers retired rather than face this. And finally, any attempt to attend to the character and the spirit of the children was gone.

So H&O are now at a school where teachers teach. They also are the administration. And they do attend to the character and spirit of the children. We are all much happier.

I had told myself when H&O were of school age that they would attend public school. I believed in local public schools and the public school system in general. We live in a town with highly regarded public schools. Unfortunately, we also live in a time where teaching has become instruction -- automated, measured, homogenized.

A bogus alternative reality

This Sunday's New York Times has a number of opinion pieces on the McCain lies. I encourge you to read read at least the Right to Smear editorial and Frank Rich's Truthiness Stages a Comeback column:
[...] McCain, Sarah Palin and their surrogates keep repeating the same lies over and over not just to smear their opponents and not just to mask their own record. Their larger aim is to construct a bogus alternative reality so relentless it can overwhelm any haphazard journalistic stabs at puncturing it.

McCain-Palin don't just lie, they lie about the lies and then play victim

Michael Tomasky's column this morning is sub-titled "McCain-Palin don't just lie, they lie about the lies and then play victim. How do you fight that?". This is deeply troubling. And that Obama does not seem to have a ready response is also equally troubling. We can't let Obama to be "swift-boated".

Beginning to not like Ubuntu

I am beginning to not like the Ubuntu Linux distribution. I needed Ruby and Ruby on Rails the other day and it required many more packages than, for example, Red Hat or plain Debian required. I needed the packages
  • rdoc
  • irb
  • libyaml-ruby
  • libzlib-ruby
  • ri
  • libopenssl-ruby
  • ruby1.8-dev
  • build-essential
  • sqllite3
  • libsqlite3-dev
Plus, the Ruby Gems package was ancient and so I needed to install this from source.

Today I needed vncserver, but after installing the RealVNC package -- called "vncserver" -- I and was unable to get it to use the ~/.vnc/xstartup. (And it did not know about the default location for Ubuntu X11 fonts either.) So I switched to tightvncserver and all was well. Perhaps if I knew some of the philosophy behind the Ubuntu packaging decisions I would be less at a loss for why stuff does not work. But I don't want to know this stuff. Sigh.

Windows XP for an old T20

In a past posting I wrote about finding the right OS for an old IBM Thinkpad T20. In the end I abandoned Linux and installed Windows XP. I don't want to be a system administrator but even with Ubuntu I felt like I had to be. The many hours of trying Linux distributions did teach me that 1) Damn Small Linux is fantastic and 2) Windows XP is a fast and capable OS on low end machines. However, with that said, I think the T20's has extraordinarily well designed and matched hardware.

Physical connection to the past

Reading the introduction to the cooking book Melody of India Cuisine I was given pause by the opening sentence: "Mom and I started dabbling on our book twelve years ago on the model 1960 Underwood typewriter, the same typewriter my Dad had used to type his Ph. D. thesis." Given the book's publication date, the typewriter had been with them for 24 years. How many tools with personal history or significance do I still have? The IBM PC Jr and TurboPascal that I wrote my first compiler with? The Macintosh II Symantic THINK C++ IDE that I wrote Sextant with? Perl 4 that I used writing Mesa/Vista. To keep these tools would be silly. How do you retain this physical connection to the past?

Red Chair Studio assistant wanted

Red Chair Studio, Chris's other self, is looking for a assistant for 5 to 8 hours a week working here in her Peace Dale, RI studio. She is looking for a detail oriented and flexible person with sewing skills to help with production and shipping. Call Chris at 401-258-0817.

The software IDE and the non-software technical team

I very much like integrated development environments (IDE). Like most engineers, I suspect, I scratch a little while using them because they don't work just the way I want. However, the productivity increase is enormous in comparison to the small irritations. Yesterday, I watched the View Case Study: USDA Maximizing Collaboration with NetBeans and Codebeamer presentation. CodeBeemer adds to the NetBeans IDE development collaboration features like revision control, issue tracking, discussions, etc and all surfaced and integrated in the NetBeans approach to information design and use.

This posting is not, however, about software development but about other non-software technical groups outside of the software industry. Do these groups know about these rich tools? I would really like to see a print shop or an archeologist's dig or a municipal planning department use these tools. Perhaps the leap from the common ecology of Word documents + Excel spreadsheets + remote file system is too big a one.

Does anyone have experience with non-software teams using software IDEs to manage their work and materials?

Discontinuing a long running script

Looking at a bash script I wrote yesterday I see that I did not give the user a way of discontinuing the processing's long run. The user's only resort was to kill the process. Killing a long-running script is never good action. Where was the script in it's process coordination? Is clean-up needed? If so, from where and how? It would be better to let the script exit at a good discontinuation point.

To this end, I would like to write the script

CONTINUANCE=$(continuance $(basename $0))

for f in ...
if [ -e $CONTINUANCE ]
info Processing $f
warn Process discontinued before completion
exit 1

C=$(continuance $(basename $0))

while [ -e $CONTINUANCE ]

if [ ! -e $CONTINUANCE ]
warn Process discontinued before completion
Where the call to "continuance" creates a temporary file, sets up a HUP trap to delete the file, and prints the instructions

2008-08-11 13:58:26 INFO To discontinue processing
send "kill -HUP 31788" or remove the file
The function is very simple
function continuance {
f=$(mktemp -p /tmp $1-continuance-XXXXXXXXXX)
trap "rm -f $f" HUP
info To discontinue processing send \"kill -HUP $$\" or remove the file $f
echo $f

Invisible streetlights

I have been following the South Kingstown Parks' department's plans for changing the Peace Dale Village Green. One of the change components is that the department wants to install is "period" lighting along the pathway between the Guild and Kingston Rd. Few people like the idea for reasons historical, safety and aesthetics.

Today I found the most interesting design in Jongoh Lee's invisible streetlight. The light is little more than an artificial branch with a few leaves embeded with LEDs. Fantastic!

Lee's design does not solve the Green's lighting issues, but seeing this kind of innovation gives me confidence that there is a creative solution out there. Just got to keep looking.

On to Git

I was not looking forward to learning Subversion SCM. It just looked too complicated. The basic manual is a few hundred pages. Thank goodness that Git is taking over the SCM world. I can move on from CVS, an old and fond friend, to something that actually works for distributed development.

In my search for Git introductions I found Getting into SCM with Git. Worth reading.

Scratch and programming for kids

Scratch is another graphical programming language oriented towards children. It is built in Java and has a programming environment and a runtime environment. The runtime environment is only needed if you want to share your (read-only) program with others via the web. The programming environment is a little too modal and other ways ridged for my tastes -- that is, with my "kid" hat on. When you watch kids play they clearly have a center and a periphery. Their tools need to model this.

RESTful search

I am working on a small project to build a web application using current best-practices and tools. To start, I am working on the RESTful interface -- the application's API. What I did not understand was how to do a RESTful search. Searching is not a GET because there is no one resource or all resources returned. It is not a POST because nothing is being updated. What I discovered looking around the web was that when using REST you are not limited to the gang-of-four HTTP methods -- GET, PUT, POST, and DELETE -- but instead are open to a whole other set of methods [a][b]. There is a SEARCH method at IETF in early draft state. This is good enough for my needs.

I will use Java for this project. I know how to implement the interface using the Servlet framework but I wanted to use a framework designed for RESTful interaction. I found the Restlet framework and really like what I see so far. (I have not coded anything yet and so don't know how practical it is.) There is also JSR 311, "JAX-RS: The Java API for RESTful Web Services", which is an annotation-oriented design. I like the Restlet's class-oriented design much better.

Paper craft and architectural and landscape projects

There seems to have been a resurgence in paper craft models. I think part of the reason is the ability of tools like Pepakura Designer to take a 3D model and determine the parts needed and layout them out on pages. (I would love to know how this is done -- at a moderate level of detail.) It occurred to me that you might be able to use this with Google's SketchUp to create models of architectural and landscape projects. Perhaps the resulting paper craft models are not of a quality suitable for client presentation but it sure would be fun.

Damn Small Linux

I have been thinking about teaching my kids how to use eToys. To do this, it would be best if they each had their own box and so I have been looking for cheap laptops. I found two Thinkpad T20's for $100 on Craigslist in various states of repair and bought them. I have worked on T20's in the past and so I know their idiosyncrasies well.

One of the idiosyncrasies is getting X Windows to work. I first installed Ubuntu and everything worked but at a glacial pace. The T20 only has a 700Mhz CPU after all. But even at 700Mhz X Windows should fly. The real problems are 1) Gnome is a system hog and 2) this machine only has 128Mb. (It still feels weird that 128Mb is not much RAM. I remember very well Carnegie Mellon University's 3M machines which were to have 1 million bytes of memory, 1 million pixels on screen, and 1 million instructions per second!) So I installed Xubuntu. And this time the pace was faster but only by a smidge. And boot time was still many minutes in duration. Application switching could take a minute.

Searching for a smaller Linux distribution I came across Damn Small Linux and installed that. Well, installed is the wrong word as I first ran it from the LiveCD. It booted the T20 within seconds. X worked. Internet worked. Everything worked. And worked fast. This was a truly usable system on a T20.

The down side, and this is something I am still fighting with, is that it is not a true Debian distribution. It has its own packaging and distribution system. DSL's author had to make significant changes to the Debian core to make a workable distribution that fit on a business-card CD. DSL is intended to be a system for recovering a broken box. I understand this and the understand the impact the changes would have on DSL's ability to use existing Debian packages. However, the impact is only for a few classes of packages. Most POSIX and GTK applications will run fine. I wish the time and effort put into building the packaging system had been used to allow for the installation of standard Debian packages instead.

Before I wrestle too much longer with DSL I need to get Squeak installed. Wish me luck.

Another tag cloud

Leigh Dodds had a link to the fun Wordle tag cloud. Here is mine from What is striking is the large number of links I have tagged as "to-read". Too many interests for the time I have.

Yard Sale, June 21

Yard Sale
Saturday, June 21
8 AM to 1 PM

574 Saugatucket Rd
Peace Dale, RI
Help us clear out Chris's art stutio and my kids's rooms! Where again? Here!.

Roof woods

Roof woodsWhen you spend many weekends gardening -- new beds to dig, old ones to weed, plants to move, plants to add, grass to mow, etc -- sometimes you simply forget to look up every once in a while. Clearly, I need to clean out the gutters.

Groceries transportation

Groceries transportationNo excuse now for not riding the bike the few miles into town to buy groceries and small gadgets.

Log when your scripts start and end

When you write scripts that are the controllers for processing data make sure to log when they start and finish the work. It is so frustrating to debug an erroneous processing result -- often days or weeks later -- and not have any indication when the process actually started and finished. A missing finished log entry is the first indication that the processing ended abruptly.

So, in bash shell script use
info starting
info finished
where is somewhere in your path and has the content

# info, warning, error and fatal functions to display timestamped
# messages to stderr. Use the infof, warningf, errorf and fatalf
# functions to format the message using printf. A newline is
# automatically added and should not be included in your printf
# format string.

function info {
log INFO $*

function infof {
logf INFO $*

function warning {
log WARNING $*

function warningf {
logf WARNING $*

function error {
log ERROR $*

function errorf {
logf ERROR $*

function fatal {
log FATAL $*
exit 1;

function fatalf {
logf FATAL $*
exit 1;

function log {
echo $(date +'%F %T') $* 1>&2

function logf {
shift 2
printf "$(date +'%F %T') $level $format\n" $* 1>&2

Most other languages have extensive logging support.

Hardly seems worth posting such an obvious piece of advice, but look around into your scripts -- as I did mine -- and notice how few provide this data.

Remove shoes, cast vote

RI's Governor Carcieri has proposed that a photo-id be required to vote in RI. After all, (to paraphrase) "you need an id in order to fly". Perhaps we should also be expected to remove our shoes when voting too. It is hard to understand how our politicians continue to push for mechanisms to disenfranchise citizens.

How the States Got Their Shapes

I listened to some of How the States Got Their Shapes on the drive today. I was fascinated by the history of West Virginia -- a state that exists because of Civil War opportunism. How physical borders are negotiated is a great segue into how other kinds of borders are negotiated in families, sects, corporations, etc. Great source of projects for a high school, eg "Why is South Kingstown the shape it is?" or "Why does the Chariho school district exist?".

Oh no!

Two 3G iPhones + AT&T family plan + one mini Mac + development kit + reduced billable hours learning, developing & playing = one expensive Q3!

Goalkeeping & HTML Canvas

Henry is the goalkeeper for his soccer team. I was practicing with him the other day and trying to explain how to protect the goal by forcing the offensive player to have only one side of the goal to play the ball to and then only at a very acute angle. Henry is 8 and so none of this really sank in. Once again, I wished I had a pair of magic lens so that he and I could literally see the angles I was taking about overlaided on the playing field.

I was retelling this story to Geoffrey and I realized this was a small enough visualization project to perform my first canvas experiment. And here it is


Update: Sometimes I am so old-school. I should be referencing augmented reality.

Sweet Cakes has a web site

The sign said "Cake is the new flowers". I do like both but tend to go to Sweet Cakes far more often than to Rose Shack.

Seeing code working

These internal views of how the browser renders an HTML page are fantastic. Here is how the browser sees's home page

The OLPC laptop has the "see code" button. What it really needs is "see code working" button.

Keep only the account name and password digests

It occurred to me the other day that both the account name and the account password should be recorded in the database as MD5 digests. There is really no reason to keep a clear text version of either of these datum.

Web experience needs the equivalent of Screen's workspace persistence

My web experience needs the equivalent of workspace persistence found in Screen. I am too often working with old web applications that process data as you wait. You submit the form and the application responds with "Please wait." And I do. And, when there is a lot of data to process, the wait can be for hours. During this wait nothing untoward can happen to the network connection or the browser. If something untoward does happen then the processing has to begin all over again.

What the Screen web equivalent would allow me to do is to browse the web as I normally would. When I detach from the internet, the pages I have displayed or the ones that are in the process of displaying would be available again when I re-attach to the internet. It is important to remember that these pages are the pages retrieved when I was previously attached to the internet. They are not pages obtained by re-requesting them. Doing so would only begin all over again the long data processing.

With workspace persistence, I could start the data processing using the old web application and not have the worry about a network hiccup or browser crash losing my work.

A road to solving this problem would be an augmented proxy server.

Enriching the search query

Most successful -- that is, used! -- online search tools have a remarkably simple query interface. The single input field is where your query interface needs to be. The wealth of your data is made available not by complicating the user's interface, but instead by automatically and intelligently enriching the query. Enriching the query includes the following, in no particular order,

  • term extraction -- words, acronyms, dates, numbers, email addresses, etc
  • term ranking -- some words are more important than others.
  • hyphenated terms -- query for both with an without hyphens
  • accented terms -- query for all extant variations of the term in the data (in the US, for example, accented characters are rarely used and so American's tend to drop the diacritics).
  • term spelling variants -- "color" and "colour", etc
  • term stemming and other transformations -- soundex
  • numbers and number names -- "101" is also "one hundred and one"
  • dates and times and date and time expressions -- "tomorrow" is "Thursday, 8 May 2008"
  • social relationships between authors -- Smith and Jones are frequently co-authors and so a search for Smith should include a search for Jones (within reason)
  • reference relationships between works -- the original linking
  • professional vocabularies -- nurses and doctors don't use the same terms for everything
  • lay vocabularies -- "cardiac arrest" is "heart attack"
  • spelling correction of terms -- only use the extant terms
  • broadening and narrowing of terms -- you say "tomato" and I say "solanum lycopersicum" or "vegetable".
  • ...
As always, do as much on behalf of the user as you can before asking them for more data.

David Coté and the South Kingstown Republican party's ineptitude

David Coté, of the South Kingstown Republican party, is once again in the local news. This time, he is not calling for the removal of several million dollars from the local school budget in his and the republican party's utterly inept effort to force competitive bidding on health care insurance for teachers, nor for successfully agitating for a referendum for a $500,000 cut, but instead recommending that the town would be better off giving the $10,000 budgeted to pay for the referendum to the Chamber of Commerce. Why would the town management, that has successfully held low the municipal cost increases each year to counter the ever increasing burden of the costs associated with federal and state mandated but unfunded educational programs want to give $10,000 to the Chamber of Commerce? The same Chamber of Commerce that recently spent almost $700,000 on new office space? The same Chamber of Commerce that is supported by the 700 businesses it promotes. David Cote should not be on page one of the paper, but in the funnies.

Where is sane political debate in this town? Clearly, not from the republicans. And clearly not from the democrates who continue to sit mute on the council.

Data in HTML

Web pages have a remarkably long half-life. Data files, by comparison, have a significantly smaller one. When preparing data for display in HTML make sure to include in the HTML the data in a form that can be readily reused. Otherwise, you are doomed to screen-scraping again and again.... Now, where did I put my screen scrapper?

What is the danger of the non-passenger?

Let's assume that the TSA's procedures at the security check points are effective. That is, they can detect the danger associated with a single event -- a gun, a knife, an explosive. Perhaps they can detect the dangers associated with a small set of correlated events occurring during a small time window. But what about a very large set of apparently-random events over a very long time window?

As many passengers do, I checked-in online. I was without a printer and so was unable to print the boarding-pass. I have done this before and so knew this was not a problem as the automated kiosks at the airport will re-print it. Obtaining a boarding-pass is an anonymous act.

The boarding-pass is a complex document with many associated facts: the bar code, the passenger details, the flight details, etc. This set of coordinated details is challenging to forge. Solving a crossword puzzle is easier than making one.

The TSA uses none of this complexity at the checkpoint. The checkpoint just checks that the name on the boarding-pass matches the name on the driver's license. In a photoshop-world, how hard is it to make the name on the boarding-pass match the name on the driver's license? Creating 10s or 100s of such altered boarding passes would allow these 10s or 100s of non-passengers into the gate area.

These 10s or 100s of non-passengers don't stay in the gate area. They enter and shortly thereafter leave. The TSA doe not check the departing passenger's legitimacy of being in the gate area. Why bother to check people leaving the airport? What damage could they do?

The damage comes from the result of a large set of apparently random but actually correlated set of events spread over a long time window. Why bother to fabricate your weapon outside the gate area when you can fabricate it inside the gate area. Bring it in one spring, one plate, one activator, one chemical, etc per non-passenger over several days.

It took me less than an half-hour waiting at my gate to formulate this scenario. I hope the TSA already has this covered.

Dirty feet

Once again, I was subjected to the ridiculous security procedures of the TSA. Wearing underwear, shorts, loose shirt and sandals I was obviously conveying weapons paraphernalia. Upon a sharp rebuke from TSA personal, I was told to remove 1/4 of my clothing to ensure the protection of the other passengers. Luckily, the 1/4 was only my sandals. And so, like many 10s of 100s of people that day, I trod through the metal-detectors spreading foot-borne bacteria between Rhode Island and Virginia. The terrorist need not trouble themselves with dirty-bombs when dirty-feet will do.

Raged and faded bumper stickers like the administration itself

I just returned from the heart of Republican territory -- the Hampton Roads area of Virgina. Most of the money in the area comes from military expenditures. It is a world there Obama bumper stickers are very rare. It is also, unsurprisingly, a world of Bush Chaney bumper sticks. Now raged and faded like the presidency itself. Holding on by a few patches of adhesives. John McCain bumper sticker are equally as rare as Obama's.

Jill Bolte Taylor's My stroke of insight

Jill Bolte Taylor's My stroke of insight is a remarkable talk about her experience of a stroke. Since my mother's stroke, I have read a number of accounts of stroke experiences and other helpful guides. None of these, however, compares to Taylor's rich and emotive retelling of her stroke. It brings me closer to a knowledge of my mother now. Well worth twenty minutes of your time: have a hankie ready.

What is a pseudo-systems guy to do?

I was in the early stages of writing a JavaScript layer on top of Servlets, JMS, and Derby running under Glassfish. The whole point of this effort was to give me a rapid development environment. I need this environment for two reasons -- and so do you if you are a developer. I need a platform on which to quickly try ideas. I need a platform on which I can deploy production quality implementations. If these are one and the same platform then I have less cognitive dissonance between play and purpose. It is also a fun activity. (Yes, I am a card carrying geek.)

But what is a pseudo-systems guy to do when Amazon has its developer tools (S3, SQS, EC2, etc) and now Google has App Engine? Why would anyone build an application on local infrastructure when these global infrastructure are so powerful and readily usable. I guess, I need to re-focus on the application that I wanted the infrastructure for in the first place. This is a good thing but the parting does bring sorrow.

Web applications and desktop integration

Much of my tool set is now housed inside the browser. The tools are web applications. My email is there. My calendar is there. My collaboration is there. My industry awareness is there. Etc. This web application suite is not uncommon. The trouble is that neither the Microsoft Windows desktop nor FireFox work together to provide me with a common application management experience. For example,

• The Windows "task bar" shows one task button labeled "5 Fire.." rather than one task button for "Email & Calendar", another for "Bloglines", etc.

• I can place URL shortcuts into the Windows "Start" menu which helps with organizing these tools. FireFox even suggests that I can drag-n-drop the URL from the browser's address bar into the start ment, but it fails to drop and so I must manually add the short cut.

• When I use the Email shortcut, for example, FireFox opens another browser tab to the URL even if already have the URL open on another tab. What I want is Email brought to the front just like an ordinary desktop application.

• FireFox does nothing to help me NOT replace my email tab or calendar tab with other transient content. When I am using the "Email & Calendar" browser I want all non-email and calendar content to be opened another browser window.

I want a browser with great desktop integration. There are some great FireFox extensions that make manipulating tabs easy but doing this is manual. They also only make FireFox better. The Windows desktop needs a new set of rules for integrating web applications. Mozilla's Prism might help. I suppose I could code something up with Greasemonkey but I really don't have the time and what time I do have I have other more interesting uses for.

How do you manage the web application and desktop integration?

Making visible intellectual work

I love how David Macaulay in this 2002 TED talk reconstructs how he worked out his book Rome Antics. I wish more people did this: So much of our work -- intellectual work -- with its seemingly random explorations, horrible missteps, beautiful tangents, self doubts and sublime flow is invisible and when it is visible the sights are no more than shadows. No wonder my kids would rather be dirt bike racers and spys. (Then again, sometimes, so would I.) So, watch David Macaulay's talk with your kids.

Technologies, talent, experience and, soon, data

More data usually beats better algorithms is an interesting post about more sources of data used together with simple algorithms do a better job then complex algorithms. Where this is interesting to me and my customers is that we need to find or develop data sets that better support the customer's users more successfully using the customer's site.

Most high-end web development shops are not yet at the point where their technologies, talent, and experience alone are not enough to turn a lead into a customer. However, we should be thinking about adding data to this mix now.

Value of chemicals found in the human body

The images show an original picture and a re-imagining of it using whole images found on Flickr. This has been done before at the pixel level where images are picked for their approximation to the pixel in overall color and brightness. Using a coarser granularity, however, produces a much more interesting image. Chris, my wife and artist, likens the result to the the value of chemicals found in the human body.

Technically, it is also a fantastic use of Hadoop. Hadoop is an implementation of the MapReduce algorithm made famous by its use in Google's software.

Why are natural numbers still used for primary keys?

Most database products have a facility for creating unique numbers. These numbers are commonly used as the primary key of a record. You can often see this exposed as a small integer in the URL of a web page. This posting is not about the risks of exposing primary keys but rather about why natural numbers are still used for primary keys.

The observation I want to make is that 1234 can easily be mistaken for 1324. Suppose I have equal access to both the records associated with 1324 and 1234, what mechanisms prevent or at least hinder me or my agents using of the wrong one?

UPCs numbers are numeric but not natural numbers. Credit card numbers are numeric but not natural numbers. ISBNs numbers are numeric but not natural numbers. Each of these are examples of identifiers exposed as "numbers." These identifiers have useful characteristics. They are self validating so that some transcription errors invalidate the identifier. Some identifiers are self correcting. And in the vast universe of value for, for example, 13 digit numbers only a small fraction of the values are valid ISBNs.

As ever more systems record URLs as references to data we need to be ever more careful about our identifiers. Public identifiers should not be natural numbers. Perhaps any system of identification that does not require ordering by identifier should not use natural numbers. It is time that database products had a facility for creating and using unique identifiers that is as easy to use a sequence table or an automatically incrementing integer. Until then, add a check digit.

ISBNs are not stricly numeric as the check digit might be 10 and this is represented as "X".

RI, go vote today for Barack Obama

My reasons for supporting Barack Obama are my two 8 year old boys. The next president is likely to leave office when they are 16. I want these years for them to see America at its best. I want them to see and experience leadership that rights wrongs. I want them to experience an optimistic America. I want them to participate in an optimistic America. I fully expect the next years to be difficult for this country. I believe that Barack Obama can lead us through this hardship.
bōst-: A self-serving blog posting. ~-ing: The act of publishing a self-serving blog posting.

New Common's working space

Last week I spoke with Robert Leaver and Michelle Gonzalez at their New Commons offices. It was a great conversation and one I hope to continue. Robert's and Michelle's manner and their working space immediately make you relaxed. I am usually confident and comfortable in new situations but the feeling of welcoming there was intense.

Their working space is very well designed. Robert's work creating spaces and places would of course lead you to expect this. Here is my rough [*] floor plan of the space. I particularly liked the open hallway and the offices's French doors onto the meeting space.

[*] Visio's floor planning tool is very easy to use. However, the formality of the final visual hides too well the inaccuracies of dimensions and the mistakenly forgotten or added features.

Marrying the working environment with the working materials

I love this team-space designed by Clive Wilkinson Architects and this personal-space design by Francois Perrin. I am attracted to these carefully designed spaces exposing raw materials because I spend so much time inside software. There is a comfort that comes from marrying the working environment with the working materials.

Good example of labeling

The Virginian Pilot, the local newspaper in Virginia Beach, VA, has a good graphic design department. I have always liked their maps and this Sunday's edition had another fine example detailing the changes on Witchduck Rd. One element of their design program I really like is how points of interest are labeled using a modified balloon -- a simple pin shape with the label text below. All the points of interest clearly labeled and aligned at the edge of the map and the connections acute triangular shape does not interfere with the map's content.

Does anyone have pointers to specific services or tools for cleaning up newspaper scans? It just seems that the page color can be subtracted without touching the printed image.

Give Your Readers Something To Do

"For some content providers, looking at content through the prism of actionable content is also a good way to decide whether it is content worth publishing at all. If your readers can’t do anything with the content – meaning they can't use it - then how useful is it?" Vinny Goldsmith, Give Your Readers Something To Do. The full posting makes other good points too.

Make the "holding page" useful

Building a web site always takes longer than expected. And the size of the web site does not seem to matter. There is an inverse relationship between cost and commitment: A $10,000 job with 100% commitment takes about the same time as a $1,000 job with only 1% commitment. And who can commit to any project 100%. The point of this is that that web site's "holding page" better be useful. An "under construction" sign does not help your client or your client's audience.

For example, the holding page at Social Venture Partners Rhode Island ( is a one page web site. It tells you who they are, what they do, what is happening, and who to contact. All this on one page and maintained with an insignificant tool -- a few dozen line PHP script.
Mofuse, from local Pawtucket company Swift Blue, has an interesting tool that makes your web site mobile phone friendly. Here is its take on this blog. I like the idea that this can be done, but would rather see sites consider the information architecture and visual design of their content for mobile users.

Simple & Direct Tools

I have been using sftpdrive to enable me to mount the directories of remote file systems as local drives with Microsoft Windows. I chose this product over others as it does not cache much data about the remote directories and files. Not caching is important because I tend to modify files both via an editor in Microsoft Windows and an editor on the remote system. Without caching there is a noticeable delay in directory navigation. However, this is a small price to pay for having absolute consistency in the content from Windows and from the remote system.

One of the tools I use that mitigates the directory navigation delays is a Java class and package browser I wrote several years ago. The browser is a 300 line perl script that displays an interactive list of packages on the left and an interactive list of classes on the right. The significance of the class view is that it lists all the classes under a given package hierarchy and orders them by class name. For example, the figure shows all (4) classes under the "com.ingenta.oso" packages. Double-clicking a class name opens it in an editor.

The tool is very simple but surprising useful. There is no comparison between it and powerful package and class browsing components in Eclipse and NetBeans. However, these weighty tools sometimes offer too much when all you want is a class file list and quick access to the source.

Google Calendar and Travel Time

A small improvement to Google Calendar would be to allow you to define travel time around an event. For example, the illustration shows that to attend the 6 pm meeting you need to leave by 4:30 as the meeting's location is an hour away and you need to find parking. (The drive home is shorter because you don't need to find parking!)

30G iPod for sale

If you are interested in a very lightly used 5th generation, 30G, white iPod I am selling mine for $165. You can buy it from me or buy it via SOLD. I have had so much success with Marketplace. I highly recommend it.


I decided that it would be useful to get the international domain name ag× for Andrew Gilmartin & Associates. (I could not get "ag&a", "ag+a", "ag•", or "ag†" and there is no IDN for "№1".) I used the IDN Conversion Tool to get the ASCII version of the domain name "" (this is the Punycode version). With this, I registered the name with GoDaddy. Neither nor Yahoo! support IDN just now. In a day or so I will have the domain forwarded to this blog.

Use the Unicode charts to concoct your own strange international domain name.

Oh, under Windows you need to type ALT-0215 to get the ×.

Model Railways

I attended the Amherst Railway Society Railroad Hobby Show yesterday. I have only gone to this show once before with Geoffrey some ten years ago. And it hasn't changed much. The crowd has gotten grayer but it continues to be a horrifying collection of gimps, geeks and retards. I fitted right in.

I started and ended active model railwaying in the '70s in England. A family friend was an avid railwayer and I caught the bug from him. He was a wonderfully patient and enabling mentor. It is a great hobby for me because of large range of skills it takes to be effective. If you want to model operations you have to design a track layout, rolling stock, and schedules that work together. If you want to model landscapes you have to learn the skills necessary to represent 3D space in miniature. Lots of fine mind and fine motor skills are necessary. I was never at a loss for something to do because on a model railway there is always something to finish or something to start.

For the last year I have been toying with the idea of starting again. When my mentor passed away, now several years ago, he passed along the collection of locomotives and rolling stock he had with him at the end. Every now and then I take them out of the boxes and have a look. Henry, especially, likes to look too. It consists mostly of modern US traction and sock. I am not sure why, near the end, he stopped modeling his beloved Welsh steam railways. So what to do?

There are two kinds of model railwayer or railroader. The first kind likes to create a place in time and so tends to focus more effort on the scenery. You construct buildings with great detail and place them in naturalistic settings. Buildings, people, and trains are aged and weathered. There is litter. This is the kind of modeling I did. The second kind likes to run a railway. They manage a division or, sometimes, the whole railway. Trains move from place to place because there are passengers to carry, live stock to transport, and goods to distribute. There needs to be rules of governance and operations and a timetime to keep. My mentor straddled the two kinds. I have vivid memories of sitting with him at the dinning room table with a blue school notebook, a ruler, and pencile and working out of the distances and timings of station runs. Accompanied by background sounds of locomotives coming from his record player -- an 0-6-0 shunting wagons in the fiddle yard with too much of a head of steam. I also straddle the two. So what to do?

What the show did for me was resolve the question. Do both. I will build an N-gauge layout primarily for operations. N-gauge allows for long runs in a smaller space and long runs are necessary to make the operations work interesting. I will also find a club that has a modular layout that uses a larger gauge -- perhaps On30 gauge -- so that I can create detailed scene. The Narragansett Bay Railway & Navigation Company is a fine example of which.

You had me at "Hello"

I have been been thinking about and exploring Groovy for the last few weeks. I have been looking (forever it seems) to something like the Rails web framework built on Java. There is much to like about Ruby but in the end it is just another imperative, object-oriented language with poor performance, bad threading, libraries "built on a learning curve" [*], and non-existent internationalization. Rails, however, is much more interesting than Ruby. The two parts of Rails that I most like are (1) the "convention over configuration" approach to associating components and attributing behaviors to them and (2) the integrated support for a version's development phases and migrations between versions. These are great boosts to productivity.

However, I am committed to Java for the foreseeable future for most commercial work that I do. (I also work in Perl and PHP but these languages are mostly used for integration projects.) Java offers a wealth of best of breed components of which most are open-source and use the generous LGPL or BSD licenses.

So I was excited to find out about Grails and, as mentioned in an earlier post, after reading Getting Started with Grails I found that it satisfied my need for a Rails work-alike based on Java. My concern was that it was written in Groovy and not directly in Java. And so, I spent the better part of three weeks working my way through Groovy in Action.

Groovy in Action covers the language in detail, its idioms, and its use in several problem domains -- XML, relational databases, GUIs, web application, etc. I am very impressed with Groovy. I was not expecting to be. It is another imperative, object-oriented language with reasonable performance, great threading, best of breed libraries, and end-to-end internationalization. The "You had me at "Hello"" moment came a third of why through the book when the author explores using "builders" to construct XML documents. The code is
def builder = new groovy.xml.MarkupBuilder()
build.number {
description 'Square and factors of 10..15'
for ( i in 10..15 ) {
number( value: i, square: i*i ) {
for ( j in 2..<i ) {
if ( i % j == 0 ) {
factor( value: j )
} } } } }
which outputs
<description>Square and factors of 10..15</description>
<number value='10' square='100'>
<factor value='2' />
<factor value='5' />
<number value='11' square='121' />
<number value='12' square='144'>
<factor value='2' />
<factor value='3' />
<factor value='4' />
<factor value='6' />
<number value='13' square='169' />
<number value='14' square='196'>
<factor value='2' />
<factor value='7' />
<number value='15' square='225'>
<factor value='3' />
<factor value='5' />
This is a compact and maintainable solution to the XML document creation problem.

The use of builders to construct complex hierarchical structures be they XML documents or Swing GUI interfaces is a very powerful facility. This shows the strength of Groovy's meta programming and its support for closures.

Next week I plan to take a few days off and build a web application with Grails and Groovy. Wish me luck.

[*] I love this phrase. So much of what is open-source is written by people either learning the language, learning the domain, or learning the patterns. The older the language or the library and the greater the likelihood that what you have in hand has had several generations of revision and revisers and is the best that it can be. SmallTalk's libraries are the epitome of this. Most new languages and libraries (Groovy included) are written by folks while they are "on the learning curve." The result is often baroque, idiosyncratic, flaky near the edge cases, etc. ¶ With that said, one of my employability facets is that I build on the learning curve all the time! ¶ I have no idea where I first heard the phrase "building on the learning curve." If you know to whom I can attribute it to please contact me.

Looking for a better web browsing history

I browse the web a lot. I see too much stuff for bookmarking either via client bookmarks or service bookmarks like and Yahoo! and so I need something else. What I have been thinking about is an augmented history. The history is a chronalogical list of sites visted. Each visit can be augmented with tags, notes, page captures (HTML+CSS+JS), and page image captures (PNG image). All of this is browsable and searchable. Does anyone know if there is such a beast? I am willing to use a FireFox extension, a local (desktop) proxy server, or a remote proxy server.

Update: A key notion I did not mention above is that I don't want to "catalog" the site during browsing but do so at some time later in the day. While browsing might not seem like a flow activity it is. Interruptions should be limited. Bookmarking is a secondary activity, at best, but it requires that I keep the activity in mind while browsing and take action at unstructured times.

Dartmouth Comprehensive's 1979 graduating class.

Dartmouth Comprehensive class of 1979I'm the one bottom row third from the right.

Forcing MySQL to use an index

Again, MySQL does stuff I don't expect. For example, the statement select * from T has very poor performance. While, the statement select * from T order by PK, where PK is the primary key, has very good performance. Since I don't care about record order both of these statements are effectively the same. MySQL's explain command says that the first statement does not use an index while the second does. Why on earth would an optimizer not use an index for the first statement? The upshot is that my DML is now scattered with extraneous order by clauses. Sigh.

Barack Obama speaking at Dr. Martin Luther King Jr.'s church in Atlanta

His hope and my hope is why I continue to live in the US.

"No Such Thing As Society"

I was born in England and lived there until 1979 except for the short 1972-1976 period. I have very fond memories of this time: The kids of Mayflower Close, playing in Old Mill woods, exploring the Britannia Royal Naval College, long walks to the castle and long bike rides to Blackpool Sands and the model railway shop in Totnes. Except for the occasional bulling I was unaffected by the deep depression that was going on in society. The memorable moments of this for me where the scenes on TV of strikers clashing with the police, the conflict in Northern Ireland, and the growing political power of the hate groups like the National Front. The only personal memory of the turmoil were the black-outs by striking electrical workers. reminded me of al this this morning with the notice of the new book No Such Thing As Society. The blurb says
"The human costs of de-industrialisation and globalisation were the great central themes of the documentary photographers active in the North of England in the late 70s and 80s. The social disasters captured in Chris Killip's work extended into the darkly coloured, claustrophobic interiors of DHSS offices photographed by Paul Graham, and Martin Parr's lividly coloured documents of holiday makers in New Brighton, Liverpool."
There are a few photos at the accompanying exhibition . The two that stood out for me were these.

The generational change shown by the suited man and the half-naked woman.

The punk more afraid of his world than frightening to it.

Patent for bagging the parts of a model by the model's sub-assemblies

The FIRST Lego League Robotics Challenge is happening tomorrow (12 Jan 2008). This got me thinking about Lego's Mindstorms, wondering if I should ask Jessica to sell me hers, seeing what the price is on eBay, Googling Lego and Mindstorms stuff, and ending up at LEGO Engineering Symposium hosted at Tuft's this week. This took me to Googling the presenters which took me to patent EP 1 011 834 B1 A method of packaging toy building elements. The patent is for bagging the parts of a model by the model's sub-assemblies. I wonder if you can patent putting on your socks before putting on your shoes? Probably not, but how is this more obvious than bagging parts by sub-assembly?

I hate to pick on Lego because as far as I am concerned their products are some of the greatest on the planet. And, as a proto-information-designer, their assembly manuals (example) are fantastic. Henry and Owen agree.

p.s. MySQL statement performance close to infinity

I have had to restart the MySQL server twice today because it was unable to recover from an unexpected termination of the Java client -- that is, me killing the Java process 'cause the MySQL stuff was taking too long. I have never had to restart PostgreSQL due to a client error. Not on Linux, Solaris, or Windows.

MySQL statement performance close to infinity

I am using MySQL for a client's project. I am not a fan of MySQL -- I much prefer PostgreSQL -- but the client asked that I use it and I am learning a lot in the process. Somethings in MySQL are great, for example the consistent use of SQL for all management, and somethings are bad, for example, performance and performance analysis tools.

For example. I have a table (Q) with 22,000,000 records that a Java process is working through in batches of 200,000. After each batch I need to delete the processed records. Since the Java process can have multiple instances a temporary table (T) is created and initialized with a batch of records and then the Java instance processes the records in the temporary table. At the end of the batch the processed records are removed from the table Q and the temporary table T is dropped. The delete query was

delete from q where id in ( select id from t )

The performance of this statement is close to infinity as far as I can tell. Don't use it. Instead use

delete from q using q, t where =

MySQL's explain does not compare to the PostgreSQL version. So, I don't know why the first statement is so bad and the second so good. It's a mystery. The good news is the client's project can now continue forward.

Juice Analytics's Chart Chooser

If you need to make a chart in Excel or PowerPoint please start with the templates offered by Juice Analytics's Chart Chooser.

What to expect in an initial web site design delivery

I am managing the web site redesign for a RI non-profit. We recently engaged a firm and received their first delivery this week. This was their initial visual design for the site and it consisted of three home page designs. There is not enough substance in this for a customer to respond too. Every customer’s web site has several different kinds of content and I expect to see in an initial presentation a few significant pages. Having these in hand gives me more confidence that the visual design will fit the needs of the site.

There are two broad categories of visual presentation for a web page -- the casing and the content. The casing is the parts of the page that are used consistently across the whole site. The part’s purpose remains the same but the content might vary. For example, mast-head, logo, copyright notice, navigation, primary content, and secondary contents. The content is the body of the page, the primary content, and the reason that the page exists at all. This visual presentation is typically a layout of several paragraphs, pull quotes, side bars, tables, and figures.

Regards the casing, there are three kinds of presentation I would like to see:
  • "Home page."
  • "Section page", for example, the partners’ page.
  • "Detail page", for example, a partner page.
Navigation also needs to be presented as part of the visual design. There is the navigation that appears in the casing and the navigation that appears in the content. There are five kinds of navigation:
  • "Parent to child", for example, partners to partner.
  • "Child to parent", for example, partner to partners.
  • "Peer to peer", for example, partner to partner.
  • "Internal link", for example, partner to an engagement.
  • "External link", for example, to RI Foundation.
If you have pointers to exemplary public examples of this please send them to me. The examples that I have belong to Dynamic Diagrams, Ingenta, and their customers.

The flip table

I want to build this coffee table!

Treehuggers's long page navigation tool

I was at Treehugger today and noticed a novel interface tool. The home page is a list of stories -- much like any blog. Not all stories are interesting to all people. And so, to the right of each story is a vertical bar that when clicked skips the page ahead to the next story. Three bars are outlined in red in the illustration. That the navigation is ever present and in the same relative location makes this a very effective means of both skipping and reviewing a long page of stories.

GameKeeper moving to "the bucket"

The GameKeeper, a game store that for so long has been in the Arcade in downcity Providence, is moving as of Monday, January 14 to the Hope Artiste Village at 1105 Main Street, Suite 2225, in Pawtucket. The GameStore's owner is a great source of information about games for all ages. Henry, Owen, and I have had a great time playing the various games he has recommended.

FireBug and memory consumption

FireBug (for FireFox) makes debugging DHTML both possible and effective. The downside is that over the course of a long debugging session it consumes all the memory on the machine. As the picture shows, FireFox/FireBug is consuming 1.5G of virtual memory but worse 1.1G of actual memory.

Mixing literate programing and wiki markup

I have always liked the ideas around Knuth's Literate Programing and today I came across the LiteratePrograms! site. This site is a MediaWiki instance enhanced to support literate programming using the Noweb syntax. Best of all, one page can be used to generate multiple source files as demonstrated for this Java rot13 implementation.