Little need for fully sentient beings

In the Winter of 2001 the US Navy began the re-development of the DD-21 destroyer class warship after Congress slashed the original budget. The new ship class DD(X) and named after Admiral Zumwalt was to be the Navy’s first fully integrated command and control (C2) vessel featuring a small crew, multipurpose vertical launch tubes, large flight deck, and with enough electric generation capacity to power small cities or the anticipated kinetic and laser weapons.

What was not divulged at the time was that the Zumwalt’s C2 left little need for fully sentient beings to operate the ship. It has been learned that a black budget division within the Navy’s Marine Mammal Program has been replacing the crew of the DDG-1000, currently at sea in the San Diego, CA area, with mollusks and cephalopods. Mollusks, due to their limited intelligence, (newly hardened) protective shells, and stationary nature, have been integrated into the ship-wide computing networks and are now responsible for intermediating with all sensors and actuators — from missile launch tubes to freshwater valves.

How far along the chain of command has been replaced with coleoidea is unclear. There is strong circumstantial evidence that the ship's captain is a human-octopus hybrid. The recent waterproofing polyethylene application to the inside walls of the ship’s bridge and mission control centers suggest that the hybridization is more octopus than human.

More news to come.

Furthermore on dashboards for maintenance developers

What would a dashboard for visualizing and managing the flow of messages between actors and their dependents look like? My skills run in the direction of information architecture and not visual design, unfortunately. I can tell you what should be on the dashboard, the relationships between the contents  controls, the visual and interactive priorities of these, and how all of this changes over time. On a Friday night, this is not something I am prepared to do; which is really an excuse as I am nearing the end of my interest in this thought experiment. Nevertheless, here are a few ideas.

The transitions of a message between the actor and the dependent are

- Actor sends the message to Dependent, or fails to send it.

- Dependent receives the message, or fails to receive it.

- Dependent responds with a response-message to the received-message, or fails to respond.

- Actor receives the response-message, or fails to receive it.

We can symbolize these statuses as

A display for showing details of actors, messages, and listeners (dependents) might be

Each message is given a row and the delivery and response status is detailed for each listener. My bare tabular example would have much more detail on it in practice. This is not a overview display and so concerns with overwhelming the user with too much detail is not of relevance. If you only have to show a few listeners and perhaps a brief time window of messages this display will quickly show specific problem with delivery and response. 

Inverting and aggregating the above example leads to the tighter display

We are, again, showing each message, but the columns now indicate how many listeners are in each of the delivery or response states.

Lastly, as everyone loves a set of small multiples graphs here is 40 minutes of activity and status where the green line is the number of listeners, the black lines show positive trends and the red negative ones. 

Man, that is an ugly chart (and is playing loose with small multiples).

More on dashboards for maintenance developers

Yesterday I wrote about providing tools to visualize & manage the internal workings of applications for maintenance developers. That note was long on advocacy and sort on detail. This note somewhere between those two.

We are all familiar with run-time debuggers. We can get our job done with gdb [1], but for day in and day out work a visual debugger is a must have. A debugger is a point-in-time dashboard of the execution of the programming language used to implement the application. It is not a dashboard of the application, it does not directly represent the application's data or processing models. Nor does it provide a history of where the application was or a prediction of where it will be heading too next. Nevertheless, it is a useful analogous image of the application dashboard to keep in your mind.

If, for example, your application relies on the observer pattern [2] then you will need to visualize & manage the actors (observed), their dependents (observers), and the messages. The effort to implement this dashboard can be significant, which is the exact opposite of implementing the pattern itself. The pattern is often implemented as a list of callbacks. This implementation is so insignificant that your high performing developers probably choose not to use a framework at all, but to re-implement for each use. And there is the root of the trouble ahead for the maintenance developers, there is no one place to watch this running application. Callbacks and messages are being created and dispatched all over the place.

Suppose there had been a reusable implementation. The management of the listeners was done in a single place. The dispatching of messages to the listeners was done in a single place. Each reuse required that the developer explain the purpose, configuration, and expectations of this reuse. For example, "Status of the nightly recalculations of base prices: the messages are life cycle events; messages are ticketed and numbered for correlation; the listeners can be local or remote; the listeners are contacted asynchronously; no receipt acknowledgement is wanted, but when given will be logged; expect a dozen messages over a 2 hour period in every 24 hour period to be sent to a few hundred listeners." When this pattern is visualized on the dashboard not only is the context clear, but timelines of planned to actual and switches to toggle synchronicity will automatically be included. And what if you could dynamically toggle receipt acknowledgement? How many of your application's ailments are the result of lost messages sent to remote listeners?

The rise of DevOps has brought with it both tools and new (renewed?) appreciation for visualizing processes and their artifacts. Lets keep on pushing these ever deeper into the application itself.

[1] gdb, https://www.gnu.org/software/gdb/
[2] Observer pattern, https://en.wikipedia.org/wiki/Observer_pattern

We all want visibility into a running application

We all want visibility into a running application. The need becomes acute when the application is ailing and that visible schedule, map, and cartogram of the data and processing will swiftly guide you to the failing part.

What is less recognized is that these same visuals are needed in maintenance. Your original, high performing developers are unlikely to be around after the application has been in production for a few years. They will have moved on to new green pastures. If you are lucky and they are still with the company then they will be somewhat available to the maintenance developers. If a little cantankerous about that.

The maintenance developer usually has little broad knowledge of the application’s operation. Her first task at resolving an application’s ailment will be to find suitable entrance and exit points. This is not an easy task for our applications now run continuously, across multiple processes and hosts, and are highly asynchronous. Hopefully, those high performing developers adhered to standards, used a common framework throughout, and kept to best-of-class external libraries and supporting tools. If they did then she might have a straightforward resolution path. In all likelihood, however, those high performing developers probably invented too much and documented too little. [I speak from the experience of my younger self.]

What the maintain crew needs is not logs alone. They need meta tools that connect to the running application and visually show its operation. Once connected the tool must be able to slow down the application to a human pace. Restrict it to one active user at a time. It might even need to switch it to run synchronously.

These are not easy features to implement, but the cost of not doing it is not just the ever lengthening backlog of ailments to fix, but the erosion of customer confidence and, in the end, loss of customers.

All modern applications have three users. The first is the customer. The second is the customer support. The third is the maintenance crew. It is way past time that the maintenance developer is provided the meta tools needed to look inside the running application and keep it fit for market.

Warhammer 40k: I don't really like the game

I have enjoyed my time with Warhammer 40k. The Games Workshop plastic models are very well produced and a joy to assemble and paint. The background lore is deep and some of its retelling is masterfully done in the Black Library publications. I particularly like the Horus Heresy stories even if they are set 10,000 years before the world I am playing in. Unfortunately, I don't really like the game for the simple reason that there are far too many stats. There are stats from the individual weapons, to figures, to units, and so on all the way up to the whole army. This makes for a great variety of play as every game truly is a unique combinatorial experience. But, I can't play a game like that. I don't have the inclination or, frankly, the time to dedicate to learning not only my faction's specifics, but also those of my opponent's army. An opponent whose army can be made up from many dozens of races.

I have decided to keep the figures. I had intended on selling them, but, the figures are very nice and perhaps I will play the One Page Grimdark Future Rules one day.

A top software development position in a startup wants ...

Having just taken a job with Raytheon and lamenting my lack of success getting a top software development position in a startup a friend asked "What is needed for new startups?"

A top software development position in a startup wants several years of experience with AWS infrastructure and a comprehensive knowledge of JavaScript use (and packaging) in the backend (node.js) and the frontend (Angular, React, and sometimes Vue). The startup still want the other skills of scalable architecture design (processing and data), excellent written and spoken communication, and mentoring, but having these is not enough. I was also surprised that even startups that will hire remote staff still want that staff close to home base.

As to the Java language and its ecology, this has little interest for them. In fact, many of the infrastructure foundations that are today implemented in Java -- ActiveMQ, Kafka, Zookeeper, Tomcat, etc -- are being replaced with "lighter" Go and Rust implementations. And with the accelerating move to managed infrastructure startups care little about how these foundations are implemented. They would rather rent, eg, a queuing service from AWS or GCP, and let the provider worry about implementing turnkey, scalable performance.

Even Citizens Bank, after having recently become independent of the Royal Bank of Scotland, is replacing their Java J2EE implementation with an AWS hosted, node.js microserviced, and React front ends. I have even heard that some of their services are from a "bank in a box" supplier.

I don't disagree with these changes except for the unfounded confidence in JavaScript -- the one language to rule them all. I would not want to run my own data center and infrastructure anymore, either. Assuming, that is, I had the very sizable budget for it.

Unwelcome advertisement for throat lozenges

The content on YouTube is really tremendous. I have used it to watch tutorials for fixing a food mixer, replacing my minivan's factory radio, painting miniatures vikings and ultramarines, learning boardgames, and refreshing some math theory. I have used it to watch documentaries and fan-made movies. I have listened to audio books. Sometimes, I even listen to music and other forms of audio & video performances. Unfortunately, the interstitial advertisements are beginning to kill YouTube.

I accept the need for YouTube to generate revenue with advertising, and, as crazy as it sounds, if the ad is short, I let it play through rather than take action to skip it. What I can't accept is the jarring interruption of the interstitial ads. You are listening to a 7 minute recording of a live performance. The performance beautifully builds for the first 3 or 4 minutes and then, just when it reaches a crescendo, an ad for throat lozenges breaks into the middle and ruins the whole thing.

I will continue to use YouTube as it has so much content useful in my daily life. I am going to skip most everything else until they replace the interstitial ads.

Phoenix Checklist

[Copied from Boing Boing.]

The "Phoenix Checklist" is a set of questions developed by the CIA to define and think about a problem, and how to develop a solution.

The Problem

• Why is it necessary to solve the problem?
• What benefits will you receive by solving the problem?
• What is the unknown?
• What is it you don’t yet understand?
• What is the information you have?
• What isn’t the problem?
• Is the information sufficient? Or is it insufficient? Or redundant? Or contradictory?
• Should you draw a diagram of the problem? A figure?
• Where are the boundaries of the problem?
• Can you separate the various parts of the problem? Can you write them down? What are the relationships of the parts of the problem? What are the constants of the problem?
• Have you seen this problem before?
• Have you seen this problem in a slightly different form? Do you know a related problem?
• Try to think of a familiar problem having the same or a similar unknown
• Suppose you find a problem related to yours that has already been solved. Can you use it? Can you use its method?
• Can you restate your problem? How many different ways can you restate it? More general? More specific? Can the rules be changed?
• What are the best, worst and most probable cases you can imagine?

The Plan

• Can you solve the whole problem? Part of the problem?
• What would you like the resolution to be? Can you picture it?
• How much of the unknown can you determine?
• Can you derive something useful from the information you have?
• Have you used all the information?
• Have you taken into account all essential notions in the problem?
• Can you separate the steps in the problem-solving process? Can you determine the correctness of each step?
• What creative thinking techniques can you use to generate ideas? How many different techniques?
• Can you see the result? How many different kinds of results can you see?
• How many different ways have you tried to solve the problem?
• What have others done?
• Can you intuit the solution? Can you check the result?
• What should be done? How should it be done?
• Where should it be done?
• When should it be done?
• Who should do it?
• What do you need to do at this time?
• Who will be responsible for what?
• Can you use this problem to solve some other problem?
• What is the unique set of qualities that makes this problem what it is and none other?
• What milestones can best mark your progress?
• How will you know when you are successful?

From the book, Simply Brilliant: Powerful Techniques to Unlock Your Creativity and Spark New Ideas, by Bernhard Schroeder

The splendor and complexity of networked service at scale

The splendor and complexity of networked service at scale. Josh Evans' talk Mastering Chaos - A Netflix Guide to Microservices at InfoQ.

Why and how to give a conference talk

I really liked Emily Dunham's short talk on why and how to give a conference talk.

"Contempt isn't cool"

I really like Benno Rice's talk "The Tragedy of systemd" about accepting change. I had resisted systemd only because I didn't need anything different than inet.d for what I was doing. This was a mistake and one I wish I had rectified sooner. I was missing out on taking advantage of all the other great services systemd offered. Had I switched earlier I would have enjoyed the slow, methodical accrual of a deeper knowledge and practical experience.

The link above is to a point near the end of the talk to the "Contempt isn't cool" slide. Accepting change requires to accept ones own limitations, biases, and blindspots. Do this respectfully as others have done it with you. Contempt is a destructiveness that you never want to bring into your team.

Waste Wars

I was reading the New York Times last year and came across an article about the hazardous working conditions of private waste management companies. The work is hard, the pay too little, and the time pressures too great. I don't mean to make little of this horrible situation, but, later, my mind wandered, as it often does, to Terry Gilliam’s movie Brazil and the gorilla plumber. I have always been intrigued by the plumber’s statement that plumbing was the last radical trade in the near distant future. What if garbage collection was too? Could its work be made into a game?

"The year 2017 saw the rise of the litteral cut-throat waste management companies. Their goal was to grow their business by any means possible. By 2027 waste management had become paramilitarised." Waste Wars

I never completed the game development. What I have developed is more, I think, a source of ideas for an actual game developer or for repurposing an existing game.

Gluing Warhammer 40k Space Marines

Started to glue together my Warhammer 40,000 Space Marine models. I am not fond of the aesthetic of their uniforms, but the modelmakers have made good use of it whereby each successive layer can be used used to hide the mistakes made underneath.

I don't know if other hobbyists do this, but after assembly I give all the surfaces where I scraped off the mold lines a light wash of plastic glue, aka acetone. For these models this seems to further help hide the mold lines. Will know more once they are primed.

Update: Is it just mine or are all 40k space marines right handed and turning towards the left?

Update: I should not have put them on bases and attached the guns before painting them!

Update: Awaiting the enemy ...

Fiskars Easy Change Fabric Knife

The Cool Tools blog posted a review of several alternatives to the ubiquitous, badly design, barrel handle X-Acto knives. The Fiskars Easy Change Knife is the favorite and so I bought two via the posting's associates link. They both have NO grip on the blade and are dangerous. Do not buy that version. There seem to be 2 versions of the knife in the world.

I contacted Fiskars and they sent me two replacements that are awesome! The replacements are Fiskars Easy Change Fabric Knife (3 blades) 164010-1001. The only visible difference between the two is that the Fabric Knife's has a gray tinted, translucent cap (rather than untinted). I hesitate to provide a product link as none that I found show the packaging. If you do find a package image it should look like the image in this posting.

Thank you Fiskars for great customer service.

Installing the stupid filter

I enjoy Seth Godin's blog. The entry today is Installing the stupid filter which is about how humans don't always accept questions or directions as stated. That is, humans ask "Are you sure?" Machines don't.

The problem is that machines are given bad data all the time and most accept it verbatim. Crossref, my employer until Jan 11, handles lots of XML encoded data. So we need to manage both complicated structures and many types of data -- publication dates, personal names, country names, company names, page numbers, volume numbers, ORCID iDs, ISBNs, ISSNs, Pub Med ids, etc. Some types have a strict syntax and so we can know if the value is valid. What we can't know is whether the value is appropriate. We have to guess.

Is a publication date 2 months from now appropriate? In most cases, the answer is "yes" as the publisher is depositing the metadata for a forthcoming publication. But what about 3 years from now? If the publication is part of a book set that is expected to take 10 years to complete publication, then "yes," too. If it is a journal article then almost certainly the answer is "no." At what point is an article title too long? Is it, as we have experienced, a misplaced abstract? It seems the more data we have the more the questions we have about it.

I don't have any answers for these questions. I just want to make the comment that even in machine to machine data exchanges there needs to made sanity checks on the data and those checks have to be within the larger context of each datum.

BitBar shout out

I use the MacOS menu bar tool BitBar for a few uses. I have the time in UTC and the status of a few servers. Behind these presentations are very simple bash or perl scripts. I was reading today of a small utility to show how many days remained until a set of events. The BitBar equivalent is I will leave removing past events from the output to the reader.

Craig Forbes' Tuckbox Generator

I wanted a simple and inexpensive way to store my X Wing miniatures. In the process of looking at how others had solved this problem I found Craig Forbes' Tuckbox Generator. Use this to create custom boxes for each miniature. See microhangers. Having many little boxes is too fiddly for my general use, but I am very glad someone wrote a bit of software to create box templates.

Tevo Tarantula 3D printer is coming together

The Tevo Tarantula 3D printer is coming together. So far I have only mixed up two support beams and stripped the treads on one stepper motor. Luckily, switching the beams around did not involve replacing a belt. And I was able to use longer M3 bolts to reach the stepper motor's bottom threads that were still intact. (Thank you Jerry's Hardware!) Now I need ferrules and fork wire terminals and, as far as I know, there is no store in RI that sells these!

Update, 2018-11-27: Jerry's and West Marine both sell fork terminals.

Make visible residence hall problems and resolutions

Letter to the University of Massachusetts Boston Chancellor:

The Boston Globe article "Falling elevators, raw hamburger, lax security at UMass Boston dorms" is a serious check on the trust, confidence, and enthusiasm I have for my son's continuing attendance at UMB. I do not expect new facilities to be without flaws. Nevertheless, a step to regaining the community's trust in the joint venture between UMB, Capstone, and Sodexo is to make public all the service and repair work orders, actions toward, resolutions of, and all other timeline details. Collecting and presenting this information every 24 hours would allow us all to see the extent of the problems and the pace of resolution. Let us see that Capstone and Sodexo put the health and safety of the students over profits and that UMB is a good steward.

Your truly,
Andrew Gilmartin

The debugging details you leave behind

One of the worst information losses of leaving a place you have worked for nine years is all debugging details you leave behind in the bug tracking system.