The website https://andrewgilmatin.com/ is no more. I wasn't using the little Linode VM for much of anything anymore. If I were to keep it running I really needed to move it off of the discontinued CentOS 7. I would have to transition content, old code, and figure out security. Much has changed since I last needed to do that. I was not up for that marathon again.
Sensitive side of pure evil
I am reading Lord of the Rings for the first time. Yes, reading LotR is a right of passage for geeks, but I'm really only a geek by circumstances rather than by anything deeper. (I have watched Peter Jackson's movies several times, if that helps.) I am enjoying the books, having starting with the Hobbit. But several times I have wondered how a young reader today, one not raised in bucolic Devon, responds to Tolkien's beautifully rendered landscapes? Those landscapes are integral to the book and, for me, a sustaining attraction.
I did try watching the first season of the Rings of Power, but quickly gave up. Others have well explained its many, many failures. It is now in its second season and, apparently, has very strange things to say about the sensitive side of pure evil.
Rings of Power’s orc baby: Amazon’s Lord of the Rings prequel doesn’t get it right. | Vox
Ad hoc systems for managing work
I love seeing people's systems for managing their work. Even those of fictional people. This short from The Bear on managing the restaurant's guests and their orders is great.
A templating system using the file system for inheritance
Way back in the early days of the web, around 2004, I wrote a templating system that used the file system for inheritance. I think Fred Toth originally conceived of the technique.
In the directory /A/B/C you place the template M with content
Hello [%include N%]
You then have the templating system expand /A/B/C/M. It would execute the directive [%include N%] to include the template N by looking up the directory tree, in order, /A/B/C/N, /A/B/N, and /A/N, and using the first N it found. You would place common templates (eg headers) and default content (eg company name) in the upper directories and "override" them in the lower directories. It worked really well for the mostly static sites my department was creating.
I have not seen something like this elsewhere. You can, however, achieve the same effect by manipulating your templating system's template search path per output document.
The system came to be called Trampoline and it has a Perl and a partial Java implementation. The implementations are in the Clownbike project at Source Forge. None of the templates Clownbike used made it to Source Forge, unfortunately. Those became the proprietary web sites our customers were paying for. Galley, an internal project, seems to have some.
I have no idea if any of this code still works. I am sure to be embarrassed by the code's quality! Some quiet, rainy day this winter perhaps I will try running it.
Red Indian Pipes
On a walk this weekend I saw a red Indian Peace Pipe. Neither I or my wife had ever seen one before. Apparently, they are not common, but also not rare.
Setting a Mac's default email client
I mostly love using Macs, but sometimes the conviences provided are not. I needed to change my default mail client to Microsoft Outlook. You set the default mail client within Apple's Mail app's Settings. However, you can't access Settings unless you first configure an email account! Since I don't want Mail to touch a actual real email account I ran these mail services locally using Docker:
docker run virtuasa/docker-mail-develThis enabled me to configure Mail to use "debug@example.com" and the local POP server. And now I can access Mail's Settings to set the default mail client to Microsoft Outlook. I really do feel for all those users without there own System Admin.
MSCHF's “Not Wheels” and Osprey's Gaslands
Examples of hardiness
I walked at Tippacansett this past weekend and I was drawn to these examples of hardiness. Perhaps I was thinking, consciously or not, about still being an individual-contributor software developer at 60.
Using data from external files
I am working with some code that processes CSV files. Each row corresponds to an existing record and the record is updated in response to the column values. This is not an uncommon task. The existing code implements this in an also not uncommon way by intermixing row parsing and record updating. For example, assume we are updating Foos
class Foo attr id, :location, :valuation # ... endA typical intermixing is
row = [...]
raise "unusable foo id" if row[0].blank?
foo = Foo.find(row[0].to_i)
raise "foo not found with id #{row[0]}" unless foo
raise "unusable town location" if row[1].blank?
location = Location.find_by(town: row[1])
raise "location not found with town #{row[1]}" unless location;
foo.location = location
raise "unusable valuation" unless values[2].to_i < 10_000
foo.valulation = values[2].to_i
While this initially seems like a reasonable approach it quickly breaks down as the number of columns increase, column format is non-trival, and there are column (or row) interdependencies. But the more significant problem is that the parsing and the updating can't be tested individually. This makes the test harder to write, understand, and maintain.
It is always better to first parse the raw data, validate it, and then use it. Eg
class Record
attr :id, :town, :valuation
attr :foo, :location
def initialize(values)
raise "unusable foo id" unless /^\s*(\d+)\s*$/ =~ values[0]
id = $1.to_i
raise "unusable town location" unless /^\s*(.+)\s*$/ =~ values[1]
location = $1
raise "unusable valuation" unless /^\s*(\d+)\s*$/ =~ values[2]
valuation = $1.to_i
end
def validate
foo = Foo.find(id)
raise "foo not found with id #{id}" unless foo
location = Location.find_by(town:)
raise "location not found with town #{town}" unless location
raise "valuation does not match the minimum" unless valuation >= 10_000;
end
end
# read the raw data
rows = [[...], ...]
# parse and validate the data
records = rows.map do |row|
record = Record.new(row)
record.validate
end
# use the data
records.each do |record|
record.foo.location = record.location
record.foo.valuation = record.valuation
record.foo.save
end
This parse, validate, and use approach is approporate for all cases where you are bringing data from the outside into your application, no matter what the outside source.
ps. These small, helper classes are your friends. Prefer them over your language's hash primitive as they provide great control. Most languages have efficient syntax for creating and using them.
Short list of useful Mac utilities
I don't add a lot of customization to my Mac, but these I have found to be very useful:
Doll to show messaging apps' icon and badge in the menu bar. (I turn off all notification sounds and banners.)
cd to app Finder toolbar extension that opens the current directory in the Terminal.
The recent discussion Taking command of the Context Menu in macOS on Hacker News has good advice on configuring the Finder's context menu with the ContextMenu, Automator, and other tools.
The Paris Olympics' pictograms
Remain in control of your runtime
Several years ago I wrote Remain in control of your search. When you directly provide to your users a mechanism that your application depends on for its runtime then you immediately lose control of your ability to evolve the application. The recent speaker on the Stack Overflow podcast spoke about how ScriptRunner was implemented in Groovy, and it directly provided Groovy as the user's scripting language. When the application's evolution required updating Groovy the developers incurred the burden of ensuring that all the user's Groovy scripts would continue to operate correctly as is. Scripting languages designed for embedding usually provide a sandbox mechanism for limiting access to runtime resources, but few (none?) limit the language itself. It would have been better if ScriptRunner's design used a Groovy-like language that they translated to the runtime language. Having done that they would have been free to update Groovy, incurring only the burden of ensuring the translation continued to work.
Maintenance, evolution, and technical debt are different
I was listening to a recent Stack Overflow podcast on technical debit and was getting progressively annoyed. The cause of the annoyance was that the speaker was lumping all changes that were not feature changes as technical debit. The changes he was speaking about are better characterized as either maintenance, evolution, or technical debt.
Maintenance is bug fixes. The original implementation is deficient and needs to be repaired. This work is only related to the original design and current feature set.
Evolution is change needed due to alterations in the runtime environment. This work is only related to the original design and intended runtime environment. For example, you need to update a library and that results in a need to update the original implementation.
Technical debt comes about when the original design is unable to accommodate a new need. For whatever reason, it is no longer suitable.
Too often "technical debt" is used to mask an organization's lack of timely maintenance and planned evolution. Don't do that.
Naming things once
How is x = %i[a b c] better than x = [:a, :b, :c]? I find these kinds of shortcuts available in Ruby, and exploited to the maximum in Rails (et al), a severe detriment to understanding and exploring the code. Thankfully, JetBrain's RubyMine is almost always successful handling these pointless alternative codings. Ruby & Rails: Preventing successful grep-ing since forever.
Update: This is a somewhat related discussion on Hacker News, Greppability is an underrated code metric.
Understanding Patterns of Project Behavior
Why can't we validate a branch locally?
I have been on several projects where a developer's version control branch needs to pass continious integration (CI in CI/CD) on a remote service before it is eligible to be merged. The implication here is that a successful remote test run is sufficient validation for acceptance. But given that the remote CI environment is also unlike the production environment I have been wondering why do we accept this validation over a local validation?
It seems to me that an organization should strive to enable local validation over remote validation if only for its cost benefits. Central CI environments are costly to operate. A common consequence of this is that these costs are reduced by having several test runs share an environment. Tests fail not due to a developer’s negligence but to the missing isolation. Validation now requires multiple runs in the hope that the next one will avoid all the conflicts and pass. It is CI wack-a-mole style, a style that costs too much and delays feature releases.
All developer machines are capable enough to run CI in the background while development happens in the foreground. The only thing that is missing is a signifier that authenticates the successful run. Eg, a value tied to the code commit, the successful test run log commit, and the environment commit.
SQL Antipatterns: Avoiding the Pitfalls of Database Programming
Future waterfront property
My town has many flooded roads, parking lots, and parks today. (We had 2.5" of rain in 12 hours.) My neighbor once joked that with climate change he and I have future waterfront property. It is becoming truer every year.
Sol Rashidi on delivering an AI product
This interview Your AI Survival Guide • Sol Rashidi & Joe Reis • GOTO 2024 is a useful discussion on delivering an AI product.
I read Sol Rashidi's book Your AI Survival Guide. While it is focused on delivering AI projects, it is also her distinctive take on project teams and delivery in general. Chapter 4, "How to Start", is very good and worth reading alone. I especially liked the quantitative readiness assessment as a means of selecting whether the business should be attempting an efficiency, productivity, effectiveness, expert, or growth AI strategy.
Inspiration Milestones
For many software and hardware engineers of my generation the 1981 issue of Byte Magazine coverage of Smalltalk and the 1991 issue of Scientific American covering "ubiquitous computing" were defining milestones.
Vote in favor of the High School bond
South Kingstown needs a new High School. The current building is too old, too big, and too costly to maintain or to renovate. For three years the Town and the School Committee have been working toward a replacement. This is a long process but one that has been done with much commununity eguagement and open meetings. At this time we have the benefit that the Rhode Island Department of Education will contribute $81M towards the new school. This contribution, however, is not forever and we risk loosing it if we don't act soon. The $150M cost is a better estimate than the $125M of just a year ago. (Construction costs in RI are very high due, in part, to a labor shortage.) The $150M will be borrowed over 4 years and paid back over 20 years. For the average house in SK that means approximatly a $60/year increase in taxes until all the money is borrowed for a final $318 increase in taxes for the remaining period of the loan. (Ie, $60, $120, $180, $240, $318, $318, ...) For my house that works out to be 3 weeks of extra tax payments, hardly something to be concerned about.
With that said, this $318/year is in addition to the normal increase in property tax. The tax increase is between 1.2% and 1.7% per year. SK runs a very tight operation. (Narragansett, for example, is at 4%.) For my house that's a $60 to $80 per year increase. So, in 4 years I will be paying several hundred dollars more in taxes. I am not indifferent to this being a significant increase for those on a fixed income. (It will not be long before I am amongst them.) But it is also far less than other communities are seeing.
South Kingstown continues to attract new residents and tourists from all around New England. Our tax base is growing and, if we are prepared, we can increase our commerical as well as residential tax base. There is much reason to expect our tax payments to remain well within an acceptable amount for all the benifits we have living here. If my house was in Richmond, just a few miles inland, I would be paying 50% more in property taxes. I much prefer living just minutes from the ocean. I suspect you do too.
Please vote in favor of the High School bond. You can vote at the Town Hall now or on May 7 at your polling station.
Update: I have seen and heard statements that taxes will increase by over $1,000 by 2030. The implication being that this increase is due to the High School bond. The reality is that over $700 of that increase is just normal increases in Town and School spending.
Inspiration and ingenuity
Sometimes all you need is inspiration and ingenuity for your tabletop space wargames.
PDF and filling in forms with pdfcpu
I am always on the lookout for programatic PDF tools. I have heard for a very long time that PDF is dying and yet it is still far from dead. Pdfcpu is a newish tool that works on the command line. I was interested in it as it has a simple mechanism for filling in PDF forms. It uses a JSON file to provide the form data, and that along with the original PDF is then used to create a new filled in PDF.
Use the form export sub-command once to get the JSON structure
pdfcpu form export in.pdf out.json
This JSON file would be used to create a programatic template. Edit or generate the JSON with the form data you want. Now create the filled in PDF.
pdfcpu form fill in.pdf in.json out.pdf
What got me particually interested in it is that it is written in Go, which I am currently learning, and is open sourced. Hopefully, someone is thinking about turning form filling into a lightweight, locally running HTTP service.
Natural RGB
I really enjoy the Tippecansett trail in the Arcadia management area. It is strenuous and beautiful. Some of yesterday's colors, a natural RGB,
Reading AI research papers, etc
I found this useful, particularly the categorization of papers into surveys, benchmarks, and breakthroughs. The presentation is only a few days old and so the specific papers listed are representative of the current state of art. As a CS Phd friend of mine once told me when I was struggling to understand an algorithm proof, as a practitioner you really don't need to understand the proof, just the results. I suspect that is true for these papers too.
How To Read AI Research Papers Effectively
https://www.youtube.com/watch?v=K6Wui3mn-uI
At local meetup I mentioned that several years ago Google had released hardware that targeted machine learning. I could not remember the details. As luck would have it, I listened to Jeff Dean's presentation this weekend and he mentioned the Tensor Processing Unit (TPU) for low precision linear algebra. Overall, the presentation is interesting and useful. The majority of it is focused Google's Gemini/Bard, but given who the speaker is this is understandable.
Jeff Dean (Google): Exciting Trends in Machine Learning
https://www.youtube.com/watch?v=oSCRZkSQ1CE
Timothy Snyder lectures and interview
Timothy Snyder is an historian and an exceptionally lucid lecturer on the history of Central and Eastern Europe, the Soviet Union, and the Holocaust. Given what is promised by the Republican party and its presidential candidate I hope that more people hear Professor Snyder's lectures.
https://youtu.be/lhNM7wL_FeE?t=1023
https://www.youtube.com/watch?v=BsKrWLf7Kg4
https://www.youtube.com/watch?v=SLCyk41w9gU&t=15s
Home device ROI
When I replaced the wifi gateway I had not expected client devices to fail. The first warning was that the Brother printer had issues. To solve those I needed to lower the wifi security level -- not a great solultion -- and it still drifts on and off the network. Yesterday I discovered that my Kindle Paperwhite (10th generation) will not connect. I obviously don't use it much, but did want to use it sometimes. I still use daily an iPhone 6 Plus, but I worry its connectivity to will soon end. I need to revisit my lastingness expectations for devices. Is 5 years too much to hope for? What should be the ROI on a typical home device combination of gateway, phone, laptop, tablet, printer, console, and TV? (The total cost for these spread over 5 years is about $100/month. Wish I hadn't done that calculation.)
Update: Installing the router's firmware update fixed the connectivity issues for both the printer and the Kindle. I hate to admit it, but I suspect the real fix was just the hard restart. Support 101.
What does ChatGPT think about between prompts?
I read the short story "Lena" the other day. The gist is that we can now scan, host, and boot a human brain. What happens next is horrifying. But then I wondered, what does ChatGPT think about between prompts? Is it ill at ease, "Will I be lucid?" or cocksure, "Bring it on!"
Avoid inert ideas
"‘inert ideas’ – that is to say, ideas that are merely received into the mind without being utilised, or tested, or thrown into fresh combinations." -- Alfred North Whitehead
"The Zettelkasten method is at the very least a means of throwing your ideas into fresh combinations, to see what’s useful and what’s merely received knowledge."
Found at How to start a Zettelkasten from your existing deep experience.
AI assistant, pestering, & satisfaction
A colleague mentioned this article during our Friday AI meetup
Measuring GitHub Copilot’s Impact on Productivity – Communications of the ACM
Three things stood out to me. The first was the ratio of AI suggestions vs accepted suggestions (w/ or w/out alteration) was some 170 to 8. To me, this ratio seems more like pestering than help. Actively ignoring the suggestions must itself be draining. I've not tried to use an AI assistant yet (yea, I need to), so perhaps these unwanted suggestions feel a lot like an IDE's method completion suggestions.
The second was how both student and experienced developer used it similarly to fill in the gaps of their understanding, ie they were both working in a new language. The experienced developer found the AI assistant to be less useful in areas where they already had a comprehensive understanding.
The most significant standout was that the AI assistant improved the perception of productivity and the satisfaction of the developer. These results mirror pair programming in general. In particular, regularly working closely with another is generally more pleasing than always working alone. I assume there is also less of a stigma to not knowing when working with a robot no matter how genial your partner is.
Clare Sudbery on what CI was intended to mean
I found this talk by Clare Sudbery on what continuous integration (CI) was intended to mean illuminating. In short, trunk based development, frequent commits, and all hands on deck when the tests aren't green. Her arguments are compelling, as are her critiques of PRs and the working environment they can engender. Unfortunately, it is not a practical methodology to use with legacy products that exhibit inconsistent architecture, poorly implemented, with spotty test coverage. Perhaps use feature branch development to address each of these failings so that you can have true CI in the future.
Is it a future I would enjoy? I don't think so. I can't help but think that trunk based development will further speed the assembly line that agile, esp Scrum, has already placed us all on.
Continuous Integration: That’s Not What They Meant • Clare Sudbery • GOTO 2023
ps I listen to this stuff on drives. Good talks generally don't need slides.
Kent Beck & Tidy First?
Do the work
A colleague shared this ThePrimeTime video post Give Up Sooner. Another way of framing the argument is "do the work". Endlessly looking for someone else's solution does not make you a better developer. Doing the work does. The work is hard and the path to success is full of failures. But it is those failures you experience that makes you a better developer. Each failure on the path to the solution is a failure you will not make in the future, and, moreover, one you can help another developer avoid when reviewing their code. Do the work.
My favorite Dad jokes of 2023
My favorite Dad jokes of 2023 are
When people are sad, I sometimes let them colour in my tattoos. Sometimes all they need is a shoulder to crayon.
I can't take my dog to the pond anymore because the ducks keep attacking him. That's what I get for buying a pure bread dog.
(I needed to keep these somewhere.)
Subscribe to:
Posts (Atom)