The whole Long Now seminar series is very good. If you have iTunes and an iPod just add their pod cast to your iTunes library. Note that the lecture's audio is free but video requires membership. Here are specific lectures I keep coming back to
Mary Catherine Bateson
Live Longer, Think Longer
Jesse Schell
Visions of the Gamepocalypse
Geoffrey West
"What's with the moribund twitter bio?"
Eliel asked "What's with the moribund twitter bio?" I am thinking that it must have been a "can't get anything right" kind of a day when I wrote that. But in all honesty, I am not a natural father and so find myself consciously working at it everyday and the same for marriage (perhaps every other day here). And, how many times have I been granted but never exercised stock options? I am a very optimistic guy. While everyone else (ok, many) are grasping for success I am happy working at being a little more whole one day at a time.
Keeping power and maximizing shareholder value
In the 1970s our US government representatives stopped caring about their constituents and started caring only about keeping their own job and its access to power. The corporations didn't initiate this change. They are, however, proficient at using the change. Corporations are just doing what corporations always have done and that is to maximize shareholder value by aiding representatives to keep their jobs. I hate to deliver bad news but y'all are the shareholders -- via pensions and other retirement vehicles -- and so are benefiting.
If you want to help take away the power of money in politics then support (Lawrence Lessig’s) Fix Congress First.
Facebook trapped in MySQL ‘fate worse than death’ — Cloud Computing News
Facebook trapped in MySQL ‘fate worse than death’ — Cloud Computing News
Mr Harris: When you attribute statements like
Mr Harris: When you attribute statements like
"Stonebraker said the problem with MySQL and other SQL databases is that they consume too many resources for overhead tasks (e.g., maintaining ACID compliance and handling multithreading) and relatively few on actually finding and serving data."include references to the hard proof. Pushing logic from the repository to the application does not make anything faster or slower. Cumulative work is just shuffled around. Moving logic from server to client can improve the perceived speed as you have effectively added massively to CPU and RAM capacity. So, if you have a relational data model then use SQL and a relational repository. If your data model is different then use a custom repository. The different NoSQL repositories fill this custom need. Just as, for example, FOCUS fitted the custom need for hierarchical data in the 80s! As to web scale issues, the solutions are all the same no matter how your repository models data.
I need to improve my writing
I need to improve my writing. I have much to say but find myself inarticulate at the keyboard or pad of paper. What I write unfolds from sentence to sentence. There is much backtracking to apply structure afterwards. Time to read some good, short journalism and advocacy pieces and digram them. I will start with Ellen Liberman's The Reporter column.
How to Rethrow an Exception Without Wrapping them - Java Wiki
Nicely said at How to Rethrow an Exception Without Wrapping them
Some do's and don't's with exceptions
1. Unless you have a very good reason to catch an exception, don't catch it.
2. If you can correct the problem implied by the exception. eat the exception, because you fixed it. However there are some exceptions that it is unwise to catch.
3. If you can provide additional information about the exception. catch the exception, and re-throw it as an inner exception with more information. This is a very good reason to catch an exception, but note that we are still re-throwing it.
4. Always try to catch specific exceptions. Avoid catching System.Exception whenever possible.
Some do's and don't's with exceptions
1. Unless you have a very good reason to catch an exception, don't catch it.
2. If you can correct the problem implied by the exception. eat the exception, because you fixed it. However there are some exceptions that it is unwise to catch.
3. If you can provide additional information about the exception. catch the exception, and re-throw it as an inner exception with more information. This is a very good reason to catch an exception, but note that we are still re-throwing it.
4. Always try to catch specific exceptions. Avoid catching System.Exception whenever possible.
Comment regards to "An importance of wrapping variables"
The comment was made regards to the blog posting An importance of wrapping variables. A useful feature to add to any logging tool is to allow the formatting of the log message. For example,
Doing this has three key advantages. The first is that the ease of adding relevant data to the message is trivial. This then encourages messages that truly aid monitoring and debugging. The second is that there is a very low performance cost to logging in that if the message is never used (due to the log level) then the only cost to making the call is loading and unloading the stack.
The second advantage becomes significant for debugging messages. For example, the debug logger call
is a far less costly to ignore than is
And especially so in a tight loop. One can argue that you can always check the debug level before making a debug call. In practice, however, this tends to not to be done and instead the useful debug message is never written.
The third advantage is that it is trivial in the logger implementation to both quote and escape the message's parameters. My implementation passes them through a JSON converter so that I can also see the internals of the parameter's values.
File file = ...
...
logger.info( "unable to open file: file-name={0}", file );
Doing this has three key advantages. The first is that the ease of adding relevant data to the message is trivial. This then encourages messages that truly aid monitoring and debugging. The second is that there is a very low performance cost to logging in that if the message is never used (due to the log level) then the only cost to making the call is loading and unloading the stack.
The second advantage becomes significant for debugging messages. For example, the debug logger call
logger.debug("{0} moving towards {1}", a, b );
is a far less costly to ignore than is
logger.debug( a + " moving towards " + b );
And especially so in a tight loop. One can argue that you can always check the debug level before making a debug call. In practice, however, this tends to not to be done and instead the useful debug message is never written.
The third advantage is that it is trivial in the logger implementation to both quote and escape the message's parameters. My implementation passes them through a JSON converter so that I can also see the internals of the parameter's values.
SaaS + Java + Exchange ideas & tools
I currently work for small shop where we run a number of middleware services for publishers. We build services in the traditional style of Java servlets, Spring dependency injection, Tomcat deployments, ActiveMQ message bus, MySql for transient data, and Oracle or permanent data. The engineering of this style is well known to the team. But the consequence are that: Our deployments are manual; Startup times too long; Spring has driven us to a monolithic implementation. We are servicing our customer's needs. But we need to do better. How do we get better?
If you are a SaaS + Java shop in RI, MA, or eastern CT and would like to talk about tools and processes please contact me at andrew@andrewgilmartin.com.
If you are a SaaS + Java shop in RI, MA, or eastern CT and would like to talk about tools and processes please contact me at andrew@andrewgilmartin.com.
Co-working in South Kingstown, RI
The summer is approaching and the kids will be home. Yet I need to get work done and I suspect you do too. I would like to create a co-working space here in South Kingstown. A co-working space is a space to work from without the distractions that come with home, family, and coffee shops. It has desks, chairs, wireless, lockers, air conditioning and great lighting. And, with luck, a great location. I am looking to keep the membership costs at no more than $100/month.
If you would like to discuss this further please contact me at andrew@andrewgilmartin.com and we can organize a meeting where the first half is informational and the second is planning.
Update: Everyone likes the idea but I was not able to get anyone to commit. When you work for "free" from home, at cafes and at libraries, paying a $100/month is lot less appetizing. It will happen one day. Just not for me this summer.
If you would like to discuss this further please contact me at andrew@andrewgilmartin.com and we can organize a meeting where the first half is informational and the second is planning.
Update: Everyone likes the idea but I was not able to get anyone to commit. When you work for "free" from home, at cafes and at libraries, paying a $100/month is lot less appetizing. It will happen one day. Just not for me this summer.
Apple's Java source and javadoc
Apple's Java installation does not include the source or javadoc api. Apple does have a developer download at X which does contain these files at <http://developer.apple.com/java/download/>. On my machine the package in "javadeveloper_10.6_10m3261.dmg" is installed at
If you are using NetBeans you will need to add this as a new "Java Platform". This posting assumes you called the platform "JDK_1.6". Once you have created the platform you will need to manually fix how javadoc is used. To do this, in the file
edit the //javadoc/resource element so that "!/docs/api/" is appended. For example,
For some reason, Netbeans emends the source resource with "!/src/" but not the javadoc resource.
/Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents/Home/
If you are using NetBeans you will need to add this as a new "Java Platform". This posting assumes you called the platform "JDK_1.6". Once you have created the platform you will need to manually fix how javadoc is used. To do this, in the file
~/.netbeans/6.9/config/Services/Platforms/org-netbeans-api-java-Platform/JDK_1.6.xml
edit the //javadoc/resource element so that "!/docs/api/" is appended. For example,
<javadoc> <resource>jar:file:/Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents/Home/docs.jar!/docs/api/</resource> </javadoc>
For some reason, Netbeans emends the source resource with "!/src/" but not the javadoc resource.
Commutative data types and their operations is a powerful idea.
Somewhere this fortnight I read that I should read "CRDTs: Consistency without concurrency control" <http://hal.inria.fr/inria-00397981/en/>. The basic idea is to have a data structure where for the same set of operations applied in any order will lead to the same state. If I have a number of replicants of this data structure then as long as all the operations are applied to each replicant -- in any order -- then the replicants will have the same end state. With concurrent replicants then all replicants will eventually have the same state. Commutative data types and their operations is a powerful idea.
Downgrading my iPhone to iOS 3
I finally took the steps to downgrade my iPhone 3G from iOS 4 to iOS 3. The lose of responsiveness of basic features in iOS 4 is in no way made up by the new features, especially as few of them work with the 3G hardware. I followed the directions on the Lifehacker page at How to Downgrade Your iPhone 3G[S] from iOS 4 to iOS 3.1.3.
One-liner to find all occurrences of property names within a stream of text
One-liner to find all occurrences of property names within a stream of text:
This posting is mostly to remind me that 1) print "..." while /.../ does not work and 2) to use \G to anchor the regular expression to the first and/or last match.
cat * | perl -ne 'while ( /\G.*?([-a-z\.]+\.enabled)/gi ) { print "$1=x\n"; }' | sort -u > /tmp/enabled
This posting is mostly to remind me that 1) print "..." while /.../ does not work and 2) to use \G to anchor the regular expression to the first and/or last match.
Having a Java generic class return a type reference to its specialized class. Aka, using the self-type.
Emulating "self types" using Java Generics to simplify fluent API implementation is a very good example of how to more effectively use Java's generics and inheritance. The "self type" solution is something I would not have imagined. Further, it is motivating to see what other effects can be achieved through Java's generics. Read the article for a compete understanding of self types and a use of them. For those wishing to skip to the conclusion, if you need a generic class instance to return a typed reference to the specialized class then it is coded
public class GenericsTest {
// We have a generic type here that needs to return from its check method
// a reference to the specialized type (that is, the generic's sub-class).
static class Common<SelfType extends Common<SelfType,ElementType>,ElementType> {
protected SelfType self() {
return (SelfType) this;
}
public SelfType check( ElementType e ) {
// do something here that is general to all sub-classes
return self();
}
// ...
}
static class Foo extends Common<Foo,Integer> {
public Foo doFoo() {
// do something here is is specific to Foos
return this;
}
}
static class Bar extends Common<Bar,String> {
public Bar doBar() {
// do something here is is specific to Bars
return this;
}
}
public static void main( String[] args ) throws Exception {
Foo foo = new Foo();
Bar bar = new Bar();
foo.check(1).doFoo();
bar.check("a").doBar();
}
}
// END
Tools needed to support a simple content organization and workflow
I like to use the file system. And I like to use the file system's default explorer: On the Mac this is the Finder, on Windows this is the Windows Explorer, and on Gnome it is/was Nautilus. I much prefer to have other tools integrate into the file system explorer than for the tool to have its own explorer.
When this happens, it is then possible to create your own tools that support your workflows. (Note that in this posting I am concerned with explorer integration and not command line integration.)
Unfortunately, this has not been the norm and so we have iPhoto, iTunes, etc, having there own content organization that is independent of the file system. These content specific explorers are often created due to the weaknesses in extending the system's explorer but mostly it is because creating another explorer is just easier to both imagine and code. The up-shot is that we have organizational silos with little integration between them. And, worse, the inability to create your own specialized tools from the integrations.
This Sunday I spent far too much time tracking down tools that would allow me to use the file system to collect and organize content as follows:
1. Each content collection is a folder. I am comfortable with using the file system hierarchy for organizing and I am aware of the weaknesses of hierarchical organization. However, most collections can be organized into a primary hierarchy and augmented either by search or linking (aliases on the Mac and shortcuts on Windows).
2. Each folder will have, at least, one plain text file of dated notes. Each note is separated by a simple delimiter, for example, a row of dashes. My experience is that this one file of notes is much easier to review and keep updated than to place notes into individual files. With that said, a good tool that can hide the file-ness of the notes would be acceptable too. Until then, one file of notes and using Merlin Man's tip for adding content to it <http://bit.ly/a1kfV6> will do fine.
3. The primary hierarchy needs to be supplemented with full text search. Tagging is not enough and, moreover, tagging can be achieved via search by using specialized tokens: For example, in normal English typography a colon is always followed by a space and so the expression ":foo" distinguishes the tag "foo" from the rest of the text. Spotlight on the Mac is the system's full text searching facility. It is extendable so that other content types can be incorporated. The plugin I am looking for would index the dated notes in a file as collection of "records". (This Spotlight feature was introduced in OS X 10.6.) The Spotlight plugin is the most specific tool that I need. While it is a Mac only indexing solution it is based on plain text content and so could be indexed easily under other operating systems. Unfortunately, as far as I can tell, and I spent a good amount of time looking, there is no plugin for indexing delimited content as records. There are plugins for content types that somewhat match, for example mailboxes, but there is no documentation on how to use them outside of their intended application. The up-shot that that for this very simple workflow I can not compose a supporting tool from existing parts. Looks like I am going to have to write my own Spotlight plugin: If anyone has skeleton code I can build upon please drop me a note.
When this happens, it is then possible to create your own tools that support your workflows. (Note that in this posting I am concerned with explorer integration and not command line integration.)
Unfortunately, this has not been the norm and so we have iPhoto, iTunes, etc, having there own content organization that is independent of the file system. These content specific explorers are often created due to the weaknesses in extending the system's explorer but mostly it is because creating another explorer is just easier to both imagine and code. The up-shot is that we have organizational silos with little integration between them. And, worse, the inability to create your own specialized tools from the integrations.
This Sunday I spent far too much time tracking down tools that would allow me to use the file system to collect and organize content as follows:
1. Each content collection is a folder. I am comfortable with using the file system hierarchy for organizing and I am aware of the weaknesses of hierarchical organization. However, most collections can be organized into a primary hierarchy and augmented either by search or linking (aliases on the Mac and shortcuts on Windows).
2. Each folder will have, at least, one plain text file of dated notes. Each note is separated by a simple delimiter, for example, a row of dashes. My experience is that this one file of notes is much easier to review and keep updated than to place notes into individual files. With that said, a good tool that can hide the file-ness of the notes would be acceptable too. Until then, one file of notes and using Merlin Man's tip for adding content to it <http://bit.ly/a1kfV6> will do fine.
3. The primary hierarchy needs to be supplemented with full text search. Tagging is not enough and, moreover, tagging can be achieved via search by using specialized tokens: For example, in normal English typography a colon is always followed by a space and so the expression ":foo" distinguishes the tag "foo" from the rest of the text. Spotlight on the Mac is the system's full text searching facility. It is extendable so that other content types can be incorporated. The plugin I am looking for would index the dated notes in a file as collection of "records". (This Spotlight feature was introduced in OS X 10.6.) The Spotlight plugin is the most specific tool that I need. While it is a Mac only indexing solution it is based on plain text content and so could be indexed easily under other operating systems. Unfortunately, as far as I can tell, and I spent a good amount of time looking, there is no plugin for indexing delimited content as records. There are plugins for content types that somewhat match, for example mailboxes, but there is no documentation on how to use them outside of their intended application. The up-shot that that for this very simple workflow I can not compose a supporting tool from existing parts. Looks like I am going to have to write my own Spotlight plugin: If anyone has skeleton code I can build upon please drop me a note.
Cat & tail
I needed to count the open and close events in an active log file. Since I needed to start the count at zero I needed to start counting at the first line of the log and continue as the log grew. I needed both cat and tail -f. Turns out that tail can do both using tail -c +0 -f events.log. And so all I needed to do was to pipe the output to the counting script. The script took several minutes to catch up from processing previous events to current events but the counts it displays are accurate.
Using gnuplot to plot a time-series in text
I needed to review a sizable time-series on a remote host. Since I work at the command-line here having the plot displayed in text would be very helpful. To do this I used gnuplot. The data has the form
I found the following helpful in figuring this out http://linuxgazette.net/126/peterson.html.
Update: Use the watch command to turn the graph generation into a dynamic visualization. For example watch -n 60 bash plotting_script will run the plotting_script every minute and update the terminal.
Update: See the bash script timeplot for a useful expression of this posting.
2010-11-06T00:00:01 35 2010-11-06T00:05:01 38 2010-11-06T00:10:02 45 2010-11-06T00:15:01 38 2010-11-06T00:20:02 38 2010-11-06T00:25:01 38 ...Note the two spaces between the data columns. The command line & script to plot this data is
gnuplot <<EOH set terminal dumb set autoscale set xdata time set timefmt "%Y-%m-%dT%H:%M:%S" set xlabel "Date & Time" set ylabel "Count" set title "Requests waiting on connection" plot "data" using 1:2 with lines EOH
which outputs
Requests waiting on connection
Count
160 ++--+--+---+--+---+--+---+--+---+--+---+--+---+--+---+--+---+--+---+-++
+ + + + + + + "data" using 1:2 ****** +
| ***** |
140 ++ ****** * ++
| ** * |
120 ++ ** * ++
| * * |
| * * * |
100 ++ * * * ++
| *** * |
80 ++ *** * ++
| * ** * |
| * ** * |
60 ++ **** * ++
| * ****** * |
40 +*************** * *************************** ++
*** ******** |
+ + + + + + + + + + +
20 ++--+--+---+--+---+--+---+--+---+--+---+--+---+--+---+--+---+--+---+-++
00:00 02:00 04:00 06:00 08:00 10:00 12:00 14:00 16:00 18:00 20:00
Date & Time
Must easier to see the trend in the data.I found the following helpful in figuring this out http://linuxgazette.net/126/peterson.html.
Update: Use the watch command to turn the graph generation into a dynamic visualization. For example watch -n 60 bash plotting_script will run the plotting_script every minute and update the terminal.
Update: See the bash script timeplot for a useful expression of this posting.
Vi and a little macro to step through a file of data
I needed to review a large number of data in a file each of which had a variable number of lines. Luckily, each datum started with the pattern ^"http- and so using vi I was able to quickly review the data by
1) creating a macro to find the next datum and move it's line to the top of the screen
2) use the macro to move from datum to datum
1) creating a macro to find the next datum and move it's line to the top of the screen
qn /^"http- zt q
2) use the macro to move from datum to datum
@n
I found the following helpful in figuring this out http://www.pixelbeat.org/vim.tips.html.
Subscribe to:
Posts (Atom)