Covid update

The bad news is that everyone in the family got it; the good news is that surprisingly only one of the six carers working with Grandma tested positive.¹ Alas the carer who did get it ended up giving it to her spouse and four kids, too.

So that gives us a total of ten patients in "our" outbreak. Only one person (Grandma) had to be hospitalized. The rest of our family had "mild" symptoms; the carer and her kids are in a similar boat, but her husband seems to have gotten hit a bit harder (though not badly enough to need admitting). Still that's nine out of ten recovering without heavy intervention. ²

Grandma had a harder time. After about nine days in the hospital she developed a pneumonia and had to be transfered to the ICU and intubated. Eight days later they were able to extubate her and this evening she was transfered out of the ICU. She's still pretty foggy and, of course, is deeply deconditioned. She's looking at weeks of rehab before we can have her back. In the mean time we get a few minutes a day of video chat as the hospital is fully locked down: no visitors for any reason.

---

¹ Part of our last upgrade of procedured was asking the carers to wear masks full time on shift (and we provided KN-95s for those who wanted them); my wife and I also wore them whenever we went downstairs (that is into the space shared with Grandma and the carers). However, neither Grandma nor the toddler could be convinced to keep them on. None-the-less the policy seems to have made a difference.

² Recovery is not a simple thing, however. I've been "over" this for two weeks now and I'm still suffering unusual mental and physical fatigue. Just six or seven hours of work (i.e. programming and reports) is leaving me inattentive and slow for the rest of the day. Just walking two-and-a-half miles on an improved trail in forty-five minutes leaves me feeling like the day included a marathon martial arts workout. My wife has similar complaints.

Well that explains a lot...

Me (sarcastically):

Yeah. What kind of crazy person is Daddy?

Toddler (oblivious to sarcasm):

A boy!

Covid in the house

Well, at least two of us have it.

My better half got a test late last week and while she she doesn't have the results yet a contact tracer called. At the same time honorary grandma's cold sent her to the hostpital where it turned out to be covid, too. In addition I had some kind of gunk last weekend and the toddler may have had something durning the week. A couple of Grandma's carers have gotten negative tests in the last few days, but we're still waiting to hear how the others are doing.

So far everyone seems to have a mild case, but we're all holding our breaths (and thankful that this is still an option).

Toddler to Zoom Santa...

Whatcha gonna send me?

Success! Kinda.

So, my efforts to flatpak clang-format have succeeded. If you are willing to interpret 750MB for one program as success.

But hey, the first attempt generated a 17GB package (and took three hours to build on a fast four-core machine compared to 15 minutes for the current iteration), so this looks pretty good by comparison. But I'm still remembering that my first $N$ linux boxes had smaller disks than that. Including the one I analyzed my dissertation data on.

Anyway, this being something I did for work I can't share the actual build manifest with you, but I can offer some pieces of advice:

• Get the code from the git repository rather than downloading tarbals. You can do this in the source section of the manifest.
• Don't use buildsystem: cmake (or cmake-ninja) which will just build everything, but instead use buildsystem: simple and issue the build commands yourself. This lets you do ninja clang-format and thereby get the build system to pick out the minimal set for you.
• To figure out exactly what was needed by way of installation I put some finds amung the build commands and used flatback-builder -v so that I could see all the output. With the build down to a quarter of an hour, several passes through the build process to get it right is cumbersome but tolerable. Remember that you are installing libraries as well as the executable.

I imagine that some size could be saved by a fully static build, but this will do so it's no longer worth my time.

---

Followup: A simple "static" build of the program occupies between 2.3 and 6.8 MB on the platforms I care about. I think this may be the way to go. I put scare quotes in there because on gcc doesn't like to statically link a number of libraries (glibc for one) so these things are not completely static and accordinagly not cross-platform. But the build is less than fifiteen minutes on my fast test machine, so we can afford to build it as needed while setting up a new repository.

Likea?

Should we be dynamically or statically linking?

That question used to come up for public debate and reconsideration in computing circles every once in a while, and was generally answered by the fact that resource constraints simply let you do more with your hardware when dynamic linking is the norm. Of course, that decision doesn't come free. Library updates introducing uncaught bugs to a host of dependent programs is a thing. So is DLL Hell and it's close cousin all-or-nothing-upgradeitis.

And though Moore's Law may be dead in the form that we knew and loved it decades ago, the real cost of all kinds of computing resources has come way, way down. So sometimes we have diskspace and even working set (well at least in RAM; not so much in cache) to burn.

Enter flatpak. Not just a program, but also it's whole dependency chain as a separate resource!

OK. It solves a problem for software providers ("I have to figure out how many different packaging systems and get this past how many different policy review committees to make sure my product is available to all my potential users!?!") and the flip side of the problem for users ("I want to try Newthing version 2.0, but my distribution is still stuck on 1.7!"). Granted.

But the cumulative effect of using it for everything is or would be vastly wasteful. Thankfully I don't hear of very many people trying to use it for everything. Or am I just ignorant?

Anyway, my personal policy runs along the lines of "Use your distribution's single, integrated, resource-sharing environement for as much as possible, but there is this thing you can do in a pinch."

That said, I mentioned recently that I'm preparing to enfoce the use of clang-format on my project at work. Only you have to pick a single version of the program or else chaos ensues, we have a divierse set of build environments, and they don't share a common version availble from the official (or semi-official extended) repositories.

So I am actually trying to build a flatpak to provide clang-format¹ so that my team has an easy way to install a compliant tool.

Wish me luck.

---

¹ For future references: left to it's own devices llvm builds everything by default to the tune of almost thirty gigabytes (and takes several hours on a fast, quad-core machine to do it, too). Must tell it to build less. Probably need to do the same with clang, too.

Covid getting close

Despite the notable success of my part of the country during the early months of the pandemic we're in the thick of it now. Some combination of luck running out and people getting careless through exhaustion with the continuous state of low level emergency, I guess.

My training as a physical scientists left me with an irresitable urge to put things into perspective in a particular numeric manner. I note that at this point the numbers in the news correspond to approximately 3% of the US population having been diagnosed with Covid19 at some point this year and a little less than one-tenth of a percent having died from it.

So it is very easy to not know anyone who has died from this thing and not particularly hard to not know anyone who has gotten it. Especially if you live in an area that had not been hard hit until recently.

On the other hand this means we can expect Covid19 to be the third largest cause of death among Americans in 2020, outstipping all causes of violent death.

Anyway, with the virus surging in our community we've been affected in two ways this week. First, my daughter's preschool (which had re-opened in mid summer) has closed again after two cases in a month (but apparently no transmission in the building). They won't be re-opening until after the new year. Secondly two members of the team who care for our honorary Grandma are quaratining after a relative of theirs was diagnosed with the virus. The contact tracers have not called us so *we* may be in the clear, but there are three adults and four kids who are in our immediate circle at risk. Scary times.

We're trying to tighen up our practices again, and we had also started wearing masks in the common areas of the house last week (though neither thte toddler nor Grandma pay much mind to that). The medicos have learned a lot about how to treat this thing in the last eight months, but our hospitals are running just about at capacity right now which is not promising.

The customer is often wrong

They just get to have it their way anyway.

The virtues of not being clever

I encountered a discussion of FizzBuzz (as a a test of basic programming competence rather than in other contexts) out on the internet the other day. In particular the discussion covered the annoying fact that the naive ways of implenting the thing end up giving up a slight degree of inefficiency: it's hard not to have at least one "redundant" condition evaluation.

And I had a brainstorm in passing: a slightly less naive implemnetation that avoids that pitfall. I'm positive I'm not the first person to have through of this but I don't recall seeing it written down anywhere before. So here is the decision code in c++:

#include <cstdint>
#include <string>

std::string FizzBuzz(uintmax_t n)
{
    switch (n % 15)
    {
    case 0:
        return "FizzBuzz";
    case 3:
    case 6:
    case 9:
    case 12:
        return "Fizz";
    case 5:
    case 10:
        return "Buzz";
    default:
        return std::to_string(n);
    }
}

It has only one test and the switch is likely to be efficient. So, it's great right?

Well, no. I think that it suffers relative the naive approach in terms of clarity and conveyance of intent and would generally avoid this kind of cleverness in production code unless there was a big gain from using it.

You see, the switch is horribly opaque. To understand the task by reading the code one has to puzzle over the selection of cases, while naive implentations have the conditions written out explicitly.¹ If performance was known to be an issue, I might look at the switch-based implementation, but until then I'd stick with one of the simpler but clearer ones.

Well, look at that!

While looking up links for this post I ran into a blog post that exhibts a switch-based solution albeit in ruby which means the switch has different semantics²

---

¹ Here I am assuming that if ( i % N == 0 ) will be read as "if i is divisible by N", which I suppose is a unusual outside of programming circles. Failing that I suppose the implemntor could write a predicate isDivisibleBy(uintmax_t numerator, unitmax_t denominator).

² I believe ruby's switch is syntactic sugar for a chain of if statements, while the C++ construct's (often annoying) limitations allow it to be implemented under the hood in a variety of ways that may include jump tables.

Audio goodness

I have a gamebox¹ and like to play games where the sounds carries information. So I need stereo, and I need enough volume to hear what is going on in the game soundscape.

But ...

• These games can be pretty noisy and other people in the house don't want or need to be disturbed by my entertainment
• The household also includes a toddler and a couple of people with moderate to severe health issues. I need to be able to hear when I have to stop playing right now and go take care of something important.

So getting a good pair of over the ear headphones and shutting the doors (the go-to solutions for hearing your game) are not really good ideas.

Now, a couple of years ago I spotted a solution: a wearable speaker. But at the time my finances were rather tight and before they recoverered I had forgotten. Then one popped up on woot: a JBL Soundgear.

The experience of using this thing for gaming is immersive. The sound is rich and seems to emenate naturally from all around you. It's hard to believe that everyone around you isn't hearing the same thing, but if you hand the device to a friend to try out you hear essentially nothing while they're wearing it. Of course you still hear local environmental noises which isn't ideal if you are listening for a telltale footfall in-game, but I chose this route because I need to be connected to my real-world environment.

These things might not be for everyone, but I'm pleased.

---

¹ My gamebox is a PlayStation 4. Alas, Sony cheaped out on the bluetooth audio support (in an effort either to save a few cents per unit or to squeeze a few more bucks out of the consumer by controling the supply of compatible devices. You'd find it very easy to convince me of the latter. So I also bought a Avantree DG80 USB to bluetooth audio device. Works like a charm.

Why is it always "percentage"?

I got a side assignment at work Friday: write a test harness and tests for a pieces of software a colleague is writing.

This is suppose to be a reference implementation, so having different people write the code and the tests is a good idea: it will tend to expose ambiguities in the specification and unstated assumptions in the implementations.

"No problem," I said, "but I'll need a copy of the spec."

As I'm reading the spec today I come to:

The input value, given as a precentage in decimal format (a value between zero and one)

Sigh.

Now, there is a strict reading in which this is technically correct, but that reading conflicts with the lay uage of "percentage" in a way that leaves room for ambiguity. The parenthetical clause is necessary to resolve the ambiguity. In the tradition of academic physics we'd have written "The input value, given as a fraction" explicitly because there is no chance that it will be misinterpreted.

Well, what format are we using now?

Code formatting guidelines are good. Code formatting guidelines that people follow are better.

Now, my team is pretty conciencious, and we all try to follow the guidelines, but there are two issues:

1. We don't actually have a written standard.
2. We're all humans. Reasonably attentive but rather fallible humans.

One solution to this is to adopt some kind of formatting tool and require that it be used regularly (or better yet, automate its use). Great. That's the plan. Only...

1. Now we actually have to chose a standard.
2. We already have roughly 120k SLOC written by the aforementioned fallible, moderately attentive humans and we want to minimize churn on the repository caused by simply adopting this tool.¹

The key here is to select a standard that reflects what we've actually been doing with our slightly different personal takes on things, our mixed level of attention, and our occassional outright errors. But how do we look at nearly four hundred files and distill what we've actually been doing?

Right. We get a program to do it for us.

Better still someone else has already written it: whatstyle. This thing attempts to write an optimal style file for you by trial-and-error optimization with a metric like "fewest changes". Though exactly what that means is somewhat debatable.

It supports several common beautifiers including clang-format which is what my colleagues are using. It gave what looked like quite reasonable results when I ran it on a single large source file from my repository, so now I'm running over a large fraction of my repository at once (excluding a directory of legacy code we borrowed from another project and some "include this in your project" third-pary code).² The only issue is that it took roughly a minute to do the one file (it is examining hundred to tens of thousands of style definitions in each optimization round), and now I've handed it more than three hundred files. For the moment my work laptop is functioning as a space heater.

My plan is to send the suggested format file to every coder involved in the project, ask for suggestions, and try to mediate the resulting chaos.

Wish me luck.

---

¹ You're right. An experienced project leader would have adopted the fool thing at the beginning. I'll keep that in mind next time around.

² Something roughly like this:

find ~/Projects/Foo -type f \
             -and \( -name '*.h' -or -name '*.cpp' \) \
             -and -not \( -name 'thirdpartycode.h' -or -path '*LegacyLibrary*' \) \
             -print0 | xargs -0 \
             whatstyle -f clang-format 

Then wait.

Getting what you want

[It is an] established fact that, despite everything society can do, girls of seven are magnetically attracted to the colour pink.

Terry Prattchet

We use plastic tableware. Not because of the toddler, though we use her as an excuse, but because the toddler's honorary grandmother has a neurodegenerative condition that cuases her to drop things from time to time.

Now, plastic stuff has to be replaced fairly oftern, and the latest round of soup/salad/full-meal-potion bowls came in several colors including light pink. The toddler just adores the pink.

Tonight I told her that she had to choose a smaller bowl to put her cookies in, but the only colors of smaller bowls avaliable were not acceptable. I would not be moved and left her complaining on the kitchen floor.

But shortly after I left I the schreeching abruptly stopped and I heard the sound of a chair being pushed across the floor. It was not long before the toddler appeard with the pink bowl with a couple fo cookies lurking suspiciously in its depths. Score one for the toddler. It was too much effort effort to get back up and tell her I cknew what she was up to.

"Safe" string building in c

The deficiencies of the c standard library where strings are concerned have been discussed at length in many places, but I'd like to talk about one issue in particular: building up strings from pieces.

This week I wanted to build up some strings from pieces. Lots of little pieces. In a language with "real" strings this would be easy, you'd just do something like string result = substring1 + substring2 + substring3;, but c does not support that in any general way.¹ There are really only two classes of tools available: strcat/strcpy functions from string.h and sprintf functions from stdio.h. Essentially all the "plain" functions will buffer overrun, and the "n" variants only protect you if you pass the right length (where "right" means paying attention to which functions count the terminating '\0' and which don't). Let's take a closer look at some of them .

strcat, strncat (and strcpy, strncpy)

On the face of it this is the "obvious" thing to do. After all it concatenates strings, right? The main issue is that the first string needs to occupy a sufficiently large buffer or you run into trouble. Which means knowing the desired length when you create the buffer.

If you have a separate buffer you use a "copy" function to move the first string into it and then concatenate onto the end.

As a side note, if you are going to repeatedly apply strcat, do capture the return pointers from each call to avoid a Shlemiel the Painter algorythm.

sprintf, snprintf

Same basic problem: the target buffer needs to be big enough. On the up-side, you don't need extra elements for any little connector text you want to stick in between the substrings that you are recieving from the caller, the environment, the database, or whatever.

And that is really it.

Variable length arrays or malloc

So, you need to wait until you have all the parts, find the desired length, and then create a buffer. Fine. You have two choices: a dynamic allocation or variable length arrays.

The issue with variable length arrays is that they weren't stanadrdized until 1999 and then were made optional in 2011 (and at least one major compile does not support them). Code that depends on VLAs is going to have limited poratability.

So you're going to have to put it on the heap with all the hassle and risks that entails. Great. Be sure to check the return value.

Recipe

All this is old hat and there is a well known hack to accomplish it. You use the "returns the length that would have been written" feature of snprintf functions like this:

nonnullcount = snprintf(NULL,0,...);
buf = malloc(nonnullcount+1); /* watch out for the with/without '\0' issue! */
/* check for errors */
snprintf(buf,nonnullcount+1,...); /* watch out for the with/without '\0' issue! */

It's not hugely time efficient; the two passes through the formatting engine are inellegant, but it's very flexible and does the job.

The three lines of code I exhibited there are idomatic and easy to recognise, so sprinkling them through your code wouldn't be too bad. Except that the error checking is pretty improtant and it will add to the length and break up the visual block. Not nice. So I'd like to encapsulate all that in a function. Which is where it gets technical; how are you with c's variadac function support?

Wrapped up and made pretty I get something like (written to be compatible with c89 compilers even if they would require a more up-to-date libc):

/* A "safe" auto-allocating sprintf.
 *
 * Returns a pointer to a malloc'ed buffer containing the resulting string or
 * NULL if an error occurs. Leaves errno set following the error.
 *
 * It is the caller's responsibility to free the buffer.
 */
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>

char * smallocprintf(const char * format, ...)
{
    char * buf = NULL;
    size_t nonnullcount = 0;
    va_list args;
    va_start(args,format);

    nonnullcount = vsnprintf(NULL,0,format,args);
    va_end(args);
    buf = malloc(nonnullcount+1); /* +1 for the '\0' */

    if (buf == NULL)
        return NULL; /* Leave errno intact for the caller to deal with */

    va_start(args,format);
    vsnprintf(buf,nonnullcount+1,format,args); /* +1 for the '\0' */
    va_end(args);

    return buf;
}

Frankly I imagine that code like this exists in private libraries all over the place, but I hadn't seen it myself, and it does what I need. Late edition: In particular this seems to be a close analog to the GNU c library function asprintf though there are some interface differences.

---

¹ String litterals may be simply concatenated in code, but not null terminated character arrays and buffers.

A modest proposal for further presedential debates

Each candidate to be locked in a soundproof box, and their microphones to be live only during their alloted speaking periods.

Nothing new under the iron sun

I just had this great "new" idea that could be used for the basis of a time travel story. Yahoo!.

Except that, on reflection, I believe that Charels Stross not only had it first, but followed the consequences through more completely that I'd gotten around to. ::sigh::

The good news is that he pushed the idea in one direction and I could take it in another. We'll see what comes of it.

Don't speak so loudly sir, or everyone will want one!

Is that a bug in my BeautifulSoup?

It seems that the maintainers consider prettify() to be a debugging tool and are not concerned that it often adds unmeant white-space to the output.

Uhm. Really?

I mean, I can see the utility of a tool which exposes the tag structure as clearly as possible, but I also like to have my machine generated code look good (i.e. be fairly readable) while it remains correct. Perhaps a interface like prettify(preserve_output=False) would be in order?¹

---

¹ Here I have selected the default sense to not break existing code, starting ab inito I would have made True the default.)

I'll have the BeautifulSoup

We've delived version 1.0 of my project at work; or perhaps we're in the process of delievering it.

We've handed over a complete copy of the repository and deployable binary bundles for three targets and VM based development environments for two.

We still have some time to complete and hand over the documentation. Better still, by policy I set very early on all the documentation is in the repository, so they already have it. And I tried to keep us writing and updaing documentation often enough that it didn't get too far out of synch with the code, so the documentation they have is pretty good. But we're taking this time to make it better: more complete, easier to understand, and reflecting the code we handed over even better.

There is a twist: even though the project is entirely in-house, it's paid for by a contract to which we are a subcontractor. This means that the prime contractor has the responsibility to report on the work (and the authority to make demands about that). This particular prime contractor has a tradition of including the user manuals for software projects as appendicies to the report.

But my main user documentation takes the form of a pile of individual html files that are displayed in a on-line help system. There is no linear structure imposed and no single document containing it all.

So I needed to flatten it.

There are only 20 files right now, so I could have done it by hand. But that would have taken a few hours or more to get right (all the internal links!) and I have reason to believe the project will be funded for further development meaning I'll have to do it all again in a year or two. That's excuse enough to automate the task.

The tool I settled on was a python library called BeautifulSoup which I had heard of but never used before. I loaded a html/css scaffold file, looped over a hand-tooled list/tuple structure placing my files into a few categories. In the loop I created a table of contents entry for the file; grabbed the file body, re-wrote the header levels, munged the anchors and links, base64 encoded the images, and wrapped the result in a section for nice formatting; and then appended the new section to the body of the scaffold. When the loop ended prettify and write. It just works. I'm hooked.

On styling text in a QTextBrowser

My project at work is complicated enough to need fairly extensive on-line documentation, which we write as html and diplay to the user in a QTextBrowser. Now, this thing is not a full web browser¹ and only a supports a limited subset of html/css.

Suffice it to say that if you're just had a clever CSS idea it's not supported by QTextBrowser.

At first this didn't bother me too much because I had never done css before. But as I've become more familiar with the tool (and at once awed and appalled by it), I've become aware of just what I'm missing out on. I wrote the quoted sentence as part of our guidelines for writing and formatting the help files.

I'm sticking with QTextBrowser for now because our needs are simple and we can't really expect an incoming junior dev to be able to support a sophisticated pile of css anyway.²

---

¹ That's available as QWebEngine, but it brings in a large module so I didn't want to use it.

² Because of the nature of our work, our target demographic for new hires is "scientists and engineers who can program" more than it is people with a strong development background. Having web skills would be a bonus, but it's not part of the expectation.

Duty and painting oneself into a rhetorical corner

Duty is a debt you owe to yourself to fulfill obligations you have assumed voluntarily. Paying that debt can entail anything from years of patient work to instant willingness to die. Difficult it may be, but the reward is self-respect

Robert Heinlein

"Duty" in the sense of an obligation is a politically contentious idea. I am personaly a fan of Heinlein's definition, but it suffers for some people's purposes from not being something that you can dictate as in "Doing [this thing I'm interested in] is your duty". The best you can do is explain to them why you think their self-respect should depend on doing it.

As a legal reality a few "duties" are required and you can be punished if you don't comply. If you are summoned for jury duty and neither provide an excuse the court finds acceptable nor show up you can be held in contempt and subjected to fines or jail time. If you are a young but adult male in the US and don't register with the selective service you can get in trouble. In the same vein, many people were punsihed during the Veitnam war era for draft dodging, which brings us to the culture wars.¹

The political right in the country largely partakes of a long standing view that military service is and should be required of men "in times of war" (which really means at the discretion of the political leadership). Some who don't go quite that far feel that there simply shouldn't be a way for a "real man" to not want to join the fight which is a slightly different take. In either case the point is that the community needs defense at times, and the costs should be widely borne.

The political left, on the other hand, mostly feels that it is the right of every citizen to question the motives behind and necessity of military action and to demur from participating if they find them unsavory or unecessary. Some go so far as to consider it an obligation to make such judgements. Some have even held unwilling particpants in military misadventures responsible for the failing of the political and military leadership under which they served.

That is a stark divide, and has been the cause of much dislike and mistrust between people on opposing political teams for the whole of my lifetime.

Then comes Covid19 and the issues of wearing masks in public, social distancing, enforced closures of some kinds of business and so on...

Suddenly there talk from the left of a duty to protect others by respecting these rules and talk from the right about their right to judge the urgency of the threat, the effectiveness of the suggested measures, and the motives of the leaders.

Laugh or cry.

---

¹ There is, of course, some tension between this notion and that of religious liberty. Quackers, various Menomnites and other conciencious objectors have been a bone of contension on this matter since the beginning of the nation.

Spherical cows #1: Face masks edition

Let's talk a little about the effectiveness of face masks and the way levels of participation in waering them change the overall outlook of a epidemic. I want to go there because I hear some complaints hear and there that cloths masks don't do "much" and so there is no point in waering them, and I don't think that position is terribly well thought out.

This is the first of a couple of articles I intend to write applying simple models to look for insight into policy choices related to infectous disease in general and aimed at understnding a little about Covid19 in particular. Thought I won't try to make the models numerically faithful to Covid19.

Basic reproduction number

In the jargon of epidemiology the "basic reproduction number" (givern the symbol $R_0$) is the number of people, on average, who any given carrier gives the disease to. In simple models, if this number is larger than one then the disease spreads throughout the susceptible population, if it is lower than one the disease fades away. And we can (again in simple models) further break this number into a product of the avearge number of people a carrier interacts with ($N$) and the average chance that they will transmit the disease in their interactions with a single person ($F$).

For our purposes we can say that masks reduce the chance of tranmission¹ Which means that the effectiveness of using masks translates directly into a reduction in $R_0$.

Guessing at the effectiveness of a properly worn mask

Here we consider "non medical" cloths masks that you make yourself or buy at retail. Now a mask could help in two ways: it can reduce the amount of virus a carrier puts into the environment and it can reduce the fraction of environmental virus that a un-infected wearer takes in. For simplcity I'm going to model these as each generating a equal reduction in the chance of transmission modeled by a multiplicative factor $f$ that is less than one (so it models a reduction) and greater than zero (which would be perfect effectiveness.

So, if a carrier has a single "interaction" with a uninfected but susceptible person the probability of transmission is:

• $F$ if neither is wearing a mask
• $fF$ is one (either one) is wearing a mask
• $f^2F$ if both are wearing masks

But what should $f$ be? I don't really know, so I'm going to guess. To be conservative I'm going to guess that they are not terribly effective. I'm arbitrarily assigning $f = 0.8$. That is, a single mask reduces your chances of getting infected by only one fifth, and both parties being masked gets you down to 64 percent.

What about compliance

Not everybody wears masks. This may be due to a oversight such a failing to grab one on the way out the door, lack of access, or by choice. It dosen't matter to the model. But what does that mean for the overall effectivenes of a masking policy. Not all iteractions will be between two mask wearers. Some will involve one mask and others no masks at all.

Call the rate of mask wearing $r$. If everyone wears them $r$ is one, if only people who were burned by acid wear them $r$ is essentially zero.

Sticking with our "do everything on average figures" approachand assuming that everyone has the smae chance of meeting everyone else² we can compute the overall fraction $\Gamma$ in a simple way \begin{align} \Gamma(f,r) &= \left[ (1-r) + f r \right]^2 \\ &= (1-r)^2 + 2f (1-r) r + f^2 r^2 ] \end{align} And that's the whole basic calculation. It's quadradic in $r$ and concave up. Done and dusted.

Except that we might want to manipulate the math a little in search of further insight. So we continue \begin{align} \Gamma(f,r) &= (1 - 2r + r^2) + f(r - r^2) + f^2 r^2 \\ &= r^2 (1 - f + f^2) + 2 r(f - 1) + 1 \end{align} Also quadratic and concave up in $f$.

Sticking some numbers in

Just looking around me I think it's little optimistic to expect more than 90% mask compliance, so a optimistic number of the overall effect would be $$ \Gamma(0.8,0.9) = 0.672 \;, $$ or a little better than a 30% reduction. That's not great but it is significant. If $R_0$ was as small as 1.45 that alone would be enought to squeeze off the epidemic.³

A more pesimistic value of compliance might be around 60%, leading to $$ \Gamma(0.8,0.6) = 0.774 \,. $$ which is to say less than a 25% reduction in the tranmition proability. For that to be enough we'd need to have started with $R_0$ less than 1.3.

---

¹ We could also say that stay-at-home orders and the like reduce the number of interactions. Social distancing could be modeled as a little of each. I'll say more about this in the next post.

² I'd guess that this is the worst assumption in this calculation, but making it better requires a huge increase in the complexity of the problem and probably necessitates a Monte Carlo (randomized simulation) approach. I'm doing a back-of-the-envelope calculation, so this is what we get.

³ I haven't seen a number recently but early reports suggested something larger than 2.0, and I suspect it might be a little higher than that simple because early on there wasn't much awareness of how many people get it but remain asymptomatic.

You win some, you lose some

On one hand, diapers seem to be out of our lives. On the other hand, potty emergencies are in.

Your messaging isn't reaching the virus

I talked a bit in a previous post about how nerds mostly don't rule the world. One of the big issues is that we think that we live in a world of facts, but most of humanity lives in a world of opinions masquerading as facts. And if enough people believe them then for political purposes they are facts. Politicians get their way in large messure by controlling the way people understand things; spin doctoring works at least some of the time. Which is why message control looms large in the minds of politicians.

But there are some things you can't message your way out of.

When you hear politicians complaining that too much testing is the problem you know they're locked in a mindset where messaging is more important to them than reality. They aren't making any progress in asserting their view of things because of the relentless drumbeat of daily figures, so rather than addressing the hard problem they try to just get rid of the reporting. Rather like the Ravenous Bugblatter Beast of Trall; which is to say "Daft as a bush".

---

Consdier, if you will, what we could do with enough data.

If we could test everyone all at once, we would have a very powerful tool for isolating the virus. Given that we can't do that, if we could at least test everyone who had been in contact with known carriers we would have nearly as good a grip, which is the point of contact tracing (note how quickly South Korea contained their initial outbreak as an example).

What we can actually accomplish is a lot less powerful, and without some help from the population at large it may not be enough to choke off the transmission, but asking for less data is the best way to guarantee that we don't stop this thing.

Squash bugs

My wife and I are gardeners. Not of any particular expertise, but definitely persistent. If I've counted our misses correctly we've been at it seventeen of the last twenty years. At nine sites in five states and six cities.¹ I believe we've only failed to garden at all in one of the eight "permanent" residences we've had in that time (a rental we occupied for less than a year).²

We grow flowers and vegatables in containers and in the ground. Roses, irises, marigolds, geraniums and others. Tomatoes, bell peppers, a variety of chilies, egg plant, leeks, beans, peas, assorted squash, selecetd herbs, occasional greens, the odd brasica and others. We use containers in rentals, when space doesn't leave us with a choice, and (in one case) when suspected soil contamination makes us unwilling to eat food grown in the ground.

By my count we've grown squash at seven of the nine sites. And we've had squash bugs in every single one of them. "Loathing" is simply not a strong enough word to describe my feelings about these little creeps.

They showed up here on Wednesday. I found them as the light was fading from the sky, and I spent almost an hour working my way through all our vulnerable plants trying to determine the extent of the infestation and to stop it. Each of the last few days has included an hour or more of fighting them, and we might be winning. It's only cost us one plant so far, which is not bad.

But the vile awfulness of these critters is not the point of this post. Rather it is the "discovery" I made that first evening.

Back-lit squash leaf at dusk

The hard part of hunting squash bugs is that they generally lay their eggs underneath the leaves, and when they hatch the juvenile stages spend most of thier time their as well. It's a lengthy and painstaking process (not to mention tough on the back and neck) to lift each leaf and look underneath. As the light faded, I fired up the light on my phone, and found that I could see my targets through the leaves.

It's much less work and easier on the neck to slip your phone under each leaf.³

In principle, you could have done this with any decent flashlight, but they generally have the wrong form-factor to manuever around the plant: the shape of a phone is much better for the application.

---

¹ Two of those sites were community garden plots maintained in parallel with gardens at our residences.

² The life of a repeat post-doc is not a stable one. We're good at moving and ready to stop.

³ Sorry, I don't have a photo showing any eggs or nymphs because I don't stop to document my discoveries but move to destroying them at once. It's the only way to be sure. But then, any gardener who's familiar with these things will have no problem spotting the egg arrays and the juvenile stages are pretty obvious as well.

Aerial refueling?

There is a dessert willow growing in a neighbor's yard that overhangs the wall between us. It's pretty and provides us with some nice daytime shade so I don't grumble about the bits that drop off and land on the artificial turff the last ownerr installed.

Today my wife and I were admiring the new bird feeder when a blossom fell from the tree. A hummingbird perched on the feeder tried to drink from it as fluttered downward. No idea how succesfful the attempt was, but the maneuverability of those birds is breathtaking.

"Foreign" doctors

Members of my household have accumulated a lot of experience with the US medical industry.¹ Just now we have collectively eight specialists across five disciplines (and primary care, of course), and the frequent moves I've subjected my family to have meant finding new people in those (or other) positions repeatedly in the last few decades. We've interviewed many dozens of specialized physicians over the years.²

And I've noticed something about what doctors are available.

Most of the time we get offered appointments in less than a year with two kinds of doctors: people who turn out to be a poor match for us and "foreign" people (meaning people who have a noticeable accent or did part of their training in another country, and often meaning "not white").

Now, to be licensed in this country a doctor needs to do a residency here no matter what credentials they may have elsewhere, so every one of these doctors has been checked out by the local medical establishment (and because we're talking about specialists they have been checked out twice). We've found some of our very best specialists this way. But we've also heard from other members of our communities that they "weren't comfortable" with these same people.

Okay, in one case the physician had a pretty marked accent; one I remember finding pretty hard to process when I first started working with people from the same part of the world.³ I can buy that many residents of that rural and somewhat insular part of America might have found dealing with the accent one problem too many on top of dealing with medical issues. But in most cases these folks have mild accents (presumably owning to spending many years in the States) that merely give their speech an exotic edge.

A strategy

My unscientific conclusion is that some portion of the American public is avoiding a subset of our good doctors for being "not my in-group" (I won't speculate if this is overt racism, implicit racism, or just provincialism) and that means that it's easier for you to get access to those doctors.

The point is that it can be hard to get specialists, especially good specialists, so you don't want to give up any available edge. Don't be put off by an accent or a face of a different shape or color than your own.

Aside on exceptions

We also have very good specialists who are white. And some who are male. And some who are getting on in years. Our current crop includes one who is all three of those things and he's a serious keeper. But we didn't get him by calling the clinic and asking who was available, we asked the specialist at the "from" end of our last more to intervene for us and we got in with this guy on the basis of inter-doctor networking.⁴ That's been the case with a lot of our specialists who look like they came from central casting.

---

¹ It wouldn't really be right to call it a "system" except in the sense of "ecosystem". There is no organizing principle, and while many parts of it are subject to oversight of one kind or another the oversight level isn't terrible well coordinated.

² Yes, "interviewed" is the right word. We go into a new specialist's office with the questions "Is this the right doctor?" just as much in mind as the history and needs of the patient. Doctors who want to micromanage a patient used to being part of their own care are a bad match. Doctors who don't want to answer questions when a patient feels they need to know more about 'Why?' are a bad match. Doctors who tell a couple comprised of a graduate engineer and a physics professor "You don't understand the math" actually get laughed at in front of their own staffs (I feel a little sorry about that, but not very; she is, indeed, one of the elite thinkers in our society but dismissing us out of hand was out of line).

³ Much of US graduate education has a lot of diversity in country of origin even when there is not so much in gender, economic background, or race among the Americans. And experimental particle physics is an international endeavor so I got a lot of practice during my years as a scientist.

⁴ This is another strategy. If you ever have to leave a good specialists (for whatever reason) see if they can hook you up with a replacement. They already know your case and your style and may be able to choose a good match. And their influecne can get you in to see doctors who are "not taking new patients at this time".

Neolithic

Image courtesy of wikimedia

I'm taking the position that awk may be a stone aged tool, but like a Clovis point it represents the very finest craftsmanship of that age: simple in form, highly functional, and beautifully constructed with everything it needs for its task and nothing else.

And no, perl is not a better example. Larry bolted everything including the kitchen sink onto it. That certainly made it more functional, but it also made it ugly. I'm not highly conversant with the vast array of choices in the rapid development marketplace, but I would judge that both ruby and python come a lot closer than perl to the goal of being at once all-singing-all-dancing and beautiful. Tcl/tk is right out and I an happy having no basis on which to judge php.

"Rights"

I've been thinking about 'rights' as in "constiutional rights" in the context of mask orders.

Leaving aside the reasonableness of mask orders and the long history of the courts upholding public health measures, I've been wondering in particular about why people might¹ claim there is (or should be) a 'Right' to not wear a mask in a retail establishment.

And I think we should see it, at least in part, as pedictable fallout of a long-running campain from the political left.

I'll explain.

Originalism

It is a common argument on the political right (or at least the political right as it was in my younger days) that 'rights' are something that the people have against the government. That's basically the content of the Bill of Rights, after all: things the government may not do to "the people" or to individual people.² In that view restraints may be imposed on people vis a vis other people by way of laws, but those are different from and limited by 'rights'.

Plenty of newfangled models

But here's the thing. For more than my entire lifetime there have been 'rights' that control how a person may interact with another person.³ A business may not discriminate against a person on the basis of a moderately long list of things, right? Nor is this limited to corporate entities (which recieve privileges from the state), but also applies to a person who is simply doing business with their own money. Similarly a business (i.e. even a person) generally may not discriminate in hiring against a person with a recognized disability nor in most cases forbid some service animals.

Now, I think a pedantic examination would suggest that these aren't 'rights', but rather requirements of the Civil Rights Act in the former case and the American's With Disabilities act in the latter (and various pieces of legislation that have followed) and as such are "laws".⁴ But the narrative I see in the popular press uses 'rights' or 'right' on a regular basis. To the extent that the descriptivists have won the argument over the meaning of words, that's a change in the understanding of what a 'Right' is from the beginning of the republic.

Not that am I terribly worked up about it. I'm strongly in favor of treating people decently whatever their color, gender, religion, etc and even ::shudder:: poltical beliefs or favorite college football team.⁵ And I recognize that a coordinated campaign of "individual" intolerance by the majority is completely asymmetric and every bit as oppresive as legal inequality,⁶ so allowing people to freely discrimiante in their public lives would be little if any movement toward equality. Nor do I think it is reasonable to say to the members of an oppresed minority "Well, I think you're right about this, but you really need to suffer for another couple of generation so that we can have a smoother transition with less friction", which seems to be the alternative.

But it sets the rhetorical stage for the idea that people can have 'rights' to deploy against other people

Right 'Rights' right back

The political right has been getting beat up using this language for roughly sixty years, so it shuoldn't come as a surprise to find that they have decided to retailiate in kind. In this the left have won the cultual wars: the rhetorical weapons of choice have become those the left selected way back before I was born.

---

¹ I say "might" because I am just guessing.

² And we'll just not talk about how incredibly wonky it is to have a system that on one hand holds up these shining ideals that people are in many ways more important than governments and on the other lets people be held as property. Denial is not just a big river in Egypt, after all, and even people who try to be rational and principled can suffer from it in startlingly large ways.

³ Here I'm attributing existance to positions held up by the courts and by the Supreme Court in particular. You're welcome to believe (as I do) that some of these are in error, but they are the things that will be enforced for good or for ill.

⁴ Generally the rulings are that the governement may in fact make such a requirement in law.

⁵ "I don't have anything against Buckeyes, but would you want your sibling to marry one?"

⁶ Even if we posit a lack of organized and legally unaddressed violence and intimidation ala Jim Crow (and even if you can belive it starts that way how long do you think it would last in the face of any resistance?).

On toddler fearlessness

Can you get a subscription to the emergency room?

Un-plan for growing up v0.1

When I was growing up there were certain thing my parents "made" me do. I didn't always like it at the time, but looking back on it I'm glad they did it. It gave me a set of skills that have served me very well as I moved into adulthood. Some of those skills like

• cooking
• typing
• navigation in the city or the woods

have been directly applicable to day to day life, but others like

• persevernce through failure
• the conviction that I can learn the basics of almost anything
• the knowledge of what it takes to turn my own suckitude into at least bare competence

are more abstract and perhaps more important.

My folks admitted later on that some of these lessons were consciously thought out (and by implication that some were not), but I also lived what would now be characterized as a "free range" childhood and learned some of them on my own or with my friends.

I think that both the structured and unstructured parts of my childhood have been hugely (and mostly positively) influential on my adult life, and I'd like to give my daughter the same advantages. Towards that end I've been trying to build a plan for what my spouse and I should require and not require of her. My first pass at a list is ... lengthy but not too structured. Hopefully it can be characterized as "ambitious but not suffocating". In any case, I'm sure the child will cause revisions as we go along.

So here it is for posterity:

Things you should do (besides your schooling) while you live at home to a lay a foundation for a good life as a grown-up.

• Learn the tasks of keeping a house. Practice all the things that are part of independent living
  • cooking
  • doing the laundry
  • cleaning various parts of the human environment
  • changing the AC filters, the batteries in the smoke alarms, etc
  You're going to have chores. Petition to have the changed from time to time.
• Practice the arts, get a basic grounding in technique of several kinds
  • Musical performance (vocal or instrumental)
  • Conveying stories and emotional states (acting, prose writing, poetry, dramatic reading, or story telling)
  • Visual arts (drawing, painting, photography, videography, etc)
  and spend a couple of years working on at least one
• Study a foreign language, and use it at least a little. Learn about the (or a) culture that comes with the language.
• Pursue a diverse range of individual skills:
  • Intellectual or technical skills (chess, stage-lighting, cooking, logic, philosophy, audio-mixing, art history, etc)
  • Physical skills (sports, horseback riding, martial arts, mountain biking, etc)
  • Mechanical, building, or programming skills (kitting, carpentry, automotive maintenance, web authoring, etc)
  • Skills for dealing with emergencies (first aid, CPR, how to work a fire extinguisher, lifeguard techniques, signalling and emergency shelters for wilderness emergencies, etc)
  and stick with at least one in each category until you're pretty good at it. Fix or build something that you still use years later.
• Spend a couple of years involved in a team sport or competition. Learn to turn your efforts to the team goal and to trust your teammate to do the same. Win some; lose some.
• Spend a couple of years involved in a individual sport or competition. Learn to rely on yourself. Win some; lose some.
• Experience the great outdoors in multiple environments and get comfortable in at least one. Have at least a small adventure.
• Complete (on your own or as the leader) projects with varied requirements
  • One that has many steps and requires planning well in advance; carry a clipboard, make lists
  • One that takes weeks of concentrated effort
  • One that takes longer but comes a little at a time
• Spend some long days in hard, physical labor. Learn to pace yourself, and learn that you can keep on keeping on.
• Spend some time taking care of others. Empathize to anticipate their needs.
• Connect with other people
  • Make friends
  • Learn to disagree respectfully
  • Learn to respect someone you don't like
  • Be there for someone else
  • Ask for help when you need it
• Connect with your inner life
  • Play
  • Do nothings much sometimes
  • Use your time wisely most of the time
  • Find awe in the world
  • Start learning who you are
  • Think about how you can be more like the person you want to be

It's a long list, but don't panic. You have a lot of time and individual parts of your life can count in several categories: playing in a soccer league is both a physical pursuit and a team competition and is a place to make friends; your long, hard project could produce the thing you still use years later. The hard work, the taking care of others, the winning, and the losing will all help you to learn who you are.

And finally:

• Work on something you suck at until you no longer suck. Learn that you can overcome your own limits, and what that requires of you. (If you do the rest you'll probably hit this along the way somewhere.)

The importance of writing to your audience

This is basically a rant, and one that I've alluded to before when discussing the programmer documentation for Qt.

Professor Butts and the Self-Operating Napkin (1931)

Rube Goldberg

Non-trivial software libraries are complex beasts. They have a lot of moving parts. Things have to be done in particular ways. Things often have to be done in particular orders. They may (often do) have abstractions that must be understood. Many require a particular point-of-view on the problem domain to use them well (or indeed at all).

And all this stuff needs to be documented, and that documentation has to be useful to anyone you want to use the library.

If the documentation is written by the author(s) of the library, they face a common problem; I like to call it "the expert's disease".¹ It's the inability to recall what really is basic, simple, and obvious and what only seems that way because you are so familiar with the domain. If you don't know the difference you can write comprehensive and technically accurate documentation (and burn a lot of time on the effort) only to find that isn't of any use to your users even though they are smart and capable programmers.

Well, you find out if you're lucky. Otherwise you ship it in that condition.²

In the last two years I've fought with several libraries whose documentation makes perfect sense once you already know the tools but is less than helpful when you first meet it. I've slogged through the learning process for Qt (using Q&A sites and blogs mostly³), boost::qvm (frantic trial and error, mostly), and a domain-specific tool whose roots go back to the Reagan administration (more trial and error).

This week I've been investigating logging frameworks. My project could really use one (and I can't use Qt's framework because parts of this code need to run in non-Qt contexts, too). There are a lot out there for c++ and I don't have a good apriori reason to chose anything in particular. I started looking at boost::log first because we are already using boost so it wouldn't represent another dependency. However, this beast has more moving parts than qvm and it's documentation is nearly as bad. I'm abandoning it for now while I look at other things because the documentation is no help at all for what I want to do with it.

So that's the lesson I wanted to get to: if your documentation doesn't help people new to your project they may just walk away. And you have no one to blame but yourself.

---

¹ I developed the term while I was teaching physics. I came to full-time teaching after being a post-doctoral researcher for ten years (and given the length of my stint in grad school that means after being a full-time scientist for about fifteen years). A number of my ex-students still give me grief about the difficulty I had in getting the level right. Especially students from my first couple of years. On the other hand, they've stayed in touch which must mean I did something right around all the frustration and pain I caused them.

² On my current project at work we have finally managed to get the software into the hands of a tester (after about a year of my pushing the issue and for reasons entirely unrelated to my efforts). In addition to the bugs in the software he's identified (about a dozen, none of which were deep or difficult, thankfully) he's made it painfully clear how much our documentation suffered from the insider view. When a Ph.D. holder says your documentation is too dense and assumes too much you know you have a problem.

³ The Qt forums exist, of course, but they suffer from all the faults that Jeff and Joel talked about when they announced their big new idea in 2008. I can't recall and instance where I found a solution therein.

Little surprises #6: "Stop helping me!" edition

I got a little technological help today. Unfortunately it was not the helpful kind of help.

I attend some remote precense meeting that are mediated by a bespoke telephone-based system rather than by internet. This is driven in part by the institutional inertia that comes with having had the system for a long time, but mostly by security concerns. So I dial in, switch to speaker, and prop my cell phone against my monitor stand. It has to be a cell phone because we don't have a landline.

The conference involves our sponsors, so we keep a separate chat channel open for communication amungst ourselves.

On the conference today I finsihed answering a questions and waited for a response. And waited. And waited. And noticed that the line seemed deader than usual. I enquire on the chat channel if there is a problem with the conference, and a colleague (another parent of young children as it happens) writes back "I don't know, but someone has to get in the car seat to get a ice cream."

A horrified glance out the window is enough to confirm the worst. There is the babysitter loading the toddler into my wife's car. The phone has linked to the car and my colleagues and sponsers are hearing all about the proposed trip to get ice cream.

I want it to do that, but not now

I have no one to blame but myself. Unless I can blame Apple a little, too. That sounds good. I'll do that.

Usually I want the phone to pair with the car as soon as it comes on and without asking me if that is a good idea. But maybe the behavior should be varied when there is already a phone call in progress? Especially if I am already using the speaker. Or something. I'm frustrated that progress in AI has produced a lot of powerful spying tools and general creepiness, but damn all context sensitive decision making for real life.

Little surprises #5: qmake edition

The last edition of "Little surprises" reminded my of an incident from a while ago.

This one involves the input file format fo Qt's qmake tool.

Like a lot of other tools, the input language uses # to indicate a comment extending to the end of the line, and \ as the last charqacter on a line to indicate line continuation (the current logical line is extended to include the next file line). It also uses lists of white-space separated tokens as in:

SUBDIRS=external-library module1 module2 tests

If a list gets to have more than a few entries, it can be made much more readable by using line continuation:

SUBDIRS = external-library \
          module1 \
          module2 \
          tests   # Should anything go here?

A practice further encourage if you use Qt creator because the IDE builds lists of files and directories in this way.

What about the last line?

Should you put a line continuation marker on the last line of such a list?¹

Pros:

• consistency (which may appeal to any obsessive tendency that you might have)
• reduces the number of ways to make a mistake if you reoder the list

Cons

• you don't need to
• you need to worry about what is on the next line, introducing a new way to make mistakes

As I can be a little obsessive about consistency in code and insist on a whitespace line after a list like that anyway I choose to put the trailing continuation in there. The thing that I discovered is that in qmake the "need to worry about what goes on the next line" bit includes comments. That is, if I write

SUBDIRS = external-library \
          module1 \
          module2 \
          tests \
# Explanation of what I want to do next
ANOTHER_KEYWORD

The system will consider ANOTHER_KEYWORD as an item on the list.

I'm not aware that this is documented anywhere.

What?

From my point of view this the violates the principle of least surprise. If I append the comment to the line containing test, then I have a end-of-line comment and that should be the end of it. That said, if I think about actually parsing a file like this I can see two implementation strategies that would lead to this behavior as a side-effect,² and though they are not the ones I would have chosen I don't think they are dumb either.

Arguably this lends weight to the "don't do that" side of the argument about continuing the last line, but I have chosen instead keep the consistency and insist that such a list must be trailed by a empty line.

---

¹ This kind of question doesn't come up in my mind alone. Note that C++ enum offers explicit support for ending the list with a comma.

² (A) Multiple passes with comment elimination coming before line continuation and full line comments removed entirely. (B) Line-at-a-time parsing and full line comments are handled by silently loading the next line.

When did that happen?

Of course babies and toddlers grow fast. You know that from constantly having to retire clothes and shoes and bring new, bigger ones home amung other little hints. But you don't really notice the gradual change unless something focusses your attention.

There is a local store we go to occassionaly. My daughter loves the place. Partly becuase there is lots of room to run around but also because they have a giant gumball machine with a spiral ramp for the treats to roll down.¹ The store was fully closed down for a couple of months, what with the plague abroad on the land, but they are open again (short hours and low occupancy limits, but doing business).

It's been four or five months since we'd been there, but we went yesterday.

Everytime we've gone before I had to help her insert the coin² because she couldn't see the slot (positioned above here eyes and sunken behind the handle). Yesterday, she had it in there before I even finished saying "Thank you" to the person who changed a dollar for me.

She must have put on nearly three inches since our last visit.

---

¹ Thankfully she doesn't know you're suppose to chew them, she just thinks they are balls to roll and throw. We cherish and encourage this understanding of the situation.

² Her economic philosophy right now seems to be "Coins are good because I can put them in machines". And she seems to understand that quarters are more useful that way than other coins.

Little surprises #4: string concatenation edition

Another one that came up at work today. A method was behaving as if an argument was taking on it's default value even though there was an argument right there in the code.

Abstracted to it's bare essentials it starts with a method

class Thing {
   // c'tors and so on
   void method(const std::string & s1, 
               const std::string & s2 = "");
};

which was called in a section of code that had significant history

Thing thing;
thing.method("string"// oldWayToDeduceString(),
             "second string");

Do you see it?

Only one argument is being passed to Thing::method due to string concatenation and the comma that used to separate the first argument from the second being commented out. The compiler doesn't complain because the second argument takes on it's default value.

The real failure here (and it was mine, if you really need to know) was not removing the commented code after testing and before committing this code the last time 'round.

Wordo of the day

And someone else's for once. Anyway, seen in my news feed:

fiend of the court

in a context where it is clearly meant to be the customary translation of amicus curiae.

A little amused poking around shows that the diabolical version has occassionally been used with malice aforethought. I think I'll have a look at some of those.

Question of the day

Do you ever wonder what kind of barbarian we're raising?

Un-natural scripts

“I liken starting one’s computing career with Unix, say as an under-graduate, to being born in East Africa. It is intolerably hot, your body is covered with lice and flies, you are malnourished and you suffer from numerous curable diseases. But, as far as young East Africans can tell, this is simply the natural condition and they live within it. By the time they find out differently, it is too late. They already think that the writing of shell scripts is a natural act.”

— Ken Pier, Xerox PARC

As I've mentioned before I'm basically a unix guy. I believe I first encountered the OS in the mid eighties and first used it seriously in 1989 (a C course taught out of the New Testament with exercises coded on Xenix systems). For all that, I'm not unaware of the tradeoffs involved and find The UNIX-HATERS Handbook (the sourse of the quote) to be both ammusing and informative.

However, my current professional project is cross-platform by customer requirement: it needs to run on Windows as well as Linux. So this is the perfect opportunity to upgrade my skills in Python for all those tasks where I might have written a shell script. Right?

But the habit of thinking in terms of Unix utilities doesn't go away easily.

Last week I needed to prepend some text to a large number of source files. My project is nearing its first delivery to the customer and, being new to both contract programming and project leadership, I started the work without putting the usual legal boilerplate in all the source files.¹ Hundreds of them. Obviously some kind of scripted approach was in order.

The issue is while appending is trivial in the Unix file model, there is no OS level support for insertion before the end (the authors of TUHH are laughing up a storm at this point). You must either employ a temporary file or read the whole file into memory before beginning. Anything involving temporary files is hard to get right (especially if you are worried about security which probably doesn't apply here but it makes me cautious all the time), and reading all of an input into memory is a different kind of risk (again with known inputs it's not a huge risk but thinking of these things is part of the job, right?).

This is the point at which my memory throws up a bit of useful Unix lore: sed like several other old tools has a "in place" editing mode meaning that someone else has already dealt with the hard parts. As far as my searching could tell, python doesn't provide a nice wrapper for this.

So I wrote this little tool in /bin/sh and sed. It's not part of our deliverables so it doesn't have to be cross-platform. And that leaves only two issues: a few of the files are UTF-8 with BOM instead of ASCII and actually getting sed to prepend.

UTF

Frankly I agree with people who argue that every text tool should be UTF aware. But that doesn't mean that those old Unix utilities are. In this case it was few enough file to simply handle those with manual cut-n-paste, but it leaves the mystery of how those dozen files got that way when most of the code base is plain ASCII.

Being clever with sed

If you just try the obvious sed '1r license.txt' input_file.cpp you get the license after the first line (which is what you want on anything using a shebang, so all is not lost). The next step is "OK, fine, let's make that '0r license.txt'", but sed doesn't support that address (or maybe some do, but the one I was using doesn't).

So you have to get clever. What I ended up with looks like this (only without the explanatory comments after the line-continuation marker which only some shells support):

sed -i.old \              # Edit in place but keep a backup
    -n \                  # No automatic output
    -e "1h" \             # Hold the first line
    -e "1r license.txt" \ # Insert the license at line 1
    -e "2x" \             # On line 2, swap hold and pattern space ...
    -e "2G" \             # ... then append hold to pattern (pattern space becomes Line1\nLine2)
    -e "1!p" \            # All lines but 1 print the pattern (which includes line 1 on line 2).
    input_file.cpp

---

¹ Qt Creator supports adding such blurbs automatically at the time it generates files skeletons for you, but not adding them at a later date.

Bureaucracies and Language

Bureaucracies, by their nature, need to make precisely deliniated distinctions and they need to distill those distinctions down to sinlge words (for use in reports, questionaires, statistics and so on). Where existing usage does not support exactly the distinction they need they simply assign the denotation they need to an existing word (effectively re-defining it or at least imposing artifical precision on a word that cared some natuural ambiguity).

To chose an example that came up in the public conversation in the US following Hurricane Katrina, the UN defines "refugee" more closely that most dictionaries because they need to distinguish people diplaced over national border from those who are seeking refuge inside the country where their legal situation is well established. This insistance that the word only applies to those whove crossed national borders is stronger than the original organic use of the word.

This last week we learned that the US National Parks Service defines "tear gas" to exclude pepper bombs. I'm not sure what (if any) purpose is served by this distinction, but it certainly isn't one in general use. It's purely a bureaucratic denotation.

As I indicaed above, I understand that such distinctions may serve a purpose for the agency that makes them, but I feel quite strongly that there is no obligation for actual flesh and blood humans to give a damn. Real people should use words with their real meanings and should feel free to laugh at any jobsworth¹ who sugggests that their organization's internal usage is more correct than the general population's.

---

¹ A word I'd forgotten about until I saw it in a opinion peice from a British paper this week.

Counterproductive software policy

In the Time Before (tm) I used to make regular pilgrimages to one of my employer's offices nearly an hour away. On these and other drives where I have signficant blocks of time alone I listen to podcasts in the car.¹ Today I was dispatched to do the once-a-month in-person shopping that our household needs despite copious (not to say excessive) use of internet shopping and local grocery delivery. I feel there is enough driving in that to justify story time.

Now the default podcast app on my iPhone doesn't interface well with my car's bluetooth audio system. Sometimes it thinks it's connected and running but no sounds comes from any device and it can't be fixed without rebooting the phone. So I tried Radio Public which isn't ideal but works a lot better with the car.²

Of course, sometime I start driving without remembering to start the app. So ... "Hey, creepy Siri." ... "Start Radio Public." And then Siri tells me that she can't do that while I'm driving.

Of course, I'm trying to do it by voice because I'm driving: the point is to keep my hands on the wheel and my eyes on the road.

I appreciate that Apple is concerned about the role of phones in distracted driving, but surely voice control is less distracting (not to mention more leagally acceptable) than diddling the screen with your fingers.³ This particular policy seem to be defeat its own purpose.

Anyone know of a podcast app for iThings that I can control by voice while driving?

---

¹ Escape Pod, Clarkesworld, and Drabblecast if you care.

² It would be very easy to convince me that this is the car's fault, but so what? I can't easily upgrade the car's entertainment system.

³ I know, I know. Don't call you Shirley.

Worlds in conflict

The world of adults has fragile things of significant monetary or emotional value. It has collections of small related objects that need to stay together. It has complex systems that need to be operated correctly lest they break. The world of a couple who've been together but childless for more than twenty years has a lot of these things tucked into nooks a crannies all over the place.

The world of toddlers has none of these things. Instead it has fascinating percussion instruments and whatsits that can be swung around. It has piles and full containers that are fun to scatter across the floor and abandon in search of the next entertainment. It has buttons and dials and levers that cry out to be fully exercised as they come to attention. And of course it has lots of climbing challenges and various mobile objects that can be used to surmount them.

Which means that the adult world does not have enough secure high-places for all the things that need keeping out of the way of an agile toddler.

I short: we're losing.

Why I still write low-level tests

I've seen on-line a few places where people recommend against writing "unit tests". [1 2 3 4]

At first blush that seems like a strange recommendation as writing tests is a pillar of modern software engineering, isn't it? But when you read on you find that they have a various well thought out—if rather more specific—recommendations.¹

To take the TDD argument for instance, you are working to meet a set of predefined tests that tell you if your program is working. In other words the the test which are driving the design are (by their nature) acceptance tests. Such tests (should!) specify the user's view of what the program does but not the low-level implementation details. Testing the implementation details locks you into a particular implementation and you don't want to do that.

And people have other good reasons for their advice, too. Traditional low-level tests give you very little coverage per test, don't test the interactions of components, and don't mimic user behavior. Put into less inflammatory language, the recommendation are more like

• Don't lock yourself into a particular implementation with your tests.
• Write tests with non-trivial coverage
• Write tests that mirror actual use patterns

All of which effectively push against testing low-level code units.

Why I not going to stop writing low-level tests

I am employed as a scientific programmer, which means that the codes I work on tend to have fiddly requirements for reproducibility and precision in floating point calculations, tend to execute moderately complex decision making,² and are often pretty deeply layered. None of that is ideal, but the domain drives codes in that direction.

So, a common enough scenario in my end of the business is

1. Write a set of utility routines handling a domain specific computation.
2. Start using them to solve parts of your main problem. End up with several different bits of code that depend on the utility code and seem to work.
3. Sometime later, make a change in one of the upper layers and a bug emerges.
4. Waste a lot of time looking at the new code without luck.
5. Eventually trace the issue to a poorly handled corner or edge case in the utility code.
6. Bang head on desk.

Much of this pain can be avoided by knowing that you can reason about the high level code because you know that the low-level code does what it says on the can. Which you accomplish by having a pretty exhaustive set of tests for the utility code.

Re-framing

As above, I'm going to dig into the arguments related to TDD as an example.

In the farming of some TDD proponents that would be testing a "implementation detail" and a bad thing. I assert that the problem isn't testing low-level details, it's treating the tests of low-level details as having the same immutable character as the acceptance tests (that is, treating them as enforcing a particular implementation instead of understanding that they test a particular implementation and will be replaced if you change the implementation).

We can frame this is a couple of way

1. We can maintain a categorical separation between acceptance tests that state what we have to accomplish and implementation tests which provide assurance that the-way-were-are-doing-it-right-now is working. The former are non-negotiable and the latter are fungible and exist only as long as the related implementation exists.
2. We can conceive each sub-module as it's own TDD project with it's own tests, but be prepared to swap underlying modules if we want to (which means swapping the tests along the way because the test go with the module).

Fundamentally both points of view accomplish the same thing. Which framing is most appropriate depends a bit on the structure of your code and how your build system works.³

Either way I end up with two kinds of tests.

And when I look at the other arguments I end up with the same conclusions:

I still need my low-level tests, I just need some other tests as well.

And we have names for them. I've already used "acceptance tests" which are typically end-to-end or nearly, generally have relatively large coverage per test, and often mimic actual use patterns. "Integration tests" check that components play well together and test a lot more code in one go than traditional "unit" tests. "Regression tests" help your audit changes in your user-visible output, tend to have high coverage, and are generally driven by use cases.

In any case "don't write low-level test" is the wrong lesson. The lesson you should take here is that tests come in multiple kinds with multiple purposes. Some tests are for the clients, some are for the developers, and some are for both. You need to figure out which are appropriate for your project and include all that qualify.

---

¹ The authors can be forgiven for phrasing the title in so jarring a way. We all hate click-bait but we do want to draw the audience in.

² Really complex algorithms are usually limited to a few places and almost always exists in libraries rather than written bespoke for each project.

³ I suppose that if your Process-with-a-capital-P is Test-Driven Design you want the build system to enforce the distinction for you so that you can't just wipe away an acceptance test thinking it belongs to a implementation detail.

Paradoxical success

Many parts of the US are starting to significantly relax public health restrictions related to the virus. To judge from the infection rate data one finds batted about the web some of them are being more responsible than others.

My particular corner of the world is in a weird place. The state has been largely successful in slowing the spread of the virus. Excepting one locality there haven't been any high-intensity outbreaks and the hospitals have not been overwhelmed.

Yeah! Go us! Whoo! Hoo!

But with something as contagious as this a large fraction of the population is going to get it eventually, so slowing the spread means it hangs around longer. Our hospitalization rate is actually still rising, and we're not ready for large scale relaxation of the restrictions. The Governor's existing order has been tweaked and extended.

My employer has established rules for returning work at the office (though they would be physically challenging at the actual facilities we have in this state) but we are allowed and encouraged to continue with full time telework.

On the sleep habits of celestial beings

Judging from my daughter's behavior I conclude that her shoulder-devil is a night-owl and the angelic counterpart is a early riser.

Chosing a new build system

I've started a couple of little software projects which might evolve into something worth putting out in public. Indeed, if I'm committed to open source approaches I should probably put them out there fairly early on. But even with the bare bones, just-preparing-the-ground state of these projects I'd like potential users and contributors to have a smooth path to seeing what little functionality is actually done. Which means having a well designed and at least somewhat cross-platform build.

Now, I'm a Unix guy. I can write a simple makefile starting from a bare editor without breaking a sweat. I've maintained or extended the make-based build on projects with several hundred-thousand lines of code and many hundreds of source files targeting Unix, Windows, and MacOS. I respect make. But...I've maintained or extended the make-based build on projects with several hundred-thousand lines of code and many hundreds of source files targeting Unix, Windows, and MacOS. So I know how hard it is to get a make-based build to scale up without hiccups and what a pain it is to get it to do cross-platform smoothly and reliably. There ought to be something better.

Hat-tip to Randall Monroe

I've started looking into what options are available. If you've asked this question any time in the last fifteen years or so you know exactly what I've found.

Lots and lots of contenders, lots and lots of reviews and comparisons, and lots and lots of opinions. But nothing like a consensus. A great many of the contenders are stagnant or abandoned, while others never seem to have evolved beyond some niche or another. I suppose we can conclude that a lot of people think this issue is a pain point and that the problem is harder than it looks. Certainly building has a lot of inherent complexity and and cross-platform building is more complex still.

I'm tempted by meson (and I really appreciate Evan Martin's philosophical position about separating a fast DAG walking component from a intelligent decision making component), but I'm far from confident that it is the best choice. I'm also tempted by tup simply because I'm impressed by the improved asymptotic performance of the underlying process¹ but I think the syntax is a little wonky.

Any thoughts?

Aside: If any one of my handful of occasional readers thinks that cmake is the obvious choice, I'll warn you that you're facing an uphill battle to convince me.² I've had the dubious pleasure of getting to tweak the build of projects supported by that thing and I like it rather less than qmake (which I don't like much and don't consider a contender for projects not involving Qt).

---

¹ There is an aspect of the linked paper that I really don't like. A couple of times the authors criticize Recursive Make Considered Harmful for insisting that you have to have the full DAG for correct build and they claim that they've proved that this isn't true. Except that they do have and use the full graph (remember that they have to cache the graph). They've found a way to avoid walking the whole thing (by using the change list to select the affected sub-trees), but they have to have it to start with or they have to build it before they begin. This is basically a problem with language and the way they form they claims, but it is also basically wrong. When writing a paper you certainly want to promote your work but you shouldn't over claim which is exactly what they've done there.

² I've read many claims that the syntax of cmake 3 is so much better than that of earlier editions that people who dismissed the previous version should really look again. I find that scary because I've only ever worked with the "new" syntax and it's more than enough to put me off. Can you get Stockholm syndrome from working with difficult software tools?

I didn't know that I could do that!

So, I'm watching some videos on c++. Call it continuing education.

Now, I've never formally studied c++. It wasn't part of my education, and I learned it "on the job". I have tried to read and learn as I encountered things I didn't understand, and I like to think I know a lot of things about the language. I mean, I even got a gold tag badge for it on Stack Overflow. Surely I'm not completely ignorant. Right?

One of the talks I'm watching is highlighting some parts of the Core Guidelines. The presenter brings up a case that I actually faced recently.

1. You have a class with a const accessor method that returns the result of a computations (i.e. is not accessing a feature of the current implementation but a feature of the model that can be deduced from stored elements of the implementation)
2. That deduction is expensive enough that you want to cache the result.
3. You run into a conflict between the need to update the value of the cache and the desire to have the accessor marked const because it isn't changing any feature of the underlying model.

I'm happy to say that I didn't do the very bad thing (casting away the const), but I did lose the const marking on the accessor because I didn't know about mutable.

Crap.

Learn something every day.

Human exception handlers

I've done a pretty good job of not contributing to StackExchange over the last few months, but I haven't been able to break myself of the habit of looking in. At first I could claim to be holding out hope that I would see something on meta.stackexchange that would let me change my mind, but at this point it's pretty clear that I just don't want to give it all up yet.

So I see things that, in the past, would have required me to do something and I have to sit on my hands instead. (Aside: they still haven't pulled my diamond. It's been more than three months!)

Today I see a situation that requires, in Jeff's words, exception handling: there is a user on a site who is posting dozens of pretty bad answers to old questions. By "pretty bad" I mean a range of things including "more wrong than right" through "pointless and not adding anything" to "possibly a useful insight but so badly phrased as to be more confusing than helpful". And they are putting up several of these an hour (I guess they have some time at home...). It's not being very productive on the rep front but in the past two days they've managed to net more than one hundred rep this way (averaging a bit over two points per post).

It's not something a ordinary user can do much about because any attempt to do so would be targeted voting or a sustained pattern of negative comments (which is to say "not nice"). It needs a moderator to step in a put the brakes on it. Stack Exchange moderators can use the contact users and suspension tools for on-going poor quality contributions, and this seems like a exemplar for the need.

A lot of things that call for moderator action are like this: a user exhibits a behavior that could normally be treated by other mechanisms with unusual intensity or in a manner to reduce the effectivness of the usual controls.

Color me suspicious...

My employer strives to maintain a pretty high level of IT security. Our issued computing devices have encryption at rest and two factor sign in. The network in our offices pipe everything through a VPN to corporate and we're not to connect the laptops to public network excepting that we enable the VPN which requires two-factor sign-in. Accessing corporate mail through the web interface requires two factor identification. And so on.

It's probably not perfect,¹ but there are clearly some professionals thinking hard about this issue back at the big office, which is reassuring.²

They've recently added the on-line time reporting system to the two-factor menagerie.³

But here's the thing: when I sign into the OS on my laptop, the VPN, and the webmail, I enter the code off the two-factor fob first and my password second. This new system is doing it the other way 'round. I don't consider myself expert enough on these issues to have an opinion about which way is better, but I'm pretty sure that doing it both ways can't be right.

---

¹ As a working postulate I think "It's never going to be perfect" is a good way to understand IT security.

² Actually, the bit where I don't have root on my development machine is pretty &^%$ annoying. But IT isn't being unreasonable for it's own sake on this: I can have root on a virtual machine and develop there as long as the virtual machine is behind the firewall on the "real" one.

³ And a good time, too. It's been getting a lot more access from outside the office lately.