Side Quest: Infrastructure Build-Out

My hobby programming project stalled out. Again. Because I still didn't want to npm install.

This time the issue was a test harness.

The project is a wannabe game engine. Or at least a wannabe animation engine for now: user interaction is still in the works. I had gotten far enough to be bothered by the ways the physics isn't working right.¹ But the code is growing in size, and fixing the physics code will be fiddly. That is, after all, why it doesn't work right as things stand. Unit tests are part of my process for handing that kind of fiddly.

And when you search for pure JavaScript unit testing frameworks, they all request the forbidden incantation as step one. For the same perfectly valid reasons tall the language server did, I'm sure.

Now, I've written testing frameworks of various kinds in the past. A set of tests in Fortran for my dissertation analysis after the second attempt collapsed under it's own weight; a really kludgy mess of c-preprocessor and GNU-make just because I thought I was clever,² and a command-line end-to-end driver at work. None of them are professional or polished but they all work in some sense, and that teaches a lesson: it doesn't have to be fancy to work.

With that sort of thing in mind I conceived a vague notion of a minimal test harness that would write its output into a webpage.³ Obviously I'm not the first person to have that idea, and a little searching got me an example to use as inspiration. Thanks, Alex!

Only I couldn't leave well enough alone. I made it part of my "learn to program with AI" effort, worked with Gemma4 to give it a fluent interface despite my misgivings about that whole idea and thus was TestOf born. Maybe it'll be useful for someone else.

---

¹ Embarrassing for a physicist, I can assure you.

² Please use something better than that!

³ We are talking JavaScript, after all.

Paging Generative Freud. Dr Freud, you are needed in the psych ward

I had a brainstorm today about a possible prompt to test the edges of LLM conpetence. It's longer prompt than most of the ones I use and demands a longer answer, so context lengths have to be expanded and it runs slowly on available hardware. As a result I've only begun investigating it.

But it triggered an idea for another "off the wall humor" type question which is quite succinct. The thing is that when I started asking models about that one I got a surprise, and thinking it might be perculiar to a particular model family I tried some others; and the surprise persisted. What. The. Absolute. Heck?

There is an old saw that a boat is a hole in the water that you throw money into, which is not the core of my prompt but has a similar flavor. What I'm actually asking the models to react to is a pair of definitions of two classes of boats in terms of how they make you broke. If you've spent anytime in boat-owning circles you've probably heard it. The joke is funny because (a) it acknowleges that boats are conspicuously expecnsive to own, and (b) it distinguishes between motor boats and sail bots, and you can order the definitions so your audience feels good about not being in one category until you hit them with the definition for the class of boat they own. Then it's rueful smiles all 'round.

Only ... model after model has wanted to interpret it as some kind of financial metephor. The difference between spending big for lots of velocity and binding your time doing things right in the hopes of achieving a better outcome. Or something like that.

Where are they getting that? Does their training data inclue lots of articles from the popular business press? Or something?

Weird.

Oddly it seems to come outstronger when models are allowed to reason.

Does any one know?

Did Sir Pterry every address the question of which divinty attends to tangled cables? I mean, it seems likely to me that the duty would fall to Anoia. Right?

Yes. Yes, she is.

When Geddy Lee and Alex Lifeson announced, last year, that they had chosen a drummer to join them for a few more Rush shows, I was at once squealingly excited and deeply fearful. I mean, I knew intellectually that there is no way the guys would agree to go out with a less than fantastic drummer, but the emotions do not play by the same rules as reason. Well, I found a few videos and a few expert opinions, and that allayed much of my worry. And then a few months ago we go the video from the Juno awards and that was very promising. But there were a few lingering, irrational concerns. Was she good enough?

Well, I'm back from the opening show.¹

There is a lot that could be said about things like song selection (we go La Villa Strangiato and YYZ!), and the tributes to Neil (sniffling back tears), and how incredible Time Stand Still sounds with Aimee Mann harmonizing on stage, and on and on but I'll let others cover that.

The answer is unambiguous.

All the delicacy and subtlety on display in the videos I've seen of her earlier work was there, but when Neil's drum parts demand The Beast she's got that too.² The crowd's reaction to the first big fill in Tom Sawyer was emphatic, and Geddy's reaction to that was something to witness.

---

¹ Thanks to Mrs. NoSwampCoolers for deciding that this was going to happen while tickets were still available.

² And while she looks completely different from Neil, in the hard parts her face carried the same grim determination. But when the drum chart was merely fast and demanding joy and satisfaction would break through.

A modest UI proposal

Not-good-at-tech mode

I begin to suspect that certain apps could use a new, over-arching control. One that (a) increases the size and weight of all text elements, (b) forces the facility into the lowest-common-denominator do-what-it-says-on-the-tin mode, and (c) hides all the complex controls. It should be easy to find and disable, of course, as we're not trying to take options away from anyone: just give them an easy button.

Though perhaps I should reconsider my proposed name for the feature...

---

Full disclosure: I asked Gemma4:31b to write the CSS and HTML for the sample.

I wonder what the experts would think of my take?

Learned about "property based testing" today. A little. Initial impression: it's (at least in part) fuzzing at the unit test or integration test level instead of the end-to-end level.

Also I already do something like a lite version of that in a few cases. Indeed, I've been trying to formulate a good blog post about one of my practices that I don't see much in other discussions of unit testing. Alas, it seems there's a whole community out there stealing my thunder.

Sometimes one facepalm is just not enough

Still working on my little JavaScript side-project as a way of learning AI workflows. Still not using AI for a lot of code-gen but instead for friction reduction. It's going fine, thanks. I mean, the last few weeks haven't seem many new features, but instead a couple of re-factoring passes intended to enable big things in the future, but I count that as progress.

Anyway, while thinking about the evolution I had a question. I was fairly sure the language would do things the "right" way, but felt the need to test, nonetheless. And being an old guy, I wrote the test code by hand:

const primes = [2, 3, 4, 5, 7, 11, 13, 17];
const fibonacci = [0, 1, 1, 2, 3, 5, 8, 13];
let combined = [...primes, ...fibonacci];
combined.sort();
console.log(`primes: ${primes}`);
console.log(`fibonacci: ${fibonacci}`);
console.log(`combined: ${combined}`);

the point of which was just to assure myself that sorting the combined array would not scramble the source arrays. I would have been very surprised if it had, but I had to check.

That said, the results were not what I expected:

primes: 2,3,4,5,7,11,13,17
fibonacci: 0,1,1,2,3,5,8,13
combined: 0,1,1,11,13,13,17,2,2,3,3,4,5,5,7,8

Wat?

An array filled entirely with numbers has been sorted lexically according to their naive string values.

And on reflection, that's not terribly surprising in a language, like JavaScript, that is both dynamically typed and quite compact because it is the prescription most likely to work in most situations. But it sure came as a surprise to a guy used to strongly typed systems intended for systems and scientific work.