Side Quest: Infrastructure Build-Out

My hobby programming project stalled out. Again. Because I still didn't want to npm install.

This time the issue was a test harness.

The project is a wannabe game engine. Or at least a wannabe animation engine for now: user interaction is still in the works. I had gotten far enough to be bothered by the ways the physics isn't working right.¹ But the code is growing in size, and fixing the physics code will be fiddly. That is, after all, why it doesn't work right as things stand. Unit tests are part of my process for handing that kind of fiddly.

And when you search for pure JavaScript unit testing frameworks, they all request the forbidden incantation as step one. For the same perfectly valid reasons tall the language server did, I'm sure.

Now, I've written testing frameworks of various kinds in the past. A set of tests in Fortran for my dissertation analysis after the second attempt collapsed under it's own weight; a really kludgy mess of c-preprocessor and GNU-make just because I thought I was clever,² and a command-line end-to-end driver at work. None of them are professional or polished but they all work in some sense, and that teaches a lesson: it doesn't have to be fancy to work.

With that sort of thing in mind I conceived a vague notion of a minimal test harness that would write its output into a webpage.³ Obviously I'm not the first person to have that idea, and a little searching got me an example to use as inspiration. Thanks, Alex!

Only I couldn't leave well enough alone. I made it part of my "learn to program with AI" effort, worked with Gemma4 to give it a fluent interface despite my misgivings about that whole idea and thus was TestOf born. Maybe it'll be useful for someone else.

---

¹ Embarrassing for a physicist, I can assure you.

² Please use something better than that!

³ We are talking JavaScript, after all.

Paging Generative Freud. Dr Freud, you are needed in the psych ward

I had a brainstorm today about a possible prompt to test the edges of LLM conpetence. It's longer prompt than most of the ones I use and demands a longer answer, so context lengths have to be expanded and it runs slowly on available hardware. As a result I've only begun investigating it.

But it triggered an idea for another "off the wall humor" type question which is quite succinct. The thing is that when I started asking models about that one I got a surprise, and thinking it might be perculiar to a particular model family I tried some others; and the surprise persisted. What. The. Absolute. Heck?

There is an old saw that a boat is a hole in the water that you throw money into, which is not the core of my prompt but has a similar flavor. What I'm actually asking the models to react to is a pair of definitions of two classes of boats in terms of how they make you broke. If you've spent anytime in boat-owning circles you've probably heard it. The joke is funny because (a) it acknowleges that boats are conspicuously expecnsive to own, and (b) it distinguishes between motor boats and sail bots, and you can order the definitions so your audience feels good about not being in one category until you hit them with the definition for the class of boat they own. Then it's rueful smiles all 'round.

Only ... model after model has wanted to interpret it as some kind of financial metephor. The difference between spending big for lots of velocity and binding your time doing things right in the hopes of achieving a better outcome. Or something like that.

Where are they getting that? Does their training data inclue lots of articles from the popular business press? Or something?

Weird.

Oddly it seems to come outstronger when models are allowed to reason.