2021-07-22

Reading from files, single responsibility principle, and testability

Say that your program needs to read some kind of structured data from a file. In a object oriented language the obvious thing to do is encapsulate the parser and the storage of its results in a class. Perhaps something like:

class DataReader { public: DataReader(std::string filename) { std::ifstream in("filename") if ( in.good() ) { parse(in); } } //... }

Indeed some of the legacy code my project calls had one of these as the main access point for a bunch of important functionality, and I've been asked to extend it and authorized to do some re-factoring along the way. Whoohoo!

But I can't afford to break existing code (or at least I have to know what I'm breaking and why...), which means I need some detailed test. Yes. It doesn't have a test suite. I mentioned that it was legacy code, didn't I? And that is where the problems start, because how do you write a self-contained test for DataReader?

  • Actually creating a (or more likely several) test files break self-containment and means your test code needs to know where to look for the files, and test can fail for reason that have nothing to do with DataReader. Yuck!
  • You could encode the test input as strings in your tests, create and write a temporary file, run the test on that file, then clean up the temporary file. That solves the self-containment and "where to find it" problems and doesn't clutter up the disk with a large number of similar test files. But it still breaks if there is a problem with the disk. More problematically all the set-up and tear-down code is a place to write bugs, and interferes with the clarity of your test. Still pretty bad.
  • The third option is to refactor to make it clear that opening the file and dealing with the contents are separate "thing"s for the purposes of the Single Responsibility Principle. The class should take a stream not a filename. Now to avoid breaking existing code we can relax our pedantic insistence of separation of concerns by leaving the existing c'tor as a convenience but defering most of the work to a new c'tor that takes a stream. Something like: class DataReader { public: DataReader(std::istream in) { if (in.good()) { parse(in); } } DataReader(std::string filename) : Datareader(std::ifstream(filename)) {} //... } Then we use a std::istringstream in the test eliding much of the setup and all of the tear-down.

I shouldn't have to ?*(<|#% register to do that!

The house we bought out here is not only bigger than any we've had before but it is bigger than we actually needed. There just wasn't anything smaller that met our needs on the market when we had to buy, so we had to take what was available.

The shear size of the place has been a problem from a WiFi perspective.1 There has been signal and bandwidth everywhere, but in some corners both sometimes drop pretty low. One of those corners is the guest bedroom which is where I work-from-home.

Well, complaints from my better half finally got me off my duff. I bought a couple of meshing extenders. Nice ones that maintain the same SSID none-the-less. The instructions included in the box tell me to download the app and use it to configure the widgets. I grumbled a bit, but fine. Except that once I have the app I learn that it requires me to set up an account with the manufacturer's web site.

Are you kidding me? Why would I want to do that?

It's a serious question. What makes them think I want to register?

In a word: No!

Now, it turns out these widgets support a WPS pairing mechanism.2

However, that wasn't mentioned in the material in the box and was pushed to the very bottom of the help web page: a tiny paragraph of plain text under screen-fulls of colorful pictures and enumerated lists of steps supporting the two ways to register with them to get it done.

I can only assume that this is done in bad faith. It is a malicious act in the service of the company to the detriment (admittedly small) of their customers and it makes the world a tiny bit worse.

Worse, I think the printed docs included the no-registration instructions at some time (the controls for it are labeled on the diagram but never referred to). At some point a marketing jerk told the tech writers to make the documentation worse and the tech writers complied, but they didn't take the time to scrub all the traces of their lost efforts.


1 The house actually has some kind of wire-in-the-wall-and-RJ45-jacks system from Honeywell pre-installed, but we didn't get the printed docs from the previous owner and I have never figured it out. Naively it looks like some kind of star topology that assumes you're putting the control next the main panel (i.e in the laundry room). So we're living with wireless. Besides there are relatively few ports and most of them are in inconvenient places.

2 Just as well for them. I'd have sent them back rather than register. And so should you. Fight this nonsense.

2021-07-10

Well, obviously...

We've had my in-laws here for the last week with our nephew for "Camp Cousins" to ensure that the youg'uns know one another. Lots of activities around the house: arts, crafts, games, and even an intoduction to Roblox development for the eight year old (which is why it had to be here and not in California where the in-laws live); and trips out to the zoo, aquarium, science museum, and even putt-putt.

I had shirts made. But they took longer than expected to come, so we didn't get them until Thursday and couldn't wear the to the zoo for the planned pictures. We rescheduled that for today (the last day). When the time came we headed out to the (new) planned location (where we could include the same mountains in the background as I had put on the shirts), only to find that it was starting rain (out of season) even as the dust storm continued (very much in season).

Rain and dust storm. Of course.

2021-06-28

Oh, no! It's worse than that!

Me
[Rants for a while about a desired mechanism for solving a technical project-management issue at work]
Better-half
But there's no way to do that?
Me
Oh, no! It's worse than that! There are at least six ways!

Seriously, I'm spending too much time reading about `git-subwhatever`, build-system support for external dependencies, and even dependency management systems (though this really shouldn't require going to those lengths). And I'm really thinking about suggesting we do this the old fashioned way.1


1 Take the bits we want to factor out and share between projects, make each it's own project that builds a library, and link the libraries in the overlying projects. Done and dusted. Except for the hassle of getting a new hire up and running; and versioning issues; and remembering where each bit actually exists when you need to work on it; and deciding on, setting up, and enforcing commit privileges for each library; and extra deployment complexity for software we ship. But at least those are all known problems.

2021-05-14

Well aren't you just Mr. Subtle today?

I've been getting some unsolicited political texts recently. Here's a sampling:1

Trump won't ask again: Will you join him on his new social site. Friend? You have 1hr to stand w/ Trump until link closes for good [minified link redacted] stop=end
Have you abandoned Trump, Friend?! He launched his new site & you haven't told him you'll join. You have 1hr to stand w/ Trump [minified link redacted] stop=end
Trump won't ask again: Will you join him on his new social site, Susan? You have 1hr to stand w/ Trump until link closes for good [minified link redacted] stop=end
We've texted 9x & you still haven't answered Trump: WIll you join his new site, Susan? 20min or he'll know you've sided w/ Dems: [minified link redacted] stop=end
It's Steve Scalise & I'm wondering if we lost you, Susan. True Patriots will steps up. Socialists will ignore this. Which are you? [minified link redacted] stop=end

And so on. In case you're wondering, my name isn't Susan nor does that name belong to anyone I've shared a mobile plan or a mailing address with, so the contact database they are using is suspect.

The main thing that stands out to me here is the "subtle" menace in much of the phrasing here. Oh, it's ambiguous enough to provide cover against legal action—your basic plausible deniability—but it is also clear in the natural reading. The semantics here are those of bullies and crooks.


1 Before last year's election I was getting a bunch from pro-democratic sources. Those were run of the mill electioneering. The partisan selection and nature of this spam changed recently.

2021-05-10

Can't I just make a picture of this 3D model?

It sounds like a simple request: take this set of object models and make a set of pictures so we can vdiff them. No problem, right?

On one level that is right: there are a lot of tools out there than can draw you a picture of a 3D model.

But there are some unstated requirements:

  • These represent the same object at different points in the evolution of a physical effect, so we want to image them all from the same point using the same viewport and the same lighting. Maybe even generate a movie, but that's a bonus.
  • There are a lot of these objects because they are are generated by a scripted process, so we'd like to be able to initiate and control the rendering from the command line.
  • If our renderer uses a light model more sophisticated than pure ambient light (and it probably should, even if we don't need it to be particularly sophisticated), we want to control the lighting.
  • We'd actually like to add a simple legend to the scene, though this a "may" item that can be dispensed with.

And now the problem is actually pretty hard, and there are many fewer tools that offer the facilities we need.

Let's take a closer look at the requirements.

Scene description

The first thing to know is that a plain object model (in wavefront (.obj) or stl format) isn't enough to define a picture. What direction do you see it from? How much of the frame does it fill? Which way is "up" in the frame? How is it lit (and for that matter, how do we model lighting)? In what color? And so on ad nauseum.

So we need some kind of additional information. We call the combination of the object model(s) and all that ancillary information a "scene description", and it is written in a scene description language or format.

There are a lot of scene description languages out there (in many cases used by a single tool), though a small number are reasonable common. From today's web browing it looks like COLLADA, the RenderMan format (.rbi), and blender's proprietary and undocumented .blend files are the big players.

Command-line interface

This operation is part of a fairly complex workflow that I expect execute a few score to a few hundred times. It really should be scripted, but doesn't justify writing a custom tool unless I can just plumb together some existing bits. For that to be practical we need a renderer with a command-line interface (because this means that someone else has already done the bulk of the work).

In an ideal world I could write something like:

 renderer -i thingy_2.5s.obj \
        -c camera.txt -l lights.txt \
        --quality=3 --resolution=1920x1080

and get thingy_2.5s.png out. Then I could just loop over the available input files (re-using the scene description elements) and get my perfectly aligned set of images.

Where that leaves us

In a lot of ways the obvious choice is blender (not withstanding that this is a hugely heavy tool for the job), but it wants the scene description in an undocumented format which I can't generate outside of the program. Strike the obvious solution.

Now both COLLADA and RenderMan are documented, but as far as I've learned so far neither one is modular in the sense that I can define the camera and lights in a separate file from the 3D model and neither one lets me say "Use this wavefront file with rotation matrix M and translation T." in the singular file. In principle I could rework my code that is generating .obj files to produce .rbi instead but that would mean the producing code would need to know about my choice of camera and lights which is simply in the wrong domain for that code. Argh!

So, at least in my context, the answer to the title question seems to be "No, you can't.", which seems to be a bit of free real estate in the software utility market. Another task to add to my queue of projects that are not getting done due to the demands of my home life. Sigh.

Update 11-May: It turns out that RenderMan format does support separate inclusion in the form a "Entity files" and a ##Include directive. This represents a path forward, though it means supporting mesh output in a new format.

Update 14-May: The specification says:

Note that the Include keyword itself does not cause the inclusion of the specified file. Aswith all structural hints, theIncludekeyword serves only as a special hint for rendermanagement systems. As such, the Include keyword should only be used if render management facilities are known to exist.

Though it appears that I can provide the adequate level of "render management facilities" with a simple tool that identifies the keyword. Better still the renderer I'm looking at (aqsis) is willing to accept the standard input so it becomes possible to write something like includer lights_and_camers.rib | aqsis .

2021-05-02

Yep. Naming things is hard.

There are only two hard problems in computer science: cache invalidation and naming things.
Phil Karlton1

So I've been reaing Uncle Bob's Clean Architecture as part of my continuing education. I'm in the bit where he's talking about SOLID in gneral and trying to explain the Single Responsibility Principle in particular. Uncle Bob admits that it isn't well named. Nice mea culpa. Still, it's not too bad a job. Especially if you compare it to the naming of the principle widely know as RAII.

Anyway, moving on to defining this thing Uncle Bob gives the original statement of the principle as:

A module should have one, and only one, reason to change.

Which he immediately admits isn't particularly clear, so he then suggests the intermeidate form:

A module should be responsible to one, and only one, user or stackholder.

This is also less than perfect as there will often be a group of related users or stackholder who hold the module's specification collectively. To remedy that Uncle Bob tries:

A module should be responsible to one, and only one, actor.

Which is more pithy. Alas he needed a separate sentence to nail down the meaning of "actor" in this context.

It seems to me that "faction" might be a better word here. Admittedly there is a contation of conflict or at least competition that comes with that choice. But, honestly, the different groups of stackholders who rely on a medium to large scale software project are in competition for programmer hours if nothing else and they often have to hash out conflicting interests in specifications as well.


1 Yes, I'm a big fan of the variation that includes off-by-one errors in the list of two things. But the original is insightful enough that Phil deserves plenty of recognition and it is hard to find a authoritative origin for the expanded version.