2024-03-05

Voice-assistant fails

Accumulated over the years, but I got another one the other day that triggered me.

Me:
[Navigating to a business in the US sowthwqest]
Creppy voice assistant (CVA):
In five-hundred feet, turn left on El Camino Real Street1.
Me:
[::Sighs::]


CVA:
[Interrupts a conversation in the car]
Me:
Hold your horse, [CVA].
CVA:
[Starts reading the Wikipeia article on the idiom]


Me:
[CVA], play 2112.
CVA:
Now playing two-thousand-one-hundred-twelve.
Me:
[::Fumes:: until the music sweeps me away]

It's all about context seneitivity. Or the lack thereof.


1 With "Real" pronounced as a single sylable. Of course.

2024-01-28

Career opportunity

Desperately seaking a licensned professional to tell us that we're making good parenting choices.

2024-01-25

Bringing C Structs into the C++ Lifetime Model

In addition to legacy code in our own projects, I sometimes "get" to work against libraries (legacy or modern written in plain C. Which is OK. I learned C a long time ago and I'm not intimidated by it, though it can take a while to get back into the right mindset. Of course, there are things I miss. Static polymorphism and namespaces, for instance, are pretty small conceptual changes with significant convenience factor for the programmer.1

Now, C++ has a reputation as being a dangerous language where it is easy write really broken code. That impression is not wrong, but it is incomplete: the lagnuage also offers features that support writing code that has enforced safety in some aspects. It's not trivial and it takes both discipline and some understanding of how the features work, but in my opinion it takes less discipline to write memory-safe C++ code than memory-safe C code.2

This article covers one way to bring a C struct into the C++ lifetime model to leverage the better (or at least more automatic) memory safety of C++ library primitives.

We start with a highly artificial example struct designed to be a pain memory wise:3

 struct thing {
    int i;
    double d;
    char *s
    int *ary;
};

Each of the pointers pose us some (interrelated) questions:

  • Where do the objects that will be pointed to live? Heap? Stack? Data segment? Global memory? Memory mapped file? Something really exotic?
  • How do we ensure that the pointer is not used after the objects go away (if they go away)?
  • If they exist on the free-store, how do control deallocation?
The questions aren't unique to C, they are the same ones that must always be dealt with. But C code deals with them all every time, while other languages may have built-in answers to some of them.4

Nor can you necessarily answer the questions by static examination of the code, but in the case I faced at work, both pointers were consistently pointing at dynamically allocated objects. Moreover the number we needed could not be determined at compile time, so we were storing the thing *'s in a vector.

We had an existing C function thing *newThing(size_t array_size, const char *label) which would create a new struct thing on the heap (with a alloc family function), set default values of i and d, set the string and allocate (but not populate) the array and return the pointer to the thing. This is analogous to a C++ constructor, but for some reason (history, no doubt) we were handling the three calls to free manually each time we needed to reap one of these things.

Then we did something roughly like this:


{
    std::vector<const thing*> thing_list;
    for (const auto &input : inputs)
        list.push_back(newThing(input.name, input.size()));
    process_list(thing_list);
}

Which, of course, loses three heap allocated objects for every item in the inputs container.

Replicating a proper, but C-like, approach to memory management here would mean writing a destructor-analog (perhaps void reapThing(thing *p)) as a free function and inserting std::for_each(thing_list.begin(), thing_list.end(), reapThing); before the closing brace. That works and I wouldn't be displeased to see it in a legacy project like the one I'm working on, but I think we can do a little better.

The "doing better" interface is actually quite simple:5


#include "thing.h"

struct thing_wrapper : public thing
{
    thing_wrapper();
    thing_wrapper(size_t array_size, std:string_view label);
    thing_wrapper(const thing_wrapper &);
    ~thing_wrapper();
    
    thing_wrapper &operator=(const string_wrapper &);
}

The wrapper has the same data, but manages the sub-allocations for you. What complexity there is lies in ensuring that the constructors, assignment operators and destructor all agree on memory management of the sub-allocations.6 You might also want to add a constructor and assignment operator taking a const thing &, but this is a leap of faith insofar as nothing will enforce a consistent allocation strategy on those inputs. Similarly you can consider supporting move operations if you have a particular use for them.

With the wrapper in place we can change the original code to something like:


{
    std::vector<std::unique_ptr<const thing>> thing_list;
    for (const auto &input : inputs)
        list.emplace_back(std::make_unique<thing_wrapper>(input.name, input.size()));
    process_list(thing_list);
}

With no need, now, for explicit clean-up code.


1 Oddly, neither of these is trivial to add because they imperil the universal linkability of C (which depends on not needing a vendor-dependent name-mangling scheme).

2 At the foundational level, it is the object lifetime model that supports this, and at the practical level it is exploited in the standard library which offers a more powerful set of primitives than the C standard library. Step one for writing a robust C program at scale is to get a more robust library (which you might be able to get off the shelf or might want to write yourself).

3 It is, however, analogous to the problem I faced at work today.

4 Many "managed" languages have everything on the free-store, and use a garbage collector to resolve the lifetime question.

5 I've chosen to make this a struct rather than a class for two reasons. First because the whole interface we want to derive is public: we're not going to extend thing in any way beyond supporting the C++ lifetime model. Second because of Core Guideline C2: the C code enforces no invariant so we don't add one.

6 The safe thing to do, is use the facilities used by the code that provides the underlying structure, which in the case of pure c libraries usually means *alloc/free or some wrapper around the same. You may be able to defer to any pre-existing C functions that perform the set-up and tear-down.

2023-12-29

The limits of "Fix it when you touch it."

My main projects at work have been on minimal-spend for a few months which means I've been shifted to some feature adds for our biggest product. This thing goes back to the mid-eighties and is coded in C (updated to ANSI syntax, at least), Fortran (updated to f90, at least), C++ (with the standard containers, at least, but lots of it predates the "modern" era), and python (recently ported to python3, at least). So, yeah, it has all the issues you'd expect in a legacy codebase. Some of them in spades.1

We're basically a contract shop, so we don't do "Let's fix this entire module because it's grotty enough to be a pain", because who would pay for that? On the other hand, we really would like to have nice code, so we have a "You can fix issues with the bits you touch." policy.

Not complaining about that. My last feature add actually removed net lines of code because I replaced some really wordy, low-level stuff with calls to newer library features and factored some shared behavior into utility code to reduce repetition. So the policy makes me a happy (and perhaps even productive) programmer.

But it has it's limits. The short version is some legacy issues span a lot of code and you can't fix them locally.

Case Study

The bit of code I'm working on right now has a peculiar feature: in several places I find a std::vector<SomeStruct> paired with a count variable.2 They appear in an effectively-global3 state object and they're passed to multiple different routines as a pairs. This is very much not what you'd expect in code originally written in C++.

Sometime in the past this was almost certainly a dynamic array coded in plain 'ol C. And not even a struct darray {unsighned count; SomeStruct * data}; one paired with some management functions, but a bare, manage-it-yourself-you-wimp pairing of a count and a pointer.4 But why wasn't the count discarded when transitioning to std::vector?

Finding out requires a lot of tedious, close reading of code where the pairs are used. And, of course, seeing the old C code behind the current C++.

There are several places where the vector is resized (meaning multiple extra entries are added to the end in one fell swoop) to some "bigger than we need" value. Those extra entries are filled with default data, which the code then overwrites one at a time with freshly calculated "actual" data. This is (a) a performance optimization insofar as it prevents the possibility of multiple re-sizes and copies that exists if you added entries incrementally, (b) a fairly faithful transliteration of what would have done with the dynamic arrays in C, and (c) the wrong way to perform the trick with std::vector.5

When this was originally done in C, the "count" variable would have tracked how many entries had "good" data while the allocated size would have been known because you knew the maximum expected size was. But the container doesn't know that you want to do that and it's size() method will always return the number of objects it has (including the default valued one). So the manual count was still needed in places, and they kept the (effectively) global version because it was easier.

Result: getting rid of the extraneous count variable means fixing half-a dozen routines elsewhere in the project and I end up touching scores of lines in a dozen files. That's not "fix what you are touching anyway".

Takeaway

Some legacy maintenance is too big for purely local fixes.

Today was actually the second one of these I've looked at in the last couple of months. I was able to fix the first one mostly with global search-and-replace and only touched five files; I felt that was "local" enough for the payoff in terms of making the code more comprehensible. So I was optimistic on this one, too, but it quickly grew out of hand. None the less, I may be finished and if the regression tests are clean I'm going to commit it.


1 But, honestly, I worked with worse in my physicist days.

2 For those who don't do C++, the standard vector container is a extensible array-like data-structure. It maintains its own count.

3 Possibly a subject for another day, and another example of something you can't re-factor in the small.

4 In the Bad 'Ol Days, a significant number of programmers would begrudge the cycles lost to function calls for that kind of things when it was "easy" to do it inline at each site. Of course, they could have used a macro DSL for the purpose, but those are tricky and (even then) had a mixed reputation.

5 It's wrong for two reasons in general. The less important one here is that it default constructs the new values which can take cycles (in a C dynamic array using realloc means you just get whatever garbage was in the memory occupied by the new spaces so you don't pay for that). The more important issue here is that vector has reserve which unlike resize just makes sure you'll have room for the new stuff if you want to use it, meaning you can then emplace_back for the best of both worlds.

2023-10-29

You. Have. Got. To. Be. F*<#!^&. Kidding. Me.

There must be things in the communication world that make me angrier that a computerized voice prompt system that makes "typing" noises at you while it works, but right now I can't think of one.

2023-10-16

Didn't expect it to be quite that hard

I'm getting my first non-trivial experience with JavaScript. It seemed like a good idea at the time.

Backing up to explain, I wanted to write a talk for my colleagues for some time. On C++ templates. Another thing that seemed like a good idea at the time.

Anyway, there is some interest around the office in a brown-bag lunch series, so I've actually gotten serious about writing it. And I want to own it, so I"m doing on my own time and resources, but that means I don't have a copy of PowerPoint. Not that I'd want to use PowerPoint, anyway, of course. Now in my academic days using $\LaTeX$1 would have been a no-brainer, but I've been watching conference talks given in various HTML/CSS/JavaScript engines recently and it seemed like an ideal time to try something like that out. Yep, seemed good idea again.

Enter reveal.js, but also a headache brought on by perfectionism.

You see, it's to be a talk about a feature of a programming language and that means code samples sprinkled throughout. No problem, reveal has support for that. Only it would be nice if the code samples were syntactically correct.2 So, I'd like to...

  1. include the text of external files in my presentation
  2. wrap that text in whatever html is needed to get reveal to mark it up as syntax highlighted code blocks
  3. be able to compile the code in a separate tool to check it's correctness

Well, number three is entirely on me and it's easy. But both of the other gave me some trouble though it turns out I didn't need to know how to do the second item at all.3

It was the inclusion problem (item 1) where I got the surprise. You see, html doesn't have an inclusion primitive. You have to use JavaScript if you want to do that on the client side.4 Fine. Apparently that was messy in the past, but modern JavaScript has fetch to get the contents of an external asset and lots of DOM nifftiness to stick it where you want it with various kinds of preprocessing (as html, as script, as text, and maybe some others).

So, in the first place I want a code sample I write a little script that runs code = fetch("myfile.h") and then inserts it in a useful place. Reload. Nothing appears. Force reload. Still nothing. Open the inspector and find the tag. The tag is empty. What? Check paths. Force reload. Still empty. Try with "./myfile.h". And with "/home/noswampcoolers/projects/template_thing/myfile.h".

At this point I took some time off to curse many things. Including my own ignorance, but mostly other people and tools. Debugging by rant. Or something.

Finally notice the tabs in the developer tools. Particularly the one named "Console" and the red decoration thingy. Oh, there's an error; something about a "same origin policy" not letting me do what I want. Even though the data file resides not only on the same filesystem but in the same directory.

More ranting, then hit the web for some education.

Turns out the same origin policy says what external data can be loaded without a security negotiation. But it's not terribly well named because while it does put limits on the origin it also imposes limits on the protocol.5 Short version: I'm going to need a web server running on my machine to fetch stuff from the local filesystem.

The first solution I found for just serving a named directory was python3 -m http.server --directory "$PWD" 8080 > server.log 2>&1 &, but I think I could probably do the same thing with some npm invocation which would be more elegant in the sense of using the same tool for both halves of the task. It's also not worth my time right now. Python it is.

So at this point I could load some external code into my presentation.6 But it wasn't getting highlighted the way it did if I wrote it right in the html file. Argh!

Maybe it has to do with the order various scripts run in. Check the web and try moving my bit around. Lots of places. No dice.

More reading suggested that I needed to integrate my code with the framework by writing plugin. And just for once in this saga I was smart enough to find at least three existing plugins before starting down that rabbit hole. No idea if the one I picked is the best, but it seems to work.

All-in-all a enlightening but rather disheartening epic to achieve what sounded like a little thing. I'm sure there is a deep story in how it got to be that way, starting with "first to market is more important that getting it right" and moving on through various pitfalls of the standards process. I'd love to read about it if someone else would research it.


1 For a long time I had a custom set of extensions to slides, but eventually I graduated to beamer.

2 Semantically as well, of course. For those fragments that have semantics. But that's a lot less amenable to automated validation.

3 The issue I had with the second item is that I wanted to show C++ template code, and that means things surrounded by angle brackets: < and > which is, of course, the tag syntax of html. Before I solved item one, I learned a trick for that. You can include the text as the contents of a script tag marked with attribute class="text/template".

4 Lots of choices on the server side, but lets not go there.

5 I was too focused on solving the problem in front of me to read into the details, but I suppose this has to do with preventing externally supplied code from reading your locally stored data and sending the it back to the mother-ship. Which would make sense but it still leads to a minor absurdity.

6 Somewhere along the way I'd also taken the time to run down a DRY solution to including many snippets and adapted it to produce the same "pre-code-script" nesting I had learned for including code directly in the file. Shout out to Mustafa even though I didn't end up using the code. Plus, I enjoyed recognizing the immediately-invoked lambda from it's use as an initializing idiom in C++.

2023-09-14

Really, Qt?

Qt Creator is poluting my nice clear build results window with a complaint that my build directory is not on the same level of the filesystem as my source directory. It's a new complaint even though I've been using Qt Creator for ages, because I recently had to re-oganize my work a little. I have, for a long time, used a layout where by source, build, and distribution directories for each project are sub-directories of $HOME/source, $HOME/build, and $HOME/dist which actually meets Qt's strange requirement.1 Alas IT issued a new edict concerning the centalized backup system and on windows I've been forced to move my source hierarchy to $HOME/Documents/source on that platform. Which broke QT's little brain.

The policy is, presumably, intended to enfore a prefernce for out-of-source build, which is a good idea (every build should be out of source). But it is also way, way too restrictive a way to enforece that behavior.


1 This lets be share the source directory across the Windows host, several WSL linux installations, and the few VMs I still run while each platform maintains it's own build and distribution locations.