Well, what format are we using now?

Code formatting guidelines are good. Code formatting guidelines that people follow are better.

Now, my team is pretty conciencious, and we all try to follow the guidelines, but there are two issues:

1. We don't actually have a written standard.
2. We're all humans. Reasonably attentive but rather fallible humans.

One solution to this is to adopt some kind of formatting tool and require that it be used regularly (or better yet, automate its use). Great. That's the plan. Only...

1. Now we actually have to chose a standard.
2. We already have roughly 120k SLOC written by the aforementioned fallible, moderately attentive humans and we want to minimize churn on the repository caused by simply adopting this tool.¹

The key here is to select a standard that reflects what we've actually been doing with our slightly different personal takes on things, our mixed level of attention, and our occassional outright errors. But how do we look at nearly four hundred files and distill what we've actually been doing?

Right. We get a program to do it for us.

Better still someone else has already written it: whatstyle. This thing attempts to write an optimal style file for you by trial-and-error optimization with a metric like "fewest changes". Though exactly what that means is somewhat debatable.

It supports several common beautifiers including clang-format which is what my colleagues are using. It gave what looked like quite reasonable results when I ran it on a single large source file from my repository, so now I'm running over a large fraction of my repository at once (excluding a directory of legacy code we borrowed from another project and some "include this in your project" third-pary code).² The only issue is that it took roughly a minute to do the one file (it is examining hundred to tens of thousands of style definitions in each optimization round), and now I've handed it more than three hundred files. For the moment my work laptop is functioning as a space heater.

My plan is to send the suggested format file to every coder involved in the project, ask for suggestions, and try to mediate the resulting chaos.

Wish me luck.

---

¹ You're right. An experienced project leader would have adopted the fool thing at the beginning. I'll keep that in mind next time around.

² Something roughly like this:

find ~/Projects/Foo -type f \
             -and \( -name '*.h' -or -name '*.cpp' \) \
             -and -not \( -name 'thirdpartycode.h' -or -path '*LegacyLibrary*' \) \
             -print0 | xargs -0 \
             whatstyle -f clang-format 

Then wait.

Don't speak so loudly sir, or everyone will want one!

Is that a bug in my BeautifulSoup?

It seems that the maintainers consider prettify() to be a debugging tool and are not concerned that it often adds unmeant white-space to the output.

Uhm. Really?

I mean, I can see the utility of a tool which exposes the tag structure as clearly as possible, but I also like to have my machine generated code look good (i.e. be fairly readable) while it remains correct. Perhaps a interface like prettify(preserve_output=False) would be in order?¹

---

¹ Here I have selected the default sense to not break existing code, starting ab inito I would have made True the default.)

I'll have the BeautifulSoup

We've delived version 1.0 of my project at work; or perhaps we're in the process of delievering it.

We've handed over a complete copy of the repository and deployable binary bundles for three targets and VM based development environments for two.

We still have some time to complete and hand over the documentation. Better still, by policy I set very early on all the documentation is in the repository, so they already have it. And I tried to keep us writing and updaing documentation often enough that it didn't get too far out of synch with the code, so the documentation they have is pretty good. But we're taking this time to make it better: more complete, easier to understand, and reflecting the code we handed over even better.

There is a twist: even though the project is entirely in-house, it's paid for by a contract to which we are a subcontractor. This means that the prime contractor has the responsibility to report on the work (and the authority to make demands about that). This particular prime contractor has a tradition of including the user manuals for software projects as appendicies to the report.

But my main user documentation takes the form of a pile of individual html files that are displayed in a on-line help system. There is no linear structure imposed and no single document containing it all.

So I needed to flatten it.

There are only 20 files right now, so I could have done it by hand. But that would have taken a few hours or more to get right (all the internal links!) and I have reason to believe the project will be funded for further development meaning I'll have to do it all again in a year or two. That's excuse enough to automate the task.

The tool I settled on was a python library called BeautifulSoup which I had heard of but never used before. I loaded a html/css scaffold file, looped over a hand-tooled list/tuple structure placing my files into a few categories. In the loop I created a table of contents entry for the file; grabbed the file body, re-wrote the header levels, munged the anchors and links, base64 encoded the images, and wrapped the result in a section for nice formatting; and then appended the new section to the body of the scaffold. When the loop ended prettify and write. It just works. I'm hooked.