"Objective" assessment by questionnaire

Some time ago we received a recommendation that our daughter get screened for autism. Now, with generally overbooked state of medical specialists around here and the pandemic and all it's taken a year and half to make any headway, but the process is underway at last.

Great.

Part of that process involved a questionnaire for the parents. We were asked to rate our child's willingness to perform various tasks without prompting on a 0-3 scale.

0. The child is unable to perform the task
1. The child is able to perform the task but rarely or never does it autonomously
2. The child is able to perform the task and sometimes does it autonomously
3. The child is able to perform the task and often or always does it autonomously

And there is the options to indicate if we have no actual experience with this task but are extrapolating from related tasks.

So far, so good.

But then we get to list. Honestly it's not bad in general, but the pandemic has played merry havoc with some of its underlying assumptions.

Take "Does your child ring the doorbell or knock when visiting at a friend's house?" Presumably the question is about knowing and respecting social conventions, except that the pandemic has interfered with her ability to find little people to make friends with and prevented her from visiting those friends that she has. Is it time to guess? On what basis?

But that's not the worst of it. Another ran along the line of "Does our child lookup or respond when her name is called?" Well ... always when she's in a good mood, but never if she's really focused on what she's doing or suspects she's not going to like what you have to say (say when she knows it's after bedtime). Is that a 2? But it doesn't really capture the nature of what's going on, does it?

Every time I have to fill out a questionnaire for medical stuff I run into these ambiguities, and even if there is a practitioner around to ask no one knows how you're suppose to handle them.

I find myself assuming that it's down to trying to build nice objective measures out of the chaos that is human behavior. You can't do statistics on a bunch of anecdotes, so you have to covert all that disorder to some nice clean numbers. So, people gather a lot of experience with a phenomenon they want to study, make up some questions and stick the answers on a numeric scale, propose an analysis, test in on a sample of subjects identified by a less numeric mechanism, and check for correlation. If you get a strong enough signal you have a diagnostic mechanism. Tada!

In my fevered imagination that's the whole thing, but I'm pretty sure they try a little harder than that. I know that psychology people like to pepper their questionnaires with distractors, and I'm aware that some of the diagnostics are versioned which implies some kind of feedback and improvement loop. Alas I don't know anything about the details of the validation and improvement process.

Anyway, the problem with the bare process I outlined is that you can get a good correlation as long as there is some uniformity among the way subjects treat ambiguous questions—and anyone who's written assignment and exam questions knows that getting 80 or 90% of people to read a question a single way is easy but making it truly unambiguous is hard—but if you use the same list as a diagnostic you are immediately at the mercy of individual variation. Which offends my sense of order.

Sigh.

"Foreign" doctors

Members of my household have accumulated a lot of experience with the US medical industry.¹ Just now we have collectively eight specialists across five disciplines (and primary care, of course), and the frequent moves I've subjected my family to have meant finding new people in those (or other) positions repeatedly in the last few decades. We've interviewed many dozens of specialized physicians over the years.²

And I've noticed something about what doctors are available.

Most of the time we get offered appointments in less than a year with two kinds of doctors: people who turn out to be a poor match for us and "foreign" people (meaning people who have a noticeable accent or did part of their training in another country, and often meaning "not white").

Now, to be licensed in this country a doctor needs to do a residency here no matter what credentials they may have elsewhere, so every one of these doctors has been checked out by the local medical establishment (and because we're talking about specialists they have been checked out twice). We've found some of our very best specialists this way. But we've also heard from other members of our communities that they "weren't comfortable" with these same people.

Okay, in one case the physician had a pretty marked accent; one I remember finding pretty hard to process when I first started working with people from the same part of the world.³ I can buy that many residents of that rural and somewhat insular part of America might have found dealing with the accent one problem too many on top of dealing with medical issues. But in most cases these folks have mild accents (presumably owning to spending many years in the States) that merely give their speech an exotic edge.

A strategy

My unscientific conclusion is that some portion of the American public is avoiding a subset of our good doctors for being "not my in-group" (I won't speculate if this is overt racism, implicit racism, or just provincialism) and that means that it's easier for you to get access to those doctors.

The point is that it can be hard to get specialists, especially good specialists, so you don't want to give up any available edge. Don't be put off by an accent or a face of a different shape or color than your own.

Aside on exceptions

We also have very good specialists who are white. And some who are male. And some who are getting on in years. Our current crop includes one who is all three of those things and he's a serious keeper. But we didn't get him by calling the clinic and asking who was available, we asked the specialist at the "from" end of our last more to intervene for us and we got in with this guy on the basis of inter-doctor networking.⁴ That's been the case with a lot of our specialists who look like they came from central casting.

---

¹ It wouldn't really be right to call it a "system" except in the sense of "ecosystem". There is no organizing principle, and while many parts of it are subject to oversight of one kind or another the oversight level isn't terrible well coordinated.

² Yes, "interviewed" is the right word. We go into a new specialist's office with the questions "Is this the right doctor?" just as much in mind as the history and needs of the patient. Doctors who want to micromanage a patient used to being part of their own care are a bad match. Doctors who don't want to answer questions when a patient feels they need to know more about 'Why?' are a bad match. Doctors who tell a couple comprised of a graduate engineer and a physics professor "You don't understand the math" actually get laughed at in front of their own staffs (I feel a little sorry about that, but not very; she is, indeed, one of the elite thinkers in our society but dismissing us out of hand was out of line).

³ Much of US graduate education has a lot of diversity in country of origin even when there is not so much in gender, economic background, or race among the Americans. And experimental particle physics is an international endeavor so I got a lot of practice during my years as a scientist.

⁴ This is another strategy. If you ever have to leave a good specialists (for whatever reason) see if they can hook you up with a replacement. They already know your case and your style and may be able to choose a good match. And their influecne can get you in to see doctors who are "not taking new patients at this time".