2020-02-11

Moderation, quality control and being "welcoming"

There exists a large disconnect between the culture of some Stack Exchange sites (I'm particularly familiar with Stack Overflow and Physics Stack Exchange) and the expectation of a lot of people out on the web.

I'd like to propose a categorization that might let us talk about the conflicts, and agree on what we can and should work on. And perhaps on where we simply don't agree.

The categories  I propose (and which I will discuss in more detail lower down are)

1. The fact of rules.
2. What is in the rules.
3. How the system notifies users of violations and enforcement
4. How other users explain the rules and act to enforce them
5. Cultural differences
6. People being jerks and how moderation of these behavior presents to other users

That is a lengthy list which means that minor annoyances can add up, so we also need to consider

7. The gestalt experience.

Nomenclature of this post


For the purposes of this post, "quality control" mean voting posts up or down and making decision about what content should is on- or off-topic (so open/close decisions) or constitutes a duplicate, while "moderation" means action to control or remove conflict that is not remaining civil (the removal of comments and deletion of post, suspending users and so on).

This distinction isn't always clear, but it might be good enough.

Taking the points of conflict one at a time


I'd like to propose a categorization that might let us talk about the conflicts, and agree on what we can and should work on. And perhaps on where we simply don't agree.

The fact of moderation and quality control.


Let's face it, quality control is a process of making and acting on quality judgements about content and moderation is a similar process where the judgement in on behavior and adherence to site expectations or customs.

Saying "This content does not meet out standards" no matter how nicely praised it not going to make the hearer feel good, and many will take it as a personal attack. You can make an ethic of "vote the content, not the user", but even if you get your indoctrinated users to live it newcomers may not believe it.

In my opinion this is non-negotiable for a successful sites of any kind that should run on a Q&Aish engine, but it is a place where you necessarily lose people who don't believe it.

Nature of the rules about topicality


My impression is that many users come to Stack Exchange with expectations that the scope of the sites will match their experiences elsewhere on the internet, and are surprised to learn about the rules circumscribing the opinion-based contents and polls and largely banning jokes. In other words they are surprised by the fun-hating nature of the network.

To understand where these rules came from you need to know something about the early history of the network (they weren't announced from on high or part of the original ethos of the Stack Overflow beta, we had to learn that they were needed and invent them (or at least the version used by Stack Exchange). Nor are the uniform across the network which offers a window into how variations on these rules might affect the resulting culture of a site.

Well, at least you can let us have a little fun


The place of fun (meaning polls, surveys, joke questions, and so on) was a matter of great contention in the first few years of Stack Overflow. Joke questions and polls were around during the beta, and as you would expect they attracted a lot of attention and votes. Which was the problem.

As a compromise a site (called Not Programming Related after the title of the closing category) was spun off as a place where the new, strict rules of Stack Overflow wouldn't apply and fun could flourish. However, in time the regulars of the site became dissatisfied with the way that site was going too. Not Programming Related underwent a major overhaul and tightening of rules to become Programmers and later after another iteration the Software Engineering we know and love today. In my opinion it is much more useful in its current form.

A counterpoint is Code Golf and Programming Puzzles which started life with a strict requirement for an "objective" winning conditions (meaning not "most votes wins"). But the site didn't really take off until that condition was relaxed.

I favor strong rules limiting polls and opinions stuff on technical Q&As.


Well, you shouldn't exclude absolute beginners


Early in Stack Overflow's history founder Joel famously proposed a "No question too basic" ethic and backed it up by asking a particularly basic example question. In time the notion was dropped.

My personal take is that Q&A is a bad format for getting newcomers over the hump simply because they don't yet know what questions they should be asking. What complete newbies need is an interactive and focused tutoring sessions provided by a sufficiently experienced mentor. In any case the internet doesn't need a permanent record of their exploratory efforts: it just clutters up the search space without being discoverable for people with the same misconceptions.

In principle the necessary mentoring can happen in a chat environment, but on Stack Exchange the user would either need to have enough rep on some site to qualify for the association bonus or somehow score a up-voted post before they begin. Either of which is tricky for someone completely new to the network.

It is not obvious to me that this decision needs to be made any particular way. For a compare and contrast in the Stack Exchange network consider the differences between Mathematics (welcomes very basic stuff), Physics (excludes some basic stuff on a basic that is hard to explain to beginners), and Math Overflow (research level by designed and has a slightly different history than the others). All three can reasonably be described a "successful", but they have very different feels.

I prefer the Physics model to that of Mathematics, but I don't think it is necessary.

The way the system presents quality-control or moderation actions


So a quality control or moderation action has been taken, what does the user on the receiving end see? How clear are they on what they did that triggered the action? How easily can they get more detailed information or help understanding? Do they know where to take questions, and what kind of reception will they get?

This is partly a technical issue and partly a communication one (and see the point on cultural differences), Stack Exchange has been banging away at the technical end of the problem with some small success, but it remains a problem for the network.

This stuff needs to be watched. Sites need to hear users complaints and take them seriously in aggregate, but I don't think it will be possible to satisfy everyone.

The way other users inform newcomers about rules and  expectations


Separate of the actions (like closing) that are presented as system actions users may receive comments and downvotes. These actions are necessary, but they contribute to the experience of new users. I mostly want to address this in the section on overall experience.


Differences in cultural expectations


Technical culture in the United States has a fairly direct and terse style of communicating (though my observation is that some of my Dutch, German, and Swiss colleagues are even more blunt), and people with different expectations may read communications in that style as unnecessarily harsh.

If I write a comment "You should provide a MCVE." as a response to a beginner's question and then wander off to more interesting posts (even without down-voting or voting to close) they may feel that I'm sending an exclusionary message by using insider terminology and not explaining why that would be a good idea. Non withstanding that the link explains both what is meant and why it is a good idea.

Differences between regional cultures (Asian versus European, for instance) generate a similar level of frictions. Heck, even differences in idiom between dialects of English can generate troubling misunderstanding. To take a personal example, it took me time and deliberate effort to stop getting my hair up when a speaker of Indian English wrote that they had a "doubt" (that is to say a question) about basic physical theories of long standing.

These days Stack Exchange has an automated (machine learning driven) process that flags potentially troublesome comments for human review and has arguable made significant progress with it.

We can write codes of conduct that press people to presume good faith, encourage a encouraging rather than discouraging frame for comments, and provide comment catalogs which have the desired tone (thought this means an on-going effort to maintain the catalogs...), but again you can't please everyone.

People actually being jerks


For some people that's just a way of being. Others lash out after too many hours of trying to make some headway on the quality control front. And sometimes well meaning people just don't understand the way they are coming across.

Whatever the cause the site has to moderate this stuff efficiently.

It may not be apparent that anyone is taking the victim's side


While I am generally in favor of the impersonal and unannounced nature of most Stack Exchange moderation, I suspect it may contribute to new users sometimes getting the impression that people are being allowed to be mean to them and no one is defending them.

I don't know a good way to handle this in general. As a moderator I would sometime leave
I've deleted some comments that were infringing on the "Be nice" ethic.
type comments, but I don't think that scheme is sufficient.

The whole experience taken as a package


But here is the rub. I've broken these issues into a bunch of categories but when I read the many blog posts around the internet about how much people disliked their introductory experience of trying to be an active user of Stack Overflow the thing that stands out to me is the way they react to the combination.

They get downvotes, and negative comments, and the question gets closed, and the automated message is the wrong one or is not helpful, and no one offers helpful answers to their questions about the whole affair. Then, if they are persistent, they go to meta and things usually don't get any better.

Not sure where to go about this.

1 comment:

  1. Attention to the text near: "encourage a encouraging rather"

    (This comment need not stay up for long.)

    ReplyDelete