No, you can’t read a debate scoresheet

Overall score

The average score was 4.24 out of 10; the median was 4. Here’s a histogram:

Breakdown by question

The “correct answer” is the number intended by the adjudicator. In some cases I could use ballot math to infer this; in the others, I asked the chair responsible. Rows highlighted in green are questions that a majority of respondents answered correctly.

Contentious questions

Some digits proved particularly divisive, either in evenness of split, or in the number of different responses.

Results by country and region

There’s not much instructional value in this, but I thought the chart was fun. When interpreting this data, remember that it’s based on scoresheets submitted at a United States national championship. More precisely, six of the images are courtesy of Americans, three of Canadians and one of a Mexican — if you expect at all people to be slightly better at deciphering the handwriting of their compatriots. (The data doesn’t really bear this hypothesis out.)

Average score by country (error bars are 95% CIs)
Average score by region (error bars are 95% CIs)

Results by debating and tabbing background

The short of it is that there’s nothing interesting here. No differences were statistically significant. Judges with a major break and retirees sort of did worse than everyone else, but not by enough for it to be convincingly anything other than noise.

More detailed results

This post just presents some highlights. If you’d like to see more statistical detail, or even play with the raw data, here are links to my working spreadsheet and Jupyter notebook:

What can we do about this?

If poor handwriting is a perennial problem on the debate circuit that repeatedly causes errors and delays, what can we do?

Electronic ballots

Counterintuitively, we don’t believe electronic ballots will help. Obviously, they’ll eliminate handwriting problems, because they don’t have handwriting. But we have good reason to believe it’s likely to just move mistakes from paper, where tab teams can detect them, to online forms, where we can’t.

Complete double-blind entry

Handwriting ambiguities should never go unnoticed. Tab assistants are instructed to reject ambiguous scoresheets, and each one is double-checked by a different person. However, as I learnt at USUDC 2018, an enter-and-check system still seems too susceptible to human error.

Ballot design

Noting that handwriting ambiguities are almost always in the last digit, another idea is to have chairs confirm the last digit of their score by circling a choice on the scoresheet, like this:

Runners rejecting ballots

If we stop ambiguous ballots and insist on fixing them at the point of collection, we can greatly reduce the delay in tracking down the responsible chair. It therefore makes sense for runners to be the first line of defence; for some tabbers, this is already standard practice.

Example bank

At tournaments that members of the WUDC Cape Town 2019 tab team run between now and WUDC, we’ll set aside incomplete and illegible ballots to compile some “how not to fill out a ballot” slides. We’re considering running these slides in the briefing room at WUDC, to provide constant reminders to chairs about how ballots can be ambiguous in ways that they might not expect. Unlike with this quiz, we may name the chairs who provide these examples. Consider yourself warned.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store