How To Fact-Check an Ethnography (or Anything Else)

Recent years have seen increased emphasis on fact-checking in journalism, politics, and academics. Ethnography presents a particular set of challenges that some scholars have directly addressed. Research sites are often masked, subjects are anonymized, and details are altered to prevent close scrutiny. Even so, there are ways to test the accuracy of questionable ethnographic findings.There are ways to test the accuracy of questionable ethnographic findings.

The following is a how-to-do-it guide to fact-checking dubious claims, using a vignette from a well-regarded ethnography as an exemplar. I have omitted the name of the author and the title of the book because my goal is to show the means of investigation, rather than critique any one individual or specific work.

The author’s field site was a university cafeteria, where they obtained a job in order to observe staff and customers. After some weeks, it occurred to the author that many of the mostly African American employees could barely read. Some workers even had trouble with the timecards because their names were written in cursive, which, according to the author, a significant number of employees could not recognize.

Some of the literacy-challenged older women would therefore stand under the clock, saying they could not reach their cards, and asking a taller person to retrieve them. Some of the younger men would ask friends to log out for them, giving the appearance that they were stealing time, but, like the women, only covering up their inability to read their names.

According to the author, the reading problem affected several people in each age group, amounting to nearly 10% of the 70-person work force, but that struck me as quite implausible. Could there actually be such a significant number of people in the 21^st century United States who could not recognize their own names?Could there actually be such a significant number of people in the 21^st century United States who could not recognize their own names?

While it is perhaps conceivable that many employees were relatively poor readers, it is hard to imagine they were so illiterate that they had to use elaborate ruses to punch out every afternoon. After all, they had been able to complete job applications, navigate the transit system, cash their paychecks, and pay rent and other bills.

I worked for two years as a legal services attorney in a low-income neighborhood on the West Side of Chicago. As a poverty lawyer, I interviewed around 1,000 people, nearly all of whom were African Americans. Unlike the author’s co-workers, a great number of my clients were unemployed. Many received public assistance or disability payments, some lived in public housing, and all had legal trouble. I did not encounter a single person who appeared to have difficulty recognizing their own name (they all signed consent forms, almost invariably in cursive).

I could not go back in time to the author’s research site, but there were other ways to evaluate the reading claim. I surveyed the cafeteria workforce at a similarly located university, to see if there was even one person who fit the author’s description. I also contacted literacy organizations and reviewed national studies to determine the prevalence, if any, of such profound reading deficits among employed adult Americans.I could not go back in time to the author’s research site, but there were other ways to evaluate the reading claim.

The head of Northwestern’s Chicago Campus food service told me that he had never encountered an applicant or employee who was unable to read his or her own name. In 12 years of food service management, he had seen some applicants who could not effectively read recipes, and he had employed one person, as a cashier, who could not adequately read or submit reports (he moved this employee to another position). He was surprised, however, at the suggestion that an employee could not identify his or her name, and immediately said that it had never happened.

The National Assessment of Adult Literacy, conducted in 2003, found that 14% of adult Americans fall below basic reading skills, although that number included those who were cognitively impaired or not literate in English. The least skilled English speakers were able “to locate or identify information” in “familiar or uncomplicated materials.” I did not find any data on the inability to identify one’s own written name. That elementary skill was not included in either the 2003 study or in the 1992 National Adult Literacy Survey, evidently because it is not characteristic of a measurable population group. Instead, the very lowest literacy level—called “below basic”—comprises those who can still locate information in a sports story or underline a sentence explaining an action stated in an article.

I also contacted the Director of Programs for Proliteracy, an organization that “develops and promotes adult literacy learning.” She confirmed that their lowest level of data is for people reading at or below the third-grade level. Such people, she told me, would have trouble reading the instructions on a prescription bottle, but would still be able to recognize their names or other written prompts. The only populations unable to read simple words would be either non-English speakers or, as she phrased it, the intellectually or developmentally disabled (IDD). “Born English speakers,” she said, would all be able to recognize their own names (although there were no separate data for cursive).I surveyed the cafeteria workforce at a similarly located university, contacted literacy organizations, and reviewed national studies to determine the prevalence, if any, of such profound reading deficits among employed adult Americans.

Finally, I spoke to two representatives of Best Buddies, International, an organization that provides job placements for people, again as they phrased it, with intellectual and developmental disabilities. I asked whether their IDD clients in the food service industry would be able to recognize their own names. “Absolutely!” said one of the representatives, with more emphasis in her voice than can be conveyed with an exclamation point. “Every single one can read their own name.” I pursued the question of cursive reading with the second representative. “More than 90% can read cursive,” she said of her IDD clients, and the rest could be taught to recognize their names in cursive.

The IQ range of the Best Buddies population is roughly 77 and below (average is 100), yet virtually all could learn to recognize their names in cursive. The ethnography author, however, improbably claimed to have observed multiple individuals, in a single workplace, who fell short of the “below basic” standard, with poorer reading skills than the IDD clients of Best Buddies.

The ethnography’s editors, at a leading university press, evidently made no effort to confirm the inability of the workers to read their own names. The author presented this observation as a meaningful discovery, which is not unusual in urban ethnographies that prize unexpected findings, with an incentive to interpret ordinary events in novel ways.

Upon examination, however, such claims often fall apart under scrutiny. In this instance, the far more obvious explanations—relying on Occam’s Razor and common sense—are those provided by the subjects themselves. The older women could not reach the top row, and the younger men were indeed knocking off early and then killing time.

---

Steven Lubet is emeritus faculty at the Northwestern University Pritzker School of Law.