Curbstoning, to an established auto dealer, is the practice of unlicensed car dealers selling cars from streetside, where the cars may be parked along the curb. With a pretense of being an individual selling a car on his or her own, and with no fixed location, such dealers avoid the fixed costs and regulations that burden regular used car dealers, and, in the eyes of the dealers, constitute unfair competition. Hence the numerous web sites touting the honesty and fair deals you get from your neighborhood used car dealer, and warning you against the allure of curbstoners.
To a statistician, curbstoning means something completely different – it is the practice of fabricating survey data on the part of interviewers. Sometimes it is done to speed the work – an interviewer might simply sit at a desk (or, in the days when door-to-door surveys were the rule, on the curbstone) and fabricate 50 interviews in a small fraction of the time needed to actually find and talk to people. The Washington Statistical Society, together with the American Society of Public Opinion Research, held a session several years ago devoted to curbstoning. What motivations besides laziness may be at play? How can statisticians detect it?
Perhaps the mother of all curbstones is the 2014 survey by Green and LaCour concerning attitudes towards gay people. They found that if the surveyor stated that he or she was gay, and shared some personal information, the respondent was less likely to express negative views towards gay people. The findings received a lot of publicity. Recently, though, additional researchers studied the actual data and found irregularities. Most of which fell in the category of “the data are too perfect.” The sampling technique described by Green and LaCour was “snowball” sampling (see here for more information), which is prone to bias and high variance.
Green and LaCour’s opinion survey (using the “gay feeling thermometer,”) however, resembled the data in a different open-source survey far too closely – the distribution of opinion was nearly identical to that survey, in fact. Moreover, the survey’s reported change in respondent opinion was nearly perfectly normally distributed – “not one response out of thousands deviated meaningfully from this [normal] distribution.” Normal distributions are useful constructs for statistical methodology, but normal distributions this perfect are almost never encountered in the real world (except in cases where the measurement instrument has been carefully tweaked and honed over the years to produce a normal distribution – like the SAT test).
The statistical evidence was soon followed by other evidence and, with the concurrence of Green, Science retracted the article on May 28. LaCour has not agreed to the retraction.