Exit Poll Special

Exit Poll Special: Febble's Fancy Function
by DemFromCT
Fri Apr 29th, 2005 at 13:19:38 PDT

[editor's note, by DemFromCT] The data reports by Freeman, Brady, USCV, etc. can be found at the National Research Commission on Elections and Voting website.... What Febble has done is shown that the oft-used variable known as Within Precinct Error (WPE) is not a very good measure of sample bias. Sample bias can be introduced by such things as Bush voters refusing to talk to pollsters. In the parlance of the trade, this is known as `differential non-response' (i.e., the non-response to pollsters is different between Bush voters and Kerry voters), but it's also been widely described as reluctant (or `shy') Bush responders, or rBr (all those terms are used in various reports and critiques). It's what Edison-Mitofsky (the exit poll company) says is the likely reason for why the exit polls were wrong, and it's primarily what USCV says is `implausible', thereby supporting their conclusion that fraud is a leading possibility to explain the exit poll discrepancy. They heavily relied on WPEs to reach that conclusion, and so did many of the subsequent Daily Kos diaries.

However, Febble's Fancy Function shows that WPE obscures diagnosis of exit poll error by introducing a lot of noise into the analysis; the Fancy Function strips the noise and leaves only sampling bias. Her computer modeling of the Fancy Function demonstrates that the E-M hypothesis (that there were shy Bush voters) remains very plausible, indeed.

In addition, assuming that Febble's work stands up to peer review, it may become an equally valid way of looking at exit polls in future, in order to detect suspicious variation in sampling bias that would deserve to be looked into further.

Febble's work does not prove there was no fraud, any more than USCV's work proves there was. The exit polls are too blunt an instrument for that, and we don't have the precinct-level data that would be required for detailed analysis (what we have is state-wide averages). Nonetheless, this is a genuine contribution to the understanding of what's happened, and as such deserves recognition. If Febble's Fancy Function is applied to precinct-level data, it is quite possible (likely, even) that lingering questions about whether it is possible for 'differential non-response' to be a plausable explanation for the exit poll discrepancy would be resolved. Only E-M has that data, so further inquiries will have to wait until after mid-May, when the American Association for Public Opinion Research will have its annual meeting. Mitofsky will be there and 2004 exit polls are on the agenda.

In addition, Febble's idea happened because of this community, the forbearance and patience of the critics, the skeptics (and the skeptics of the skeptics) that allowed the free-flowing peer review over the internet that marks Daily Kos as a special place. Febble's Fancy Function was created here, and whether or not it goes on to be published and used, you've participated in the birth of an idea. That's a very neat thing to have witnessed....