Truth, doubt, and “whack-a-mole”

Before I can finish debunking one misconception about my studies on Wis. Jury Instruction 140 — studies available here and here — another misconception pops up.  It’s like playing a never-ending game of “whack-a-mole.”  It’s easy and virtually cost-free (in terms of time) for prosecutors and judges to launch these attacks.  But their criticisms are often so far off-base, or even bizarre, that it takes a lot of work to respond to them.  Pretty soon, however, I’ll have written so much on this topic I will simply be able to respond: “See law review article C, page six.”  Later this year I will publish the recently-completed Educating Judges and Lawyers on Social Science Research: A Case Study.  In it, I will address the latest set of attacks.  But if you just can’t wait — or if you can wait but you have a motion hearing coming up to modify J.I. 140 — you’ll find a bullet-point list debunking these new criticisms after the jump. 

Here they are: 

§         "The studies are not reliable because the author is a defense lawyer."  False.  That's an ad hominem attack and is not proper argument.  (You wouldn’t know it by the legal profession, but there actually are proper and improper forms of argument.)  My day job says nothing about the strength of the study design, findings, arguments, etc.  Drug companies test drugs.  Environmentalists study global warming.  Political think tanks recommend tax policy reform.  This doesn't make the tests, studies, and arguments invalid. 

§         "The study participants could have been all college students."  False.  That is true of many psychology studies, but not ours.  We lay out in great detail the ages, races, jury experiences, and other demographics of all of the study participants.  This is detailed in both studies. 

§         "It's misleading to say there was a fifty percent increase in convictions."  False.  This refers to the nearly fifty percent increase in conviction rates in the second study.  There is no other way to describe it.  We also provided the underlying numbers that produced the fifty percent increase.  The judge who launched this criticism probably heard of instances where it was said, for example, "Drug X doubles your cancer risk."  This is misleading if the cancer rate went from one percent to two percent and if those percentages were not disclosed.  But we do the exact opposite of that.  We disclose everything.  This judicial criticism proves that a little knowledge is dangerous. 

§         "The sample sizes were too small and the studies' results are insignificant."  False.  The sample sizes were actually very large.  More importantly, the results were statistically significant.  This is not a matter of judicial or prosecutorial opinion; rather, it's a matter of statistical calculation.  It's called the p-value and is explained in great detail in both articles.

§         "Some of the participants could have voted the way they did because of biases."  Possible.  However, as we discussed at length in the first study, random assignment into groups (when done with large sample sizes such as ours) produces statistically identical groups at the outset, i.e., men, women, the old, the young, the educated, the uneducated, the biased, the unbiased, etc., are distributed equally into the two groups.  Then, the only thing that is different between groups is the variable being manipulated.  This is a hallmark of a controlled study.

§         "The test participants did not deliberate."  True.  But this says nothing about the validity of a study.  Many, many published studies do not use juror deliberations.  And the research on the impact of deliberations on verdicts is itself mixed.  We discuss this at length in the Richmond study.  Also, in the Columbia study, we demonstrated that JI 140's language makes twice as many participants mistakenly believe that conviction is proper even if they have a reasonable doubt.  Should we continue to create this misconception in jurors simply because of the possibility that it might get cleared up during deliberations?  That would be a horrific policy.  It would be contrary to the judge's obligation to accurately and clearly instruct the jury, and it would be contrary to a prosecutor's obligation as "minster of justice" to ensure procedural due process for those being prosecuted.  

§         "The studies are not peer reviewed and therefore were never replicated."  Probably true, false, and false.  Richmond may not have used peer review.  Columbia did.  However, the judge who made this claim is confusing peer review with study replication.  Peer review simply means that, before a journal accepts an article for publication, they send it out to peers to review it and fill out a survey sheet about the article.  (I've even been asked to peer review an article on police practices that a peer-reviewed journal was considering publishing.)  Peer reviewers do NOT replicate the study.  A study replication, on the other hand, is what we did in the Columbia article.  We did a conceptual, rather than a direct, replication.  This is explained in the article itself.

§         "Because of the above criticisms, we should keep JI 140 as it is."  False.  Forget about the studies.  Forget about case law as to what language is sufficient to sustain a conviction after the fact.  Just imagine two identical jury instructions on reasonable doubt, the highest burden of proof there is.  Both instructions also provide a litany of doubts that are not reasonable and should not be used to acquit, e.g., speculative doubt, imaginary doubt, doubt based on fear, etc.  (New York used to tell jurors not to be "weak-kneed jellyfish" when rendering their verdict; Wisconsin still does this, but without using the exact term “jellyfish.”)  Again, the two instructions are identical.  However, the only difference is that the second instruction ends with the mandate "not to search for doubt" but "to search for the truth."  Wouldn't you EXPECT this second instruction to lower the burden of proof?  If a juror is told not to look for doubt but to search for the truth, and thinks something is merely "probably true," the juror would be obligated to convict.  The studies, therefore, are only confirming what is already logical and even expected.  So even if someone had a legitimate criticism about the studies, JI 140 should still be changed.