[visionlist] Statistical Criticism is Easy; I Need to Remember That Real People are Involved

[Pam Pallett]

Hi All,

I came across a blog today by Frank Harrell, Professor of Biostatistics and
Founding Chair at Vanderbilt.  His most recent post is the title of this
email.  But as I'm reading through his blog, I'm hearing a lot that has
been discussed and experienced by professors and postdocs subscribed to
this list.  We are often very separated from our neighboring departments,
and I actually found some comfort in the fact that these problems seem
spread across the board (misery loves company). Even if we have been
echoing these problems for over a decade with little effective change.

In his most recent post he says, "There are several ways to improve the
system that I believe would foster clinical research and make peer review
more objective and productive." I'm curious about what the people in the
vision community think of these suggestions and whether they are realistic
to implement in our field.  His list is at the bottom of the entry.
http://www.fharrell.com/2017/11/

For those experiencing TL;DR, here is the shortlist:

   - Have journals conduct reviews of background and methods without
   knowledge of results.
   - Abandon journals and use researcher-led online systems that invite
   open post-"publication" peer review and give researchers the opportunities
   to improve their "paper" in an ongoing fashion.
   - If not publishing the entire paper online, deposit the background and
   methods sections for open pre-journal submission review.
   - Abandon null hypothesis testing and p-values. Before that, always keep
   in mind that a large p-value means nothing more than "we don't yet have
   evidence against the null hypothesis", and emphasize confidence limits.
   - Embrace Bayesian methods that provide safer and more actionable
   evidence, including measures that quantify clinical significance. And if
   one is trying to amass evidence that the effects of two treatments are
   similar, compute the direct probability of similarity using a Bayesian
   model.
   - Improve statistical education of researchers, referees, and journal
   editors, and strengthen statistical review for journals.
   - Until everyone understands the most important statistical concepts,
   better educate researchers and peer reviewers on statistical problems to
   avoid <http://biostat.mc.vanderbilt.edu/ManuscriptChecklist>.


Best,
Pam Pallett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20171116/6734a86c/attachment.html>