[visionlist] [cvnet] Action needed on potentially damaging NIH Rule change

[Lester Loschky]

Jonathan,

I think you've hit the nail on the head.

Our discussion here reminds me, conversely, of a topic that raised a lot of
discussion a couple of years ago regarding whether a claim that a
commercial app, UltimEyes, that provided a training regimen involving
visual search for low contrast Gabor patches, could be claimed to have been
shown to improve vision.  The studies by Aaron Seitz cited to support that
claim, of course, were not done using a random sample of the population,
nor a placebo control group (what could that be?).  Thus, the FTC judged
that the claims for the benefits of UltimEyes were fraudulent, and the
Seitz was fined $150,000.

Note that in that case, the careful psychophysical research by Seitz was
deemed not to provide conclusive evidence of a clinical benefit to
consumers because it did not fit the narrow definition of a clinical
trial.  Many members of the vision science community wrote testimonials to
the FTC to the effect that Seitz's research was rigorous and showed, by
normal standards in our scientific community, a real benefit of the
training regime.  Thus, the implicit or explicit point made was that having
a random sample of the population, or a double-blind placebo control
experimental design was not a necessary condition to show a clinically
relevant effect within the vision sciences.  Double-blind placebo control
studies work well in drug effectiveness studies, but not necessarily for
visual training regimen studies.

Here, the NIH seems to have expanded the definition of a clinical trial.
In doing so, it seems that such studies as those by Seitz COULD be labeled
clinical trials, thus avoiding similarly punitive judgments by the FTC in
the future.  However, in doing so, we are faced with the possibility that a
broad range of studies without any real or potential clinical application
might be labeled "clinical trials."  This creates its own slew of potential
problems, which have been described in earlier messages.

*BTW, I am NOT claiming that the NIH changed its definition of clinical
trials to solve the problems raised in the FTC's UltimEyes case.*  I have
no idea whether there is any historical connection between the two--my
guess is *probably not*.

However, I think that both of these problems revolve around the definition
of a "clinical trial," and that the solution to both of them involves
understanding that 1) a meaningful clinical application is at the heart of
the study, and 2) the norms of the relevant scientific community as to what
constitutes reasonable scientific evidence in support of such clinically
relevant effects must be taken into account.

Best,

Les



On Mon, Jun 5, 2017 at 11:14 AM, Jonathan D Victor <jdvicto at med.cornell.edu>
wrote:

> I don’t think that a distinction based on timescale will work --  a
> clinical trial of a medicine to stop a seizure could be expected to have
> its effect within seconds (for example), and a psych experiment might look
> at memory over months to years.
>
>
>
> To me, the over-riding distinction is that of purpose:  in a clinical
> trial, the goal is to evaluate a treatment for a disease or condition, and
> a positive or negative result of the trial will influence clinical practice
> – so there’s a rationale for having true clinical trials prospectively
> registered, to reduce the effects (on public health) of a publication bias
> towards positive results, etc., and to ensure that anyone can find out
> about the studies and enroll.
>
>
>
> In a typical vision lab study, whether our subjects are healthy or not,
> whether we randomize them or not, whether we are looking for effects that
> last seconds or weeks, our purpose is to try to learn something about
> normal visual processing, and the results of the study may influence future
> research, but not patient care.
>
>
>
> Meantime, **please write** to the NIH about the dangers of an
> over-reaching definition—
>
>
>
> JV
>
>
>
> Jonathan D. Victor
>
> Fred Plum Professor
>
> Division of Systems Neurology and Neuroscience
>
> Feil Family Brain and Mind Research Institute and Department of Neurology
>
> Weill Cornell Medical College
>
> v: 212 746 2343 <(212)%20746-2343>; f: 212 746 8050 <(212)%20746-8050>
>
>
>
> jdvicto at med.cornell.edu
>
> lab home page: http://www-users.med.cornell.edu/~jdvicto/labonweb.html
>
> personal home page: http://www-users.med.cornell.
> edu/~jdvicto/jdvonweb.html
>
>
>
> *From:* cvnet-bounces at lawton.ewind.com [mailto:cvnet-bounces at lawton.
> ewind.com] *On Behalf Of *Sue Leat
> *Sent:* Monday, June 05, 2017 11:47 AM
> *To:* Susana Chung; Woods, Russell
> *Cc:* McCourt, Mark; Peli, Eli; cvnet at lawton.ewind.com
>
> *Subject:* RE: [cvnet] Action needed on potentially damaging NIH Rule
> change
>
>
>
> To add my two cents worth :-
>
>
>
> Firstly, I am wondering what problem they are trying to fix with this
> change. As it will cause a deal of additional problems and cost.
>
>
>
> Secondly, it seems that the difference between a clinical trial (as we
> currently know it) and a laboratory study (e.g. doing a task in different
> levels of illumination, but which would be randomised within subjects) is a
> question of the time. With a clinical trial (an interventions such as a
> pill or a different spectacle prescription, we are looking at an outcome in
> real life i.e over a period of days, weeks or months). But for an
> experiment, we generally look at immediate results (within one day). Or am
> I missing something?
>
>
>
> Thirdly, I would think that this would have a large adverse effect on
> education, in which students (even at the undergraduate level) do
> experiments, which would now be clinical trials and become almost
> impossible in the time and with the resources that they have.
>
>
>
> Sue
>
>
>
> Susan J. Leat, PhD, FCOptom, FAAO
>
> Professor
>
> Adjunct Scientist, Lawson Health Research Institute
>
> Head of Residencies,
>
> School of Optometry and Vision Science (Room Opt 377)
>
> University of Waterloo,
>
> 200, University Ave. West,
>
> Waterloo, ON N2L 3G1, Canada
>
> Phone 01-519-888-4567 ext 32040
>
> Fax 01-519-725-0784
>
>
>
> *From:* cvnet-bounces at lawton.ewind.com [mailto:cvnet-bounces at lawton.
> ewind.com <cvnet-bounces at lawton.ewind.com>] *On Behalf Of *Susana Chung
> *Sent:* Monday, June 5, 2017 10:29 AM
> *To:* Woods, Russell <Russell_Woods at meei.harvard.edu>
> *Cc:* McCourt, Mark <mark.mccourt at ndsu.edu>; Peli, Eli <
> Eli_Peli at meei.harvard.edu>; cvnet at lawton.ewind.com
> *Subject:* Re: [cvnet] Action needed on potentially damaging NIH Rule
> change
>
>
>
> Hi All:
>
>
>
> Just a couple of comments (more the consequences of the new definition):
>
>
>
> 1. Many journals (especially those commonly considered as “high impact”
> journals in vision science) now apply the “NIH clinical trial definition”,
> and will ask for a clinical trial registration number if subjects are from
> clinical populations.
>
>
>
> 2. Registering on clinicaltrials.gov
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__clinicaltrials.gov&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=TYhWZyIXlR6yGpC8d1ByCsj04zxUA5S63xGzvKjfap0&e=> is
> not fast.  It could take a month or so.
>
>
>
> Susana
>
>
>
> On Jun 4, 2017, at 6:36 PM, Woods, Russell <Russell_Woods at meei.harvard.edu>
> wrote:
>
>
>
> Hi All,
>
> Sorry to disappoint on the within subjects design, as that is exactly the
> example given to the NIH committee by Cheri Wiggs (NEI program officer),
> who was told that it fit the definition of a clinical trial (the example
> that I mentioned earlier of each subject having two luminance conditions).
> Like you, I had hoped that this may be a way out.  Please note that the
> “new” definition places no limit on the number of study participants, so
> your “clinical trial” need only contain one subject.  Further,
> randomisation of participants to groups or to interventions is not
> required, only that the intervention is prospective.
>
> As noted by others, the only way out will be a ruling or guidances that
> give more clarity on how the rules should be applied and when registration
> on clinicaltrials.gov
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__clinicaltrials.gov&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=TYhWZyIXlR6yGpC8d1ByCsj04zxUA5S63xGzvKjfap0&e=>
> is required (at the moment, all are) and when GCP training is required
> (now, everyone must), so that we can go back to sensible conduct of
> clinical research.  As Steve Burns noted, such guidance would assist IRBs
> in making determinations.
>
> As noted, make your opinions known to as many people at NIH as possible.
>
> best wishes,
>
> Russell
>
> **************************************************
> Russell L Woods  PhD
> Schepens Eye Research Institute
>
> phone: 1 617 912 2589 <(617)%20912-2589>
> e-mail: russell_woods at meei.harvard.edu
> serinet.meei.harvard.edu/faculty/woods/index.html
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__serinet.meei.harvard.edu_faculty_woods_index.html&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=Id1qlMB8qGTPh-Q5Tyqda_Mni5WJ6ttd-yJl2g16H34&e=>
> www.schepens.harvard.edu/profilewoods/russell-l-woods-phd/profile.html
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.schepens.harvard.edu_profilewoods_russell-2Dl-2Dwoods-2Dphd_profile.html&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=PhMjzEfJZFGbUqnAqTURXODgAJkOScA5EKF1blU0Xeo&e=>
> **************************************************
>
> On 4 Jun, 2017, at 3:10 AM, Jonathan D Victor <jdvicto at med.cornell.edu>
> wrote:
>
> If advocacy fails, this of course is a fall-back strategy – but I don’t
> think that it’s a good idea (pragmatically) to rely on a “loophole” of this
> sort.  It would be much better, from all points of view, for the NIH
> definition of a “clinical trial” to be sensible.  Please write to the NIH
> with this in mind.
>
> One could easily imagine a scenario in which an attempt to get around the
> new definition (by pointing out that one is using a within-subjects design)
> will  backfire – and that the study will be declared to be a “clinical
> trial with an inappropriate design”, and therefore not even approvable by
> an IRB or a well-meaning oversight committee at NIH that thinks that it’s
> protecting human subjects from taking part in scientifically invalid
> research.  At my institution, I am routinely challenged on the
> justification for my low number of subjects in psychophysics studies, since
> our IRB thinks in terms of drug trials.
>
> JV
>
> From: cvnet-bounces at lawton.ewind.com [mailto:cvnet-bounces at lawton.
> ewind.com <cvnet-bounces at lawton.ewind.com>] On Behalf Of Peli, Eli
> Sent: Saturday, June 03, 2017 7:33 PM
> To: McCourt, Mark; cvnet at lawton.ewind.com
> Subject: RE: [cvnet] Action needed on potentially damaging NIH Rule change
>
> Hi Mark,
> As ridiculous as it may be I think you may actually be on to something
> valuable.
> In fact we do not follow clinical trials rules in many of our studies and
> in particular in the within subjects design.
> Furthermore when one runs a  legitimate clinical trial with a cross over
> design (to try to benefit from the within subject benefit), it considered
> unacceptable for the purpose of clinical trial design by the Cochrane type
> analyses.  For that they accept only an analyses of comparison between the
> group first assigned to the first treatment and the group first assigned to
> the second treatment.
>
> I have been objecting to this idea ,as it applies to many of our studies.
> Of course it is perfectly reasonable approach when you consider drug
> trials, which is what the whole structure of clinical trials built on and
> derived from.  In a drug trial cross over (within subject design) may not
> be a reasonable approach in many cases.
>
> In any case I think we should try to pursue this crack and see if indeed
> within subjects studies which will be inappropriate as clinical trials will
> be waved form this definition for the purpose of the new rules.  Since in
> many of our studies such study design is preferred and universally applied
> – they can be declare non clinical trials.
>
> Russell and Jeremey, could you pursue this idea with your respective
> contacts.?
>
> Best
>
> Eli
>
> From: cvnet-bounces at lawton.ewind.com [mailto:cvnet-bounces at lawton.
> ewind.com <cvnet-bounces at lawton.ewind.com>] On Behalf Of McCourt, Mark
> Sent: Friday, June 02, 2017 9:21 PM
> To: cvnet at lawton.ewind.com
> Subject: RE: [cvnet] Action needed on potentially damaging NIH Rule change
>
> 1.     Clinical trials are a subset of clinical research. To be classified
> as a clinical trial, a clinical research project must involve the following
> three key characteristics:
>
> ·        Prospective assignment of subjects through a pre-defined process;
>
> ·        An intervention; and,
>
> ·        Evaluation of a health-related biomedical or behavioral outcome
>
>
> Prospective assignment of subjects implies a between-subjects design. Are
> within-subjects designs exempt from these new definitions?
>
> Mark
>
> From: cvnet-bounces at lawton.ewind.com [mailto:cvnet-bounces at lawton.
> ewind.com <cvnet-bounces at lawton.ewind.com>] On Behalf Of Jeff Mulligan
> Sent: Friday, June 2, 2017 4:32 PM
> To: cvnet at lawton.ewind.com
> Subject: Re: [cvnet] Action needed on potentially damaging NIH Rule change
>
> On 6/2/17 1:26 PM, Woods, Russell wrote:
> This is an issue that I have been working on with my NEI program officer,
> Cheri Wiggs, since December, and we have made no substantial progress.
> Cheri is on a NIH committee that has been looking at the implications of
> the rule change, and though some guidance on interpretation is expected, it
> seems that they are not backing down on the definition, despite the issues
> that you identified (and which I have raised with Cheri and she has
> discussed with the committee).
>
> An example study that Cheri confirmed with the NIH committee is considered
> a clinical trial, now, is having a subject perform a task at two luminance
> levels (hits the two defining criteria: behavioural measure,
> intervention).  It’s crazy.
> I found this document:  Questions and Answers Regarding the Revised NIH
> Definition of “Clinical Trial”
>
> http://osp.od.nih.gov/sites/default/files/resources/QAs%
> 20for%20Investigators%20on%20Revised%20NIH%20Definition%
> 20of%20Clinical%20Trial%2002-06-2015_Updated.pdf
>
>
>
> Here are some of the high points:
>
> 1. Why did NIH revise the NIH definition of “clinical trial”?
>
> The definition was revised to make the distinction between clinical trials
> and clinical research studies clearer and to enhance the precision of the
> information NIH collects, tracks, and reports on clinical trials. It is not
> intended to expand the scope of the category of clinical trials. (emphasis
> added by me)
>
> [...]
>
>
>
> 1.     Clinical trials are a subset of clinical research. To be classified
> as a clinical trial, a clinical research project must involve the following
> three key characteristics:
>
> ·        Prospective assignment of subjects through a pre-defined process;
>
> ·        An intervention; and,
>
> ·        Evaluation of a health-related biomedical or behavioral outcome
>
>
>
>
>
> I agree that considering two luminance levels to be an "intervention" is
> crazy.  It doesn't seem to be that the problem is with the definition of a
> clinical trial, rather the interpretation.
>
> Also, in many psychophysical experiments the outcome is not
> "health-related".
>
>
>
> This is all moot for me (government labs can't get support from NIH).  But
> just when you think that the bureaucracy can't get any worse, they surprise
> you...
>
> my condolences,
>
> Jeff
>
>
>
>
>
> Mass. Eye and Ear Confidentiality Notice: This e-mail and any files
> transmitted with it are confidential and are intended solely for the use of
> the individual(s) addressed in the message above. This communication may
> contain sensitive or confidential information. If you are not an intended
> recipient, dissemination, forwarding, printing, or copying of this e-mail
> is strictly prohibited. If you believe you have received this e-mail in
> error and the email contains patient information, please contact the Mass.
> Eye and Ear Compliance Line at 844-815-4401 <(844)%20815-4401>. If the
> e-mail was sent to you in error but does not contain patient information,
> please contact the sender and delete the e-mail.
> _______________________________________________
> cvnet mailing list
> cvnet at lawton.ewind.com
> http://lawton.ewind.com/mailman/listinfo/cvnet
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lawton.ewind.com_mailman_listinfo_cvnet&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=0WfvBEqLnVpqBDFYjaMDtQVLvef0cuWeEkN82eO_SWo&e=>
>
>
>
> _______________________________________________
> cvnet mailing list
> cvnet at lawton.ewind.com
> http://lawton.ewind.com/mailman/listinfo/cvnet
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lawton.ewind.com_mailman_listinfo_cvnet&d=DwMGaQ&c=lb62iw4YL4RFalcE2hQUQealT9-RXrryqt9KZX2qu2s&r=_m9BMgPkMW4vdmw0GjWDYnM7Tb7VSBSMMTUNAK2fXfQ&m=lb9A2bIc6fv5tvCoL6G-OpIq9rgbZNUFioL1ESRiyto&s=0WfvBEqLnVpqBDFYjaMDtQVLvef0cuWeEkN82eO_SWo&e=>
>
>
>
> _______________________________________________
> cvnet mailing list
> cvnet at lawton.ewind.com
> http://lawton.ewind.com/mailman/listinfo/cvnet
>
>


-- 
Lester Loschky
Associate Professor
Department of Psychological Sciences
471 Bluemont Hall
1114 Mid-Campus Dr North
Kansas State University
Manhattan, KS 66056-5302
Phone: 785-532-6882
E-mail: loschky at ksu.edu
research page: http://www.k-state.edu/psych/research/loschkylester.html
Lab page: www.k-state.edu/psych/vcl/
<http://www.k-state.edu/psych/vcl/index.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20170605/704cea67/attachment-0001.html>