[visionlist] Matlab --> online Experiments

Sun May 3 03:01:03 -04 2020

Dear Jon and vision community,

I would like to put a few things into context here and also ask some
question.

I read the paper you quoted, but our platform (https://www.labvanced.com/)
was not included in the comparison. So I am wondering whether this was a
conscious choice or whether we were not on your radar yet (of course we
would be glad to be included in future work)?

In any case, I would like to stress the point that our system was build
around timing accuracy, i.e. we use a pre-rendering approach, and precision
< 3 ms can surely be realized with us. However, aggregates of accuracy
between subjects are always a bit difficult, because individual differences
can be quite high for various reasons. Hence we measure and provide timing
accuracy for each measurement by default; and this can easily be extended
to measure accuracy for each trial.

Overall, I dont think there is a single best choice for all online
experiments. At Labvanced for instance, we provide audio and (soon) video
recordings, webcam based eye-tracking, multi-participants studies, and
other features which in my understanding are not supported by default in
PsychoPy for instance. Also with us one can execute custom clientside
JavaScript but also connect to an external sever (API) via websockets, for
instance to connect to some device like an EEG, or execute server-side
functions for optinmization problems, machine learning etc. So we try to
combine the best of both worlds, a powerful and intuitive graphical
interface, but also coding options for (very) advanced usage.

Needles to say, besides PsychoPy, there are also good native Javascript
frameworks for web based studies. So my honest and unbiased oppinion to
anyone in the community is: Try out various platforms/frameworks and pick
the one best suited for your needs!

On Sun, May 3, 2020, 00:48 Jon Peirce <jon.peirce at gmail.com> wrote:

> Sorry, I'm a bit late to this discussion, but just to expand on the
> discussion and give some more info about running studies online.
>
> We (the PsychoPy team) put in a lot of work over the last 2-3 years,
> intially funded by Wellcome Trust and now the Chan Zuckerberg Initiative,
> to port our Python library to JavaScript and allow the Builder interface to
> generate scripts in either syntax. At this point  I don't think I'm being
> biased when I say that PsychoPy is your best bet for creating a single
> study that can run both in the lab (maximal precision) and online (maximal
> portability) with minimal translation hassles.
>
> PsychoPy has excellent timing in both environments
> https://psyarxiv.com/d6nu5/
> In the lab is achieves sub-ms keyboard and audio timing (it actually uses
> the same keyboard and audio C code as PTB since Mario ported that to
> Python). It also has best-in-class timing online - it's the only package
> currently able to produce RT timing precision under 3.5ms on all browsers
> (in conjunction with a LabHacker Millikey button box).
>
> For those that are interested I've spelled out below some of the issues
> that needed solving for all this to be possible, and a few that still
> aren't yet solved.
>
> Hope the info is helpful.
>
> best wishes all,
> Jon
>
> *Porting the lib to JS*
>
> As Dee points out, is that the functions themselves need to have been
> ported to JS as well as just converting the syntax of your script.
> Translating your PTB script to JS syntax is no use because the Screen
> functions haven't been ported to JS. PTB's key selling point, that a lot of
> the critical functions run as compiled C code, is probably also the thing
> that will make it hard to port to JS.
>
> A substantial portion of the PsychoPy lib has now been ported to JS. Not
> all stimuli are there but many are, and the calls to use them are generally
> the same as in the Python lib. https://www.psychopy.org/online/status.html
> In particular, you can time your stimulus presentation by number of frames,
> and you can update stimulus parameters on every frame! Timing can't
> currently be *as good* as lab-based libraries (and don't listen to anybody
> telling you otherwise) but it is much better than most people imagine.
>
> PsychoJS (the JS port of the lib) uses WebGL where available, which allows
> a large range of hardware-accelerated graphics, including GL shaders, to be
> used so we should ultimately be able to create all your favourite stimuli
> online. :-)
>
> *Generating the JS experiment script*
>
> PsychoPy Builder can also output your experiment in either format - the
> Python script or the JS files. Python Code Components within the Builder
> experiment can even be auto-translated to JS format! Again, you can't use
> functions that don't exist, but the syntax converts remarkably well, so you
> can augment your study with conditionals, loops and even custom functions
> and these will all be translated.
>
> You can't take a hand-written script and auto-convert that. The structural
> changes required for a full script mean that isn't possible (and probably
> never will be).
>
> That said, creating a study in Builder is really fast, so starting from
> scratch isn't such a big deal, and Builder is the approach we recommend
> even for skilled programmers anyway. It typically means fewer mistakes,
> better timing and more future-proof experiments ("future you" can still
> undertand the study when it's graphical rather than 1000 lines of code).
>
>
> *What isn't done *
>
> The downside for vision scientists is that we expected this community to
> be last to want to go online so the psychophysics features have been last
> to be implemented. In particular, gamma correction hasn't been done
> (although it could be using WebGL Shaders) and we haven't ported the
> Gratings and similar stimuli yet (mostly because these aren't the useful if
> gamma isn't linearised). We also haven't yet ported over the staircase
> procedures.
>
> All these things can be done and we will get to them in due course but,
> also, if you have JS skills you could jump in and help us fill the gaps!
>
>
>
> On Fri, Apr 24, 2020 at 8:23 AM Sauter Marian <marian.sauter at unibw.de>
> wrote:
>
>> Dear all,
>>
>>
>> for those of you still seeking recommendations on how to get your
>> experiments online, our article just got pusblished open access:
>>
>>
>> Sauter, M.; Draschkow, D.; Mack, W. *Building, Hosting and Recruiting: A
>> Brief Introduction to Running Behavioral Experiments Online*. *Brain
>> Sci.* *2020*, *10*, 251.
>>
>>
>> https://www.mdpi.com/699990
>>
>>
>> Best wishes,
>> Marian
>>
>>
>>
>> *Dr. Marian Sauter*
>>
>> *Wissenschaftlicher Mitarbeiter*
>> Universität der Bundeswehr München
>> Department für Psychologie - Professur für Allgemeine Psychologie
>>
>> Gb. 161 / Raum 1016
>> Werner-Heisenberg-Weg 39
>> 85577 Neubiberg
>>
>> marian.sauter at unibw.de
>> ------------------------------
>> *Von:* visionlist <visionlist-bounces at visionscience.com> im Auftrag von
>> Caspar Goeke <caspar.goeke at gmail.com>
>> *Gesendet:* Dienstag, 21. April 2020 08:01
>> *An:* visionlist at visionscience.com
>> *Betreff:* Re: [visionlist] Matlab --> online Experiments
>>
>>
>>
>>
>> * Hello everyone, After finishing my PhD in Cognitive Science I created a
>> platform for creating and running online psychology experiments
>> (https://www.labvanced.com <https://www.labvanced.com/>), it’s easy to
>> start and has a very powerful UI. You will find our source code is open and
>> available on https://github.com/Labvanced/ <https://github.com/Labvanced/>.
>> Also we have an open access experiment library, in which there are hundreds
>> of templates for all kinds of experiments, freely to import and share for
>> everyone.  (https://www.labvanced.com/expLibrary.html
>> <https://www.labvanced.com/expLibrary.html>). I hope with that we can be of
>> value for some of you in this crisis.  Best, Caspar *
>>
>> Dear Maarten + All:
>>
>>
>> …We have been working with a Processing inspired javascript library called p5.js,
>>
>> Yes, a few of my phenomena pages are based on ithttps://michaelbach.de/ot/-misc/p5/
>>
>> In my experience its biggest plus is that it is very easy, very approachable, thus very rapidly you can get astounding results.
>>
>> What I found lacking is a good GUI (but then I’m possibly spoiled by <https://michaelbach.de/ot/-misc/cappuccino/> <https://michaelbach.de/ot/-misc/cappuccino/>), a little in timing, some deficiencies in sound*, and a possibly slow 3D implementation (but then, again, this <https://michaelbach.de/ot/sze-silhouette/> <https://michaelbach.de/ot/sze-silhouette/> does put heavy demand on on-line 3D processing).
>>
>>
>> … Anyhow, here (https://arxiv.org/abs/2004.08198) is a preprint, …
>>
>> Impressive, thank you.
>> Clearly p5.js has lots going for it.
>>
>>
>> Best, Michael
>>
>>
>> _______________________________________________
>> visionlist mailing list
>> visionlist at visionscience.com
>> http://visionscience.com/mailman/listinfo/visionlist_visionscience.com
>>
>
> _______________________________________________
> visionlist mailing listvisionlist at visionscience.comhttp://visionscience.com/mailman/listinfo/visionlist_visionscience.com
>
> --
> Jonathan Peirce
> University of Nottingham
> https://www.peirce.org.ukhttps://psychopy.org
>
> _______________________________________________
> visionlist mailing list
> visionlist at visionscience.com
> http://visionscience.com/mailman/listinfo/visionlist_visionscience.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20200503/d98c79a4/attachment.html>