<div dir="auto"><div>Dear Jon and vision community,</div><div dir="auto"><br></div><div dir="auto">I would like to put a few things into context here and also ask some question.</div><div dir="auto"><br></div><div dir="auto">I read the paper you quoted, but our platform (<a href="https://www.labvanced.com/">https://www.labvanced.com/</a>) was not included in the comparison. So I am wondering whether this was a conscious choice or whether we were not on your radar yet (of course we would be glad to be included in future work)?</div><div dir="auto"><br></div><div dir="auto">In any case, I would like to stress the point that our system was build around timing accuracy, i.e. we use a pre-rendering approach, and precision < 3 ms can surely be realized with us. However, aggregates of accuracy between subjects are always a bit difficult, because individual differences can be quite high for various reasons. Hence we measure and provide timing accuracy for each measurement by default; and this can easily be extended to measure accuracy for each trial.</div><div dir="auto"><br></div><div dir="auto">Overall, I dont think there is a single best choice for all online experiments. At Labvanced for instance, we provide audio and (soon) video recordings, webcam based eye-tracking, multi-participants studies, and other features which in my understanding are not supported by default in PsychoPy for instance. Also with us one can execute custom clientside JavaScript but also connect to an external sever (API) via websockets, for instance to connect to some device like an EEG, or execute server-side functions for optinmization problems, machine learning etc. So we try to combine the best of both worlds, a powerful and intuitive graphical interface, but also coding options for (very) advanced usage.</div><div dir="auto"><br></div><div dir="auto">Needles to say, besides PsychoPy, there are also good native Javascript frameworks for web based studies. So my honest and unbiased oppinion to anyone in the community is: Try out various platforms/frameworks and pick the one best suited for your needs!</div><div dir="auto"><br><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">On Sun, May 3, 2020, 00:48 Jon Peirce <<a href="mailto:jon.peirce@gmail.com" target="_blank" rel="noreferrer">jon.peirce@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>Sorry, I'm a bit late to this discussion, but just to expand on
the discussion and give some more info about running studies
online.<br>
</p>
<p>We (the PsychoPy team) put in a lot of work over the last 2-3
years, intially funded by Wellcome Trust and now the Chan
Zuckerberg Initiative, to port our Python library to JavaScript
and allow the Builder interface to generate scripts in either
syntax. At this point I don't think I'm being biased when I say
that PsychoPy is your best bet for creating a single study that
can run both in the lab (maximal precision) and online (maximal
portability) with minimal translation hassles. <br>
</p>
<p>PsychoPy has excellent timing in both environments
<a href="https://psyarxiv.com/d6nu5/" rel="noreferrer noreferrer" target="_blank">https://psyarxiv.com/d6nu5/</a><br>
In the lab is achieves sub-ms keyboard and audio timing (it
actually uses the same keyboard and audio C code as PTB since
Mario ported that to Python). It also has best-in-class timing
online - it's the only package currently able to produce RT timing
precision under 3.5ms on all browsers (in conjunction with a
LabHacker Millikey button box).<br>
</p>
<p>For those that are interested I've spelled out below some of the
issues that needed solving for all this to be possible, and a few
that still aren't yet solved.</p>
<p>Hope the info is helpful.</p>
<p>best wishes all,<br>
Jon<br>
</p>
<p><b>Porting the <i>lib </i>to JS</b></p>
<p>As Dee points out, is that the functions themselves need to have
been ported to JS as well as just converting the syntax of your
script. Translating your PTB script to JS syntax is no use because
the Screen functions haven't been ported to JS. PTB's key selling
point, that a lot of the critical functions run as compiled C
code, is probably also the thing that will make it hard to port to
JS.</p>
<p>A substantial portion of the PsychoPy lib has now been ported to
JS. Not all stimuli are there but many are, and the calls to use
them are generally the same as in the Python lib.
<a href="https://www.psychopy.org/online/status.html" rel="noreferrer noreferrer" target="_blank">https://www.psychopy.org/online/status.html</a> In particular, you can
time your stimulus presentation by number of frames, and you can
update stimulus parameters on every frame! Timing can't currently
be *as good* as lab-based libraries (and don't listen to anybody
telling you otherwise) but it is much better than most people
imagine.</p>
<p>PsychoJS (the JS port of the lib) uses WebGL where available,
which allows a large range of hardware-accelerated graphics,
including GL shaders, to be used so we should ultimately be able
to create all your favourite stimuli online. :-)<br>
</p>
<p><b>Generating the JS experiment <i>script</i></b></p>
<p>PsychoPy Builder can also output your experiment in either format
- the Python script or the JS files. Python Code Components within
the Builder experiment can even be auto-translated to JS format!
Again, you can't use functions that don't exist, but the syntax
converts remarkably well, so you can augment your study with
conditionals, loops and even custom functions and these will all
be translated.</p>
<p>You can't take a hand-written script and auto-convert that. The
structural changes required for a full script mean that isn't
possible (and probably never will be). <br>
</p>
<p>That said, creating a study in Builder is really fast, so
starting from scratch isn't such a big deal, and Builder is the
approach we recommend even for skilled programmers anyway. It
typically means fewer mistakes, better timing and more
future-proof experiments ("future you" can still undertand the
study when it's graphical rather than 1000 lines of code). <br>
</p>
<p><b>What isn't done<br>
</b></p>
<p>The downside for vision scientists is that we expected this
community to be last to want to go online so the psychophysics
features have been last to be implemented. In particular, gamma
correction hasn't been done (although it could be using WebGL
Shaders) and we haven't ported the Gratings and similar stimuli
yet (mostly because these aren't the useful if gamma isn't
linearised). We also haven't yet ported over the staircase
procedures.<br>
</p>
<p>All these things can be done and we will get to them in due
course but, also, if you have JS skills you could jump in and help
us fill the gaps!<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Apr 24, 2020 at 8:23
AM Sauter Marian <<a href="mailto:marian.sauter@unibw.de" rel="noreferrer noreferrer" target="_blank">marian.sauter@unibw.de</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr" style="font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:Corbel,sans-serif">Dear
all,
<p><br>
</p>
<p>for those of you still seeking recommendations on how to
get your experiments online, our article just got
pusblished open access:</p>
<p><br>
</p>
<p>Sauter, M.; Draschkow, D.; Mack, W. <strong>Building,
Hosting and Recruiting: A Brief Introduction to Running
Behavioral Experiments Online</strong>.
<em>Brain Sci.</em> <strong>2020</strong>, <em>10</em>,
251. </p>
<p><br>
</p>
<p><a href="https://www.mdpi.com/699990" rel="noreferrer noreferrer" target="_blank">https://www.mdpi.com/699990</a></p>
<p><br>
</p>
<p>Best wishes,</p>
Marian
<p><br>
</p>
<p><br>
</p>
<div id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898Signature">
<div name="divtagdefaultwrapper">
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><b><span style="background-color:white">Dr. Marian Sauter</span></b><span style="background-color:white"></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><i><span style="background-color:white">Wissenschaftlicher
Mitarbeiter</span></i><br>
<span style="background-color:white">Universität der
Bundeswehr München</span><br>
<span style="background-color:white">Department für
Psychologie - Professur für Allgemeine Psychologie</span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><span style="background-color:white">Gb. 161 / Raum 1016</span><br>
<span style="background-color:white">Werner-Heisenberg-Weg
39</span><br>
<span style="background-color:white">85577 Neubiberg</span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><span style="font-size:11pt"><a href="mailto:marian.sauter@unibw.de" rel="noreferrer noreferrer" target="_blank">marian.sauter@unibw.de</a></span></p>
</div>
</div>
<div style="color:rgb(33,33,33)">
<hr style="display:inline-block;width:98%">
<div id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri,
sans-serif" color="#000000"><b>Von:</b> visionlist
<<a href="mailto:visionlist-bounces@visionscience.com" rel="noreferrer noreferrer" target="_blank">visionlist-bounces@visionscience.com</a>>
im Auftrag von Caspar Goeke <<a href="mailto:caspar.goeke@gmail.com" rel="noreferrer noreferrer" target="_blank">caspar.goeke@gmail.com</a>><br>
<b>Gesendet:</b> Dienstag, 21. April 2020 08:01<br>
<b>An:</b> <a href="mailto:visionlist@visionscience.com" rel="noreferrer noreferrer" target="_blank">visionlist@visionscience.com</a><br>
<b>Betreff:</b> Re: [visionlist] Matlab --> online
Experiments</font>
<div> </div>
</div>
<div>
<div><b id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898docs-internal-guid-d2960667-7fff-8782-725f-4df21a96ea0f" style="font-weight:normal">
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Hello
everyone,</span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">After
finishing my PhD in Cognitive Science I created a platform for creating and running online psychology experiments
</span><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(</span><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><a href="https://www.labvanced.com/" rel="noreferrer noreferrer" target="_blank">https://www.labvanced.com</a>),
it’s easy to start and has a very powerful UI. You will find our source code is open and available on
</span><a href="https://github.com/Labvanced/" style="text-decoration:none" rel="noreferrer noreferrer" target="_blank"><span style="font-size:11pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://github.com/Labvanced/</span></a><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">.
Also we have an open access experiment library, in which there are hundreds of templates for all kinds of experiments, freely to import and share for everyone. (</span><a href="https://www.labvanced.com/expLibrary.html" style="text-decoration:none" rel="noreferrer noreferrer" target="_blank"><span style="font-size:11pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://www.labvanced.com/expLibrary.html</span></a><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">).
I hope with that we can be of value for some of you in this crisis. </span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Best,
Caspar </span></p>
</b><br>
</div>
<blockquote type="cite">
<pre>Dear Maarten + All:
</pre>
<blockquote type="cite">
<pre>…We have been working with a Processing inspired javascript library called p5.js,
</pre>
</blockquote>
<pre>Yes, a few of my phenomena pages are based on it
<a href="https://michaelbach.de/ot/-misc/p5/" rel="noreferrer noreferrer" target="_blank">https://michaelbach.de/ot/-misc/p5/</a>
In my experience its biggest plus is that it is very easy, very approachable, thus very rapidly you can get astounding results.
What I found lacking is a good GUI (but then I’m possibly spoiled by <a href="https://michaelbach.de/ot/-misc/cappuccino/" rel="noreferrer noreferrer" target="_blank"><https://michaelbach.de/ot/-misc/cappuccino/></a>), a little in timing, some deficiencies in sound*, and a possibly slow 3D implementation (but then, again, this <a href="https://michaelbach.de/ot/sze-silhouette/" rel="noreferrer noreferrer" target="_blank"><https://michaelbach.de/ot/sze-silhouette/></a> does put heavy demand on on-line 3D processing).
</pre>
<blockquote type="cite">
<pre>… Anyhow, here (<a href="https://arxiv.org/abs/2004.08198" rel="noreferrer noreferrer" target="_blank">https://arxiv.org/abs/2004.08198</a>) is a preprint, …
</pre>
</blockquote>
<pre>Impressive, thank you.
Clearly p5.js has lots going for it.
Best, Michael
</pre>
</blockquote>
<p><br>
</p>
</div>
</div>
</div>
_______________________________________________<br>
visionlist mailing list<br>
<a href="mailto:visionlist@visionscience.com" rel="noreferrer noreferrer" target="_blank">visionlist@visionscience.com</a><br>
<a href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com" rel="noreferrer noreferrer noreferrer" target="_blank">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a><br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
visionlist mailing list
<a href="mailto:visionlist@visionscience.com" rel="noreferrer noreferrer" target="_blank">visionlist@visionscience.com</a>
<a href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com" rel="noreferrer noreferrer" target="_blank">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a>
</pre>
</blockquote>
<pre cols="72">--
Jonathan Peirce
University of Nottingham
<a href="https://www.peirce.org.uk" rel="noreferrer noreferrer" target="_blank">https://www.peirce.org.uk</a>
<a href="https://psychopy.org" rel="noreferrer noreferrer" target="_blank">https://psychopy.org</a></pre>
</div>
_______________________________________________<br>
visionlist mailing list<br>
<a href="mailto:visionlist@visionscience.com" rel="noreferrer noreferrer" target="_blank">visionlist@visionscience.com</a><br>
<a href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com" rel="noreferrer noreferrer noreferrer" target="_blank">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a><br>
</blockquote></div></div></div>