<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 03/05/2020 08:01, Caspar Goeke
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">
<div>Dear Jon and vision community,</div>
<div dir="auto"><br>
</div>
<div dir="auto">I would like to put a few things into context
here and also ask some question.</div>
<div dir="auto"><br>
</div>
<div dir="auto">I read the paper you quoted, but our platform (<a
href="https://www.labvanced.com/" moz-do-not-send="true">https://www.labvanced.com/</a>)
was not included in the comparison. So I am wondering whether
this was a conscious choice or whether we were not on your
radar yet (of course we would be glad to be included in future
work)?</div>
</div>
</blockquote>
Simple lack of resources I'm afraid. There was obviously a huge time
commitment for each package we tested given that we had to learn the
package and then run 1000s of trials on each combination of
operating system and browser. Sadly many options could not be
included as a result (PsyToolkit, tatool, EventIDE, OSweb,
Experimaker... are just a few that we didn't have time to test)<br>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<div dir="auto">
<div dir="auto"><br>
</div>
<div dir="auto">In any case, I would like to stress the point
that our system was build around timing accuracy, i.e. we use
a pre-rendering approach, and precision < 3 ms can surely
be realized with us. </div>
</div>
</blockquote>
<p>I'm sure the authors of all the packages would say the same. I'm
sure all are pre-rendering stimuli, and all are using sub-ms
clocks. Nonetheless, we see found a lot of difference across
packages and browsers, mostly *not* achieving sub-3ms precision,
and none matching the lab-based packages. Audio-visual synchrony
was particularly poor acorss all the packages we tested, although
if Labvanced has cracked that nut it would be great to hear!
Ultimately, the challenge is that the browsers are black boxes
that differ in many ways (use of graphics buffers, keyboard
polling rates...) and they aren't designed for things we need
(like millisecond-precise audio onsets). As an example, in the
desktop, PTB and PsychoPy directly poll the USB port for key
presses at whatever polling rate we choose. Conversely, in
JavaScript, the script polls the browser, which polls the
operating system, which polls the USB port, and each step can add
delays and variability.<br>
</p>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<div dir="auto">
<div dir="auto"><br>
</div>
<div dir="auto">Overall, I dont think there is a single best
choice for all online experiments. </div>
</div>
</blockquote>
Agreed. If you want to run online studies only (or lab-based studies
only) then there are many options. I was specifically referring to
the case where users wanted to create a study once and run it both
in the lab (with maximal precision) and online.<br>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<div dir="auto">
<div dir="auto"><br>
</div>
<div dir="auto">Needles to say, besides PsychoPy, there are also
good native Javascript frameworks for web based studies. </div>
</div>
</blockquote>
This is true too although, again, a word of caution that
people/packages that have given this *a lot* of thought have come up
with solutions not all equal in timing precision. If you neeed very
precise timing then make sure you test the actual precision of your
study using a BBTK to make sure you are getting the precision you
expect/need.<br>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<div dir="auto">
<div dir="auto">So my honest and unbiased oppinion to anyone in
the community is: Try out various platforms/frameworks and
pick the one best suited for your needs!</div>
<div dir="auto"><br>
</div>
</div>
</blockquote>
<p>Agreed!</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CANCEh0oO_kzEHXGRUg43w+-_9v88dy+2H+dUsSP-Wi2PZd0kcg@mail.gmail.com">
<div dir="auto">
<div dir="auto"> <br>
<div class="gmail_quote" dir="auto">
<div dir="ltr" class="gmail_attr">On Sun, May 3, 2020, 00:48
Jon Peirce <<a href="mailto:jon.peirce@gmail.com"
target="_blank" rel="noreferrer" moz-do-not-send="true">jon.peirce@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>Sorry, I'm a bit late to this discussion, but just to
expand on the discussion and give some more info about
running studies online.<br>
</p>
<p>We (the PsychoPy team) put in a lot of work over the
last 2-3 years, intially funded by Wellcome Trust and
now the Chan Zuckerberg Initiative, to port our Python
library to JavaScript and allow the Builder interface
to generate scripts in either syntax. At this point I
don't think I'm being biased when I say that PsychoPy
is your best bet for creating a single study that can
run both in the lab (maximal precision) and online
(maximal portability) with minimal translation
hassles. <br>
</p>
<p>PsychoPy has excellent timing in both environments <a
href="https://psyarxiv.com/d6nu5/" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">https://psyarxiv.com/d6nu5/</a><br>
In the lab is achieves sub-ms keyboard and audio
timing (it actually uses the same keyboard and audio C
code as PTB since Mario ported that to Python). It
also has best-in-class timing online - it's the only
package currently able to produce RT timing precision
under 3.5ms on all browsers (in conjunction with a
LabHacker Millikey button box).<br>
</p>
<p>For those that are interested I've spelled out below
some of the issues that needed solving for all this to
be possible, and a few that still aren't yet solved.</p>
<p>Hope the info is helpful.</p>
<p>best wishes all,<br>
Jon<br>
</p>
<p><b>Porting the <i>lib </i>to JS</b></p>
<p>As Dee points out, is that the functions themselves
need to have been ported to JS as well as just
converting the syntax of your script. Translating your
PTB script to JS syntax is no use because the Screen
functions haven't been ported to JS. PTB's key selling
point, that a lot of the critical functions run as
compiled C code, is probably also the thing that will
make it hard to port to JS.</p>
<p>A substantial portion of the PsychoPy lib has now
been ported to JS. Not all stimuli are there but many
are, and the calls to use them are generally the same
as in the Python lib. <a
href="https://www.psychopy.org/online/status.html"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.psychopy.org/online/status.html</a>
In particular, you can time your stimulus presentation
by number of frames, and you can update stimulus
parameters on every frame! Timing can't currently be
*as good* as lab-based libraries (and don't listen to
anybody telling you otherwise) but it is much better
than most people imagine.</p>
<p>PsychoJS (the JS port of the lib) uses WebGL where
available, which allows a large range of
hardware-accelerated graphics, including GL shaders,
to be used so we should ultimately be able to create
all your favourite stimuli online. :-)<br>
</p>
<p><b>Generating the JS experiment <i>script</i></b></p>
<p>PsychoPy Builder can also output your experiment in
either format - the Python script or the JS files.
Python Code Components within the Builder experiment
can even be auto-translated to JS format! Again, you
can't use functions that don't exist, but the syntax
converts remarkably well, so you can augment your
study with conditionals, loops and even custom
functions and these will all be translated.</p>
<p>You can't take a hand-written script and auto-convert
that. The structural changes required for a full
script mean that isn't possible (and probably never
will be). <br>
</p>
<p>That said, creating a study in Builder is really
fast, so starting from scratch isn't such a big deal,
and Builder is the approach we recommend even for
skilled programmers anyway. It typically means fewer
mistakes, better timing and more future-proof
experiments ("future you" can still undertand the
study when it's graphical rather than 1000 lines of
code). <br>
</p>
<p><b>What isn't done<br>
</b></p>
<p>The downside for vision scientists is that we
expected this community to be last to want to go
online so the psychophysics features have been last to
be implemented. In particular, gamma correction hasn't
been done (although it could be using WebGL Shaders)
and we haven't ported the Gratings and similar stimuli
yet (mostly because these aren't the useful if gamma
isn't linearised). We also haven't yet ported over the
staircase procedures.<br>
</p>
<p>All these things can be done and we will get to them
in due course but, also, if you have JS skills you
could jump in and help us fill the gaps!<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Apr 24,
2020 at 8:23 AM Sauter Marian <<a
href="mailto:marian.sauter@unibw.de"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">marian.sauter@unibw.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr"
style="font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:Corbel,sans-serif">Dear
all,
<p><br>
</p>
<p>for those of you still seeking
recommendations on how to get your experiments
online, our article just got pusblished open
access:</p>
<p><br>
</p>
<p>Sauter, M.; Draschkow, D.; Mack, W. <strong>Building,
Hosting and Recruiting: A Brief Introduction
to Running Behavioral Experiments Online</strong>.
<em>Brain Sci.</em> <strong>2020</strong>, <em>10</em>,
251. </p>
<p><br>
</p>
<p><a href="https://www.mdpi.com/699990"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.mdpi.com/699990</a></p>
<p><br>
</p>
<p>Best wishes,</p>
Marian
<p><br>
</p>
<p><br>
</p>
<div
id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898Signature">
<div name="divtagdefaultwrapper">
<p class="MsoNormal" style="margin:0cm 0cm
0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><b><span
style="background-color:white">Dr.
Marian Sauter</span></b><span
style="background-color:white"></span></p>
<p class="MsoNormal" style="margin:0cm 0cm
0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><i><span
style="background-color:white">Wissenschaftlicher
Mitarbeiter</span></i><br>
<span style="background-color:white">Universität
der Bundeswehr München</span><br>
<span style="background-color:white">Department
für Psychologie - Professur für
Allgemeine Psychologie</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><span
style="background-color:white">Gb. 161 /
Raum 1016</span><br>
<span style="background-color:white">Werner-Heisenberg-Weg
39</span><br>
<span style="background-color:white">85577
Neubiberg</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(33,33,33)"><span
style="font-size:11pt"><a
href="mailto:marian.sauter@unibw.de"
rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true">marian.sauter@unibw.de</a></span></p>
</div>
</div>
<div style="color:rgb(33,33,33)">
<hr style="display:inline-block;width:98%">
<div
id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898divRplyFwdMsg"
dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>Von:</b>
visionlist <<a
href="mailto:visionlist-bounces@visionscience.com"
rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true">visionlist-bounces@visionscience.com</a>>
im Auftrag von Caspar Goeke <<a
href="mailto:caspar.goeke@gmail.com"
rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true">caspar.goeke@gmail.com</a>><br>
<b>Gesendet:</b> Dienstag, 21. April 2020
08:01<br>
<b>An:</b> <a
href="mailto:visionlist@visionscience.com"
rel="noreferrer noreferrer"
target="_blank" moz-do-not-send="true">visionlist@visionscience.com</a><br>
<b>Betreff:</b> Re: [visionlist] Matlab
--> online Experiments</font>
<div> </div>
</div>
<div>
<div><b
id="m_-9088843248191148746m_5804060629006182580gmail-m_-8903291645951822898docs-internal-guid-d2960667-7fff-8782-725f-4df21a96ea0f"
style="font-weight:normal">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Hello
everyone,</span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">After
finishing my PhD in Cognitive Science I created a platform for creating and running online psychology experiments
</span><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(</span><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><a href="https://www.labvanced.com/" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">https://www.labvanced.com</a>),
it’s easy to start and has a very powerful UI. You will find our source code is open and available on
</span><a href="https://github.com/Labvanced/"
style="text-decoration:none"
rel="noreferrer noreferrer"
target="_blank"
moz-do-not-send="true"><span style="font-size:11pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://github.com/Labvanced/</span></a><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">.
Also we have an open access experiment library, in which there are hundreds of templates for all kinds of experiments, freely to import and share for everyone. (</span><a
href="https://www.labvanced.com/expLibrary.html"
style="text-decoration:none"
rel="noreferrer noreferrer"
target="_blank"
moz-do-not-send="true"><span style="font-size:11pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://www.labvanced.com/expLibrary.html</span></a><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">).
I hope with that we can be of value for some of you in this crisis. </span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Best,
Caspar </span></p>
</b><br>
</div>
<blockquote type="cite">
<pre>Dear Maarten + All:
</pre>
<blockquote type="cite">
<pre>…We have been working with a Processing inspired javascript library called p5.js,
</pre>
</blockquote>
<pre>Yes, a few of my phenomena pages are based on it
<a href="https://michaelbach.de/ot/-misc/p5/" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">https://michaelbach.de/ot/-misc/p5/</a>
In my experience its biggest plus is that it is very easy, very approachable, thus very rapidly you can get astounding results.
What I found lacking is a good GUI (but then I’m possibly spoiled by <a href="https://michaelbach.de/ot/-misc/cappuccino/" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true"><https://michaelbach.de/ot/-misc/cappuccino/></a>), a little in timing, some deficiencies in sound*, and a possibly slow 3D implementation (but then, again, this <a href="https://michaelbach.de/ot/sze-silhouette/" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true"><https://michaelbach.de/ot/sze-silhouette/></a> does put heavy demand on on-line 3D processing).
</pre>
<blockquote type="cite">
<pre>… Anyhow, here (<a href="https://arxiv.org/abs/2004.08198" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">https://arxiv.org/abs/2004.08198</a>) is a preprint, …
</pre>
</blockquote>
<pre>Impressive, thank you.
Clearly p5.js has lots going for it.
Best, Michael
</pre>
</blockquote>
<p><br>
</p>
</div>
</div>
</div>
_______________________________________________<br>
visionlist mailing list<br>
<a href="mailto:visionlist@visionscience.com"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">visionlist@visionscience.com</a><br>
<a
href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com"
rel="noreferrer noreferrer noreferrer"
target="_blank" moz-do-not-send="true">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a><br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
visionlist mailing list
<a href="mailto:visionlist@visionscience.com" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">visionlist@visionscience.com</a>
<a href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a>
</pre>
</blockquote>
<pre cols="72">--
Jonathan Peirce
University of Nottingham
<a href="https://www.peirce.org.uk" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">https://www.peirce.org.uk</a>
<a href="https://psychopy.org" rel="noreferrer noreferrer" target="_blank" moz-do-not-send="true">https://psychopy.org</a></pre>
</div>
_______________________________________________<br>
visionlist mailing list<br>
<a href="mailto:visionlist@visionscience.com"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">visionlist@visionscience.com</a><br>
<a
href="http://visionscience.com/mailman/listinfo/visionlist_visionscience.com"
rel="noreferrer noreferrer noreferrer" target="_blank"
moz-do-not-send="true">http://visionscience.com/mailman/listinfo/visionlist_visionscience.com</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Jonathan Peirce
University of Nottingham
<a class="moz-txt-link-freetext" href="https://www.peirce.org.uk">https://www.peirce.org.uk</a>
<a class="moz-txt-link-freetext" href="https://psychopy.org">https://psychopy.org</a></pre>
</body>
</html>