# How to repeat incorrect trials only

**URL:** <https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908>\
**Category:** Builder\
**Created:** [March 5, 2019, 12:22am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908 "2019-03-05T00:22:30Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paige27](https://avatars.discourse-cdn.com/v4/letter/p/46a35a/32.png) [@Paige27](https://discourse.psychopy.org/u/Paige27)\
**Post date:** [March 5, 2019, 12:22am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/1 "2019-03-05T00:22:30Z")

</div>

**What are you trying to achieve?:**  
I am running a two-alternative forced choice task where participants make judgements about a set of images. It begins with practice task, where participants need to get 100% correct to continue to the main task. I am looking to have incorrect items repeated in a random order.

**What did you try to make it work?:**  
I created an outer loop (with the routine loop nested within it) that repeats the practice trials if less than 100% of items within the first round were incorrect, as per these instructions: [Repeat trial if answers are incorrect](https://discourse.psychopy.org/t/repeat-trial-if-answers-are-incorrect/2336/8)

**What specifically went wrong when you tried that?:**  
It worked well, but all the practice items are repeated when less than 100% accuracy is achieved, rather than just the previously incorrectly answered items.

I am new to Psychopy and have looked on loads of forums to work out a way around this but haven’t been able to make anything work.

Thanks in advance!!

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [March 5, 2019, 10:28pm UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/2 "2019-03-05T22:28:40Z")

</div>

I’m not entirely sure this will be possible in Builder with a random loop, but a possible route to investigate would be

- controlling the selected rows field of the inner loop. In the first run through, all rows would be selected but afterwards, you would just select the subset of desired rows where the response was incorrect.

I’m just not sure that would really work, because if the loop is randomised, the row numbers might not correspond any more on a subsequent loop. So perhaps this would be an alternative:

- In your conditions file, include a column number with a unique identifier per row (it could just be sequential numbers 1,2,3,…), let’s say that column is called `row_id`
- At the beginning of the routine, on the very first trial (when the iteration number of both loops is zero), initialise a list like this:

```auto
if outer_loop_name.thisN == 0 and inner_loop_name.thisN == 0:
    wrong_list = []

```

- On each incorrect trial, add the unique id to that list, like this:

```auto
if outer_loop_name.thisN == 0 and blah blah: # where blah blah is your check that the response was incorrect
    wrong_list.append(row_id)

```

Then also include some code like this in the "begin routine " tab for running in the second pass through the loop:

```auto
# one the second run through the loop, only run a trial if it was previously
# recorded as incorrect:
if outer_loop_name.thisN == 1:
    if row_id is not in wrong_list:
        continueRoutine = False

```

The above is just a sketch of what to do, and would need to be extended if there could be additional runs until the loop is completely perfectly.

---

<div class="post-metadata">

**Author:** ![Paige27](https://avatars.discourse-cdn.com/v4/letter/p/46a35a/32.png) [@Paige27](https://discourse.psychopy.org/u/Paige27)\
**Post date:** [March 6, 2019, 8:57am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/3 "2019-03-06T08:57:08Z")

</div>

Hi Michael,

That’s super helpful! Thanks so much. I am now able to get the incorrect items to be repeated.

However, after the incorrect items were repeated and answered correctly, a new set of practice questions commenced. This seems to be because the code I used to terminate the practice trials relies on a full set of items to be answered correctly.

I tried to play around with the code to change this, but it got a bit messy and did not work. Any advice for something simpler would be appreciated !!

I’ve added some screenshots incase it’s helpful.

 ![42%20pm](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/b/7/b75955a10a4d8838db0e7e81e67c94a6b837122b.png)  
 ![51%20pm](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/c/c/cc43aa23d700b4b628253ef4fd4aa72321d6d746.png)

---

<div class="post-metadata">

**Author:** ![Paige27](https://avatars.discourse-cdn.com/v4/letter/p/46a35a/32.png) [@Paige27](https://discourse.psychopy.org/u/Paige27)\
**Post date:** [March 6, 2019, 8:57am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/4 "2019-03-06T08:57:49Z")

</div>

![17%20pm](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/b/c/bc8fb0ad8e2f2706f0091cd90d1eb02fe2e570d4.png)

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [March 8, 2019, 12:00am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/5 "2019-03-08T00:00:24Z")

</div>

> [@Paige27](#):
>
> after the incorrect items were repeated and answered correctly, a new set of practice questions commenced. This seems to be because the code I used to terminate the practice trials relies on a full set of items to be answered correctly.

In the “Begin routine” section, you could do something like this:

```auto
# don't run practice a third time:
if outer_loop_name.thisN == 2:
    outer_loop_name.finished = True
    inner_loop_name.finished = True
    continue_routine = False

```

You might also need to insert a check in the “end routine” tab so that code doesn’t run on the third rep, or work this capacity into that tab, in which case you would need to check if the outer loop was on the _second_ iteration _and_ the inner loop on its last iteration.

---

<div class="post-metadata">

**Author:** ![Paige27](https://avatars.discourse-cdn.com/v4/letter/p/46a35a/32.png) [@Paige27](https://discourse.psychopy.org/u/Paige27)\
**Post date:** [April 11, 2019, 6:01am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/6 "2019-04-11T06:01:44Z")

</div>

This worked. Thanks Michael!

---

<div class="post-metadata">

**Author:** ![Lili1](https://avatars.discourse-cdn.com/v4/letter/l/b19c9b/32.png) [@Lili1](https://discourse.psychopy.org/u/Lili1)\
**Post date:** [July 23, 2020, 11:52am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/7 "2020-07-23T11:52:32Z")

</div>

I relied on this thread for my experiment. It works for the first round but after that, there are no iterations in the wrong\_list and all the items that initially were answered incorrectly are repeated.  
In our experiment, we want to repeat the incorrect trials only until everything is answered correctly.

 ![Screenshot (257)](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/a/2/a2abc56d0d357c2d52c91189e93e6561d774d6b1.png)  
 ![Screenshot (256)](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/5/5/55b7c577fe5993a09df17f12989169aac7f5bdbf.png)

I would really appreciate it if you could help me with this one!!

---

<div class="post-metadata">

**Author:** ![h.ashida](https://avatars.discourse-cdn.com/v4/letter/h/85e7bf/32.png) [@h.ashida](https://discourse.psychopy.org/u/h.ashida)\
**Post date:** [October 22, 2020, 2:13pm UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/8 "2020-10-22T14:13:47Z")

</div>

I thank you all for this useful thread, as we just need to repeat the trials until 100% correct (which seems to be a rather common case). Now I don’t need further help, but I would add some notes in case somebody may find it useful.

I wonder why the outer loop is necessary, as a single `random` loop can do it. One problem would be how to judge if we are in the first round or not, and this seems to be checked by something like  
`if loop_name.thisN >= len(loop_name.trialList):`  
unless there is a variable that represents the current number of repetition (I’m not sure).

Also, it should be better to remove the stimulus from wrong\_list (`wrong_list.remove(row_id`) once the response gets correct. This can allow third or further round to get really 100% correct, which I don’t think is done in the above discussion of disabling the third round. (nRep should be set large enough, and the loop should be finished when wrong\_list is empty at any point after the first round.)

---

<div class="post-metadata">

**Author:** ![m.hamzeloo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/m.hamzeloo/32/8130_2.png) [@m.hamzeloo](https://discourse.psychopy.org/u/m.hamzeloo)\
**Post date:** [July 27, 2021, 5:51pm UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/9 "2021-07-27T17:51:47Z")

</div>

Dear All @Michael, @wakecarter, and @jon,  
I am trying to build an Audio-visual paired association test. In the training block, I want to repeat incorrect trials until receiving a correct response.  
As you can see in the pictures, I created an outer loop with the name trials\_9 to repeat incorrect trials.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/1/1/1199a1604b3ee181d59828b4840f2f81af3f3730.png)  
 ![image](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/3/d/3d91ff1c2fa030869d1ee1a54f4ff2e1499a9a1e.png)  
 ![image](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/e/a/ea7b25857df5388f4ea122da285309be6578423a.png)

In the first round, everything goes well but for the second loop in which incorrect trials must be repeated I received this error:  
File “D:\Hildesheim University activities\Audio-visual association training ver.1\Audio-visual association training ver.1\_lastrun.py”, line 713, in   
if mouse\_9.clicked\_name[0] == CorrAns:  
IndexError: list index out of range

##### Experiment ended.

I would appreciate it if anyone can help me!!

---

<div class="post-metadata">

**Author:** ![wakecarter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/wakecarter/32/5753_2.png) [@wakecarter](https://discourse.psychopy.org/u/wakecarter)\
**Post date:** [July 28, 2021, 8:37am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/10 "2021-07-28T08:37:46Z")

</div>

I think that your issue is that you have code that skips the routine during your second loop but your code in End Routine will run even when the routine is skipped – and the mouse hasn’t clicked on any objects.

You need to add the same conditional to your End Routine code so it also gets skipped.

---

<div class="post-metadata">

**Author:** ![m.hamzeloo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/m.hamzeloo/32/8130_2.png) [@m.hamzeloo](https://discourse.psychopy.org/u/m.hamzeloo)\
**Post date:** [July 28, 2021, 8:46am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/11 "2021-07-28T08:46:26Z")

</div>

Dear @wakecarter ,

I also put this part of my code in the End Routine but I received the same error!

if trials\_9.thisN == 1:  
if row\_ID not in wrong\_list:  
continueRoutine = False

 ![image](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/4/4/44fbfcac6ea762a89ac6a6453e808b2d32a3bae6.png)

---

<div class="post-metadata">

**Author:** ![wakecarter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/wakecarter/32/5753_2.png) [@wakecarter](https://discourse.psychopy.org/u/wakecarter)\
**Post date:** [July 28, 2021, 9:03am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/12 "2021-07-28T09:03:15Z")

</div>

Please could you past the current text of your End Routine Python code so I can edit it for you to show you what I mean?

---

<div class="post-metadata">

**Author:** ![m.hamzeloo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/m.hamzeloo/32/8130_2.png) [@m.hamzeloo](https://discourse.psychopy.org/u/m.hamzeloo)\
**Post date:** [July 28, 2021, 9:08am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/13 "2021-07-28T09:08:41Z")

</div>

```auto
if mouse_9.clicked_name[0] == CorrAns:
    number_correct = number_correct + 1
    msg = "Correct"
    msgcolor='green'
    mycorrAns = 1
 
else:
        msg = "Incorrect! Try again" 
        msgcolor='red'
        mycorrAns = 0

if testing_block9_loop.thisN + 1 == testing_block9_loop.nTotal:
    if number_correct/(testing_block9_loop.nTotal) >= 1:
        trials_9.finished = True
    else:
        trials_9.finished = False

if trials_9.thisN ==0 and mycorrAns == 0:
    wrong_list.append(row_ID)

if trials_9.thisN == 1:
    if row_ID not in wrong_list:
        continueRoutine = False

print(mouse_9.clicked_name) # DEBUG
print(type(mouse_9.clicked_name)) # DEBUG
print(CorrAns) # DEBUG
print(TrialType)
print(mycorrAns)

```

---

<div class="post-metadata">

**Author:** ![m.hamzeloo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/m.hamzeloo/32/8130_2.png) [@m.hamzeloo](https://discourse.psychopy.org/u/m.hamzeloo)\
**Post date:** [July 28, 2021, 9:09am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/14 "2021-07-28T09:09:21Z")

</div>

Thank you @wakecarter for your help and support.

---

<div class="post-metadata">

**Author:** ![wakecarter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/wakecarter/32/5753_2.png) [@wakecarter](https://discourse.psychopy.org/u/wakecarter)\
**Post date:** [July 28, 2021, 9:14am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/15 "2021-07-28T09:14:17Z")

</div>

Try this

```auto
if trials_9.thisN == 1 and row_ID not in wrong_list:
        pass
elif mouse_9.clicked_name[0] == CorrAns:
    number_correct = number_correct + 1
    msg = "Correct"
    msgcolor='green'
    mycorrAns = 1
else:
        msg = "Incorrect! Try again" 
        msgcolor='red'
        mycorrAns = 0

if testing_block9_loop.thisN + 1 == testing_block9_loop.nTotal:
    if number_correct/(testing_block9_loop.nTotal) >= 1:
        trials_9.finished = True
    else:
        trials_9.finished = False

if trials_9.thisN ==0 and mycorrAns == 0:
    wrong_list.append(row_ID)

```

---

<div class="post-metadata">

**Author:** ![m.hamzeloo](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/m.hamzeloo/32/8130_2.png) [@m.hamzeloo](https://discourse.psychopy.org/u/m.hamzeloo)\
**Post date:** [July 28, 2021, 9:31am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/16 "2021-07-28T09:31:52Z")

</div>

Thank you @wakecarter it works perfectly.  
Now I want to write code that says remove a row\_ID from the wrong\_list when it receives a correct answer and then finishes the outer loop if wrong\_list is empty. How can I write it?

---

<div class="post-metadata">

**Author:** ![wakecarter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/wakecarter/32/5753_2.png) [@wakecarter](https://discourse.psychopy.org/u/wakecarter)\
**Post date:** [July 28, 2021, 9:50am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/17 "2021-07-28T09:50:16Z")

</div>

That sounds like it should be in a new thread. You could add a link back to this one but should also describe how it is currently working and what is still needed.

---

<div class="post-metadata">

**Author:** ![SanjanaHari](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/sanjanahari/32/10396_2.png) [@SanjanaHari](https://discourse.psychopy.org/u/SanjanaHari)\
**Post date:** [October 29, 2021, 6:03am UTC](https://discourse.psychopy.org/t/how-to-repeat-incorrect-trials-only/6908/18 "2021-10-29T06:03:42Z")

</div>

Hello, I tried this code. It works perfectly offline but doesn’t work when I port it online. All the trials, not just the incorrect ones repeat the second time. Please help me out. I have attached the loop structure and the code snippet here.  
I am using Psychopy v2020.2.3

 ![Screenshot 2021-10-29 113035](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/c/7/c73a72544cd02b68ca1fe94d90a1a502455b4e76.jpeg)  
 ![Screenshot 2021-10-29 113115](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/5/0/50f6d01a10b5ed90cdc1473e03078071f79f4fb6.jpeg)  
 ![Screenshot 2021-10-29 113232](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/a/4/a4b01f2f1e5df85f7e650a3c6d125063b89368de.jpeg)  
 ![Screenshot 2021-10-29 113254](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/4/8/48107cdfc9262630c62bca119322ff6da579f86b.jpeg)