# Randomly selecting a subset of 5 excel rows from groups of 10 rows on each trial

**URL:** <https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098>\
**Category:** Builder\
**Created:** [August 17, 2018, 10:55am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098 "2018-08-17T10:55:39Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![kirsten](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kirsten](https://discourse.psychopy.org/u/kirsten)\
**Post date:** [August 17, 2018, 10:55am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/1 "2018-08-17T10:55:39Z")

</div>

I am making an experiment in Builder with 5 blocks (Mac/standalone/v1.90.3). My 4 experimental adapt blocks work fine in their nested loops which randomly present each block using a block order excel file in the outer loops condition filed and $variable in the inner loops.

My control block must always be presented first, so has its own loop setup (perhaps this is not the most elegant way to do this). My question is about how I want to select rows from my control excel file in a particular way on each trial:

My .xlsx file has 40 rows each linking to a .jpg:  
The first 10 rows are part of group A  
Next 10 are part of B  
Next 10 group C  
Last 10 are D

How do I randomly select 5 rows from each group on each trial? The desired outcome is that I have 20 randomly selected (in a constrained way) .jpg files I can use as the ‘movie’ for each trial in the control block.

I don’t understand how I could make selected rows field do this.

I have been trying to work out how to import random and then shuffle conditions, but can’t see where to add the code snippets and/or which code tab.

I’ve not found this solution here

> [@Selecting different rows per iteration in a loop](https://discourse.psychopy.org/t/selecting-different-rows-per-iteration-in-a-loop/126):
>
> Dear users, I have a .xlsx file with 48 links to files that I use to get stimuli within a inner loop. Per inner loop 12 file are presented, in total the inner loop is repeated 4 times in an outer loop. How can I (in builder) define that stimulis: 1-12 are going to be presented in iteration 1, 13-24 in iteration 2 etc. I tried with selected row but I can only statically choose 1:12. Thank you for your help Best Vivian

or here

> [@Randomly selecting and grouping rows from a conditions file](https://discourse.psychopy.org/t/randomly-selecting-and-grouping-rows-from-a-conditions-file/4502):
>
> Apologies if this issue has been posted before. I have a conditions file called cond.xlsx with a single column and 6 rows, each containing a different item of fruit. [Capture] I would like to write a code component at the beginning of my experiment which randomly sorts the rows into three conditions, each containing 2 rows. Ideally this would create a ‘temporary’ dict object (apologies if I’m using incorrect terminology) that would allow me to call rows from each condition in subsequent loops …

or on other threads.

Can anyone help please?

 ![48%20pm](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/7/f/7f0dd1367ce429ef90c0b1eaa38ee6fa291d88a8.jpeg)

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [August 18, 2018, 4:49am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/2 "2018-08-18T04:49:00Z")

</div>

Hi @kirsten, you’re right, you can’t really make this work just by using the loop dialog options. The randomising and row selection work across the entire file, which is incompatible with having separate blocks within that file.

You need to put each block in its own `.xlsx` or `.csv` file. We then control the order of selection of those blocks with code. Insert a code component and put something like this in the “Begin Experiment” tab:

```auto
control_blocks = ['A.csv', 'B.csv', 'C.csv', 'D.csv']
shuffle(control_blocks) # randomise order

```

In the inner loop dialog for your control condition, put this in the conditions file field:

```
$control_blocks.pop()

```

and set the “selected rows” field to `0:5` to get just five of the rows (set the loop type to “random”).

Then surround that with an outer loop that isn’t connected to a conditions file but simply set to have an `nReps` of 4, so that the control condition runs four times (for a total of 20 trials).

Check your data output, I’m not sure if the conditions file name will be saved automatically for you (if that matters).

---

<div class="post-metadata">

**Author:** ![kirsten](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kirsten](https://discourse.psychopy.org/u/kirsten)\
**Post date:** [August 19, 2018, 1:57am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/3 "2018-08-19T01:57:22Z")

</div>

Dear Michael,  
thank you so much for this very helpful response. I have done as you suggested, but have hit a wall with a syntax error for the ‘selected rows’ field (see screenshot). If I use other notation like ‘0-1’ or ‘1,2,3’ or ‘{0:5}’ there is no problem, but of course these don’t do what I want. Reading other posts about selected row errors makes me think that I must have another error in my code - like perhaps the ordering of my components is wrong? But I haven’t been able to loose the error despite trying different orders.  
I have attached the files I have been working on, if that helps. The image folder has only 8 pictures.  
While I’m here I would also like to mention that I have gained a lot from your PsychoPy book. It’s really nicely written.  
Thanks,  
Kirsten

 ![47%20am](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/0/b/0b4b0c89af9fd0a33a0fca4477cb294900a23308.jpeg)  
[Adapt\_draft7aTest.psyexp](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/8/8431ca8fa6850cb4f524570682e3ff6c4e020c19.psyexp) (11.7 KB)  
[BlockSalty.xlsx](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/6/6f8f4d81b056a2d684fc0e7385ef76361bd09be6.xlsx) (8.7 KB)  
[BlockSweet.xlsx](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/3/305fba6ca6a6140b8ff3e7bd1d00039475d91ae1.xlsx) (8.7 KB)

> **[Images](https://www.dropbox.com/sh/pha89wkbyrefj05/AACHgfEHNiKUHFhBscoH0Rmta?dl=0)**
>
> Shared with Dropbox

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [August 19, 2018, 10:14pm UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/4 "2018-08-19T22:14:05Z")

</div>

It seems to me that adding an expression in the “selected rows” field can be a bit fragile. Once the loop dialog box gets convinced it contains a syntax error, it is hard to convince it otherwise. Perhaps it might be easiest to delete the loop and try re-inserting it. I managed to type `0:5` in your experiment without it raising an error, for example.

---

<div class="post-metadata">

**Author:** ![kirsten](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kirsten](https://discourse.psychopy.org/u/kirsten)\
**Post date:** [August 19, 2018, 11:52pm UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/5 "2018-08-19T23:52:19Z")

</div>

Thank you Michael. Thank you so much. After fulling closing and reopening Psychopy and carefully entering 0:5 it now runs with no errors. However, it is the case that the same images are always being presented. Not a random selection of the 0:5 (or 0:1) for example.  
I’ve set both loops to full random. I’m now wondering about the random seed?

…And FYI, yes, you’re right that the data output does not save the conditions file name, but that is no problem for me.

Many thanks,  
Kirsten

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [August 20, 2018, 12:44am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/6 "2018-08-20T00:44:54Z")

</div>

> [@kirsten](#):
>
> I’ve set both loops to full random.

“Full random” only applies when there is more than one repetition of a conditions file. Just “random” is sufficient here (but won’t actually alter anything here).

> [@kirsten](#):
>
> I’m now wondering about the random seed?

You shouldn’t be specifying a seed (that makes the random order constant across different runs).

If you haven’t specified a seed, then I’m not sure how you can be getting the same random selection on different runs. Are the images constant, but the order changes, or are both the order and the images constant from one run to the next?

If you just enter `0` in the selected rows field, do you always get the same image, and if so, is it the first one in your conditions file?

---

<div class="post-metadata">

**Author:** ![kirsten](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kirsten](https://discourse.psychopy.org/u/kirsten)\
**Post date:** [August 20, 2018, 1:18am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/7 "2018-08-20T01:18:42Z")

</div>

Okay yes.  
The images are constant, but the order changes.  
Yes, when set to selected rows in 0, it is always the same image, and it is the first one in the conditions file.

So the presentation might be one run  
cake, cucumber  
and the cucumber, cake  
on the next run.  
Where cake is first row of conditions file A  
and cucumber is first row of conditions file B.

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [August 20, 2018, 2:35am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/8 "2018-08-20T02:35:02Z")

</div>

Sorry, I always get this wrong: the selected rows are extracted first, prior to the randomisation being applied, which is why you are getting a variable order but from a constant selection of rows. So instead of specifying rows `0:5`, which will always be the first ones, we need to provide a list of 5 random, potentially non-contiguous indices from across the whole span of row numbers. So if there are 10 rows in your conditions file, you could try an expression like this in the selected rows field:

```
choice(10, size=5, replace=False)

```

In the “begin experiment” tab, you would first need to import that function:

```
from numpy.random import choice
```

---

<div class="post-metadata">

**Author:** ![kirsten](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kirsten](https://discourse.psychopy.org/u/kirsten)\
**Post date:** [August 20, 2018, 3:04am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/9 "2018-08-20T03:04:04Z")

</div>

Thank YOU! This works perfectly. I just popped a $ in front of `$choice(10, size=5, replace=False).  
I have attached the working exp for anyone else who is interested in this thread.  
[RandomSelectSubset.psyexp](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/1/1db3752772f4c79910ff176eb0dabf1a61ce078e.psyexp) (7.2 KB)  
[BlockSalty.xlsx](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/1/18cc3c65fa92519979e7b6d0297522ea8e24fe6d.xlsx) (8.6 KB)  
[BlockSweet.xlsx](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/2X/c/c80034e47db2d25a74ceb67c41efddf4a0778d28.xlsx) (8.6 KB)

YAY… it looks like we can start testing those participants that are booked for tomorrow! Great news! Thanks again,  
Kirsten  
`

---

<div class="post-metadata">

**Author:** ![Emma2](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@Emma2](https://discourse.psychopy.org/u/Emma2)\
**Post date:** [October 15, 2020, 11:48am UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/10 "2020-10-15T11:48:32Z")

</div>

Thank you @Michael for this great explanation of the issue above. If I can ask one more think on top of the rest: Now we are popping a “csv file” at the beginning of each routine which logically results in an empty list at the end. So after the iterations the “control\_blocks” is empty as we have popped out the choices. How can I restore this list?

I have tried adding the control\_blocks = [‘mycsv1.csv’, ‘mycsv2.csv’, ‘mycsv3.csv’, ‘mycsv4.csv’] at the start of the routine but then it doesn’t run (which actually makes sense) .

The reason that I want to restore it is because on the above example if we set the nReps to 4 then it will go 4 times through mycsv1, then continue 4 times through mycsv2 and so on. What I want is go once into mycsv1, then continue to mycsv2 and so on… but iterate over this 4 times.  
If I add an outer loop of doing this with 4 times it stops after the first iteration. I speculate this is because the control\_blocks list is empty now.

Thanks

 ![image](https://us1.discourse-cdn.com/flex002/uploads/psychopy/original/3X/d/5/d59705b9ee4c8faf3f6361e036409ed46cfb2623.png)

---

<div class="post-metadata">

**Author:** ![Michael](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.psychopy.org/michael/32/16_2.png) [@Michael](https://discourse.psychopy.org/u/Michael)\
**Post date:** [October 19, 2020, 8:38pm UTC](https://discourse.psychopy.org/t/randomly-selecting-a-subset-of-5-excel-rows-from-groups-of-10-rows-on-each-trial/5098/11 "2020-10-19T20:38:06Z")

</div>

I’m not sure I entirely understand what is going on here, but you don’t need to use `.pop()` to get an item from the list. You could just get an item from the list by an index, which would leave the list intact. e.g.

```
$operation_blocks[some_loop.thisN]

```

would give you the zeroth item from the list on the first iteration of the enclosing loop (use whatever loop makes sense, Guess here it is the next enclosing loop, called `trials`? (it does make sense to give loops descriptive names so their function is clear)).

Presumably you’ll want to shuffle that list at the start of each block, so insert a code component and put something like this in the “begin routine” tab:

```auto
if whatever_loop_name.thisN == 0:
    shuffle(operation_blocks)

```