random-parameter-set
Overview
Randomizer that provides flexible condition assignment and counterbalancing by
allowing the user to specify an arbitrary sequence of frames to create. A
set of parameters is randomly selected from a list of available parameterSets,
and these parameters are substituted in to the parameters specified in the
list of frames.
To use, define a frame with ‘kind’: ‘choice’ and ‘sampler’: ‘random-parameter-set’, as shown below, in addition to the parameters described under ‘properties’.
{
...
"frames": {
...
"test-trials": {
"sampler": "random-parameter-set",
"kind": "choice",
...
}
}
}
In addition, there are two special properties you need to define to
use the random-parameter-set randomizer:
frameList: this is just what it sounds like: a list of all the frames that should be generated by this randomizer. Each frame is a JSON object just like you would use in the overall schema, with three differences:Rather than giving each frame in the list its own name (e.g. “test_trial1”), the frameList automatically generates these names for each of the frames within it, so you can begin each frame with “kind”: “exp-lookit-text”, etc. rather than nesting it under another identifier (examples of this can be found below).
You can define default properties, to share across all of the frames generated by this randomizer, in the JSON object
commonFramePropertiesinstead, as a convenience.You can use placeholder strings for any of the properties in the frame; they will be replaced based on the values in the selected
parameterSet.
parameterSetsis a list of mappings from placeholder strings to actual values. When a participant starts your study, one of these sets will be randomly selected, and any parameter values in theframeList(includingcommonFrameProperties) that match any of the keys in this parameter set will be replaced.
Examples
Let’s walk through an example of using this randomizer. Suppose we start with the following JSON document describing a study that includes instructions, an experimental manipulation asking participants to think about how delicious broccoli is, and an exit survey:
{
"frames": {
"instructions": {
"id": "text-1",
"blocks": [
{
"text": "Some introductory text about this study."
},
{
"text": "Here's what's going to happen! You're going to think about how tasty broccoli is."
}
],
"showPreviousButton": false,
"kind": "exp-lookit-text"
},
"manipulation": {
"id": "text-2",
"blocks": [
{
"text": "Think about how delicious broccoli is."
},
{
"text": "It is so tasty!"
}
],
"showPreviousButton": true,
"kind": "exp-lookit-text"
},
"exit-survey": {
"debriefing": {
"text": "Thank you for participating in this study! ",
"title": "Thank you!"
},
"id": "exit-survey",
"kind": "exp-lookit-exit-survey"
}
},
"sequence": [
"instructions",
"manipulation",
"exit-survey"
]
}
But what we really want to do is have some kids think about how tasty
broccoli is, and others think about how yucky it is! We can use a
random-parameter-set frame to replace both text frames:
{
"frames": {
"instruct-and-manip": {
"sampler": "random-parameter-set",
"kind": "choice",
"id": "instruct-and-manip",
"frameList": [
{
"blocks": [
{
"text": "Some introductory text about this study."
},
{
"text": "INTROTEXT"
}
],
"showPreviousButton": false
},
{
"blocks": [
{
"text": "MANIP-TEXT-1"
},
{
"text": "MANIP-TEXT-2"
}
],
"showPreviousButton": true
}
],
"commonFrameProperties": {
"kind": "exp-lookit-text"
},
"parameterSets": [
{
"INTROTEXT": "Here's what's going to happen! You're going to think about how tasty broccoli is.",
"MANIP-TEXT-1": "Think about how delicious broccoli is.",
"MANIP-TEXT-2": "It is so tasty!"
},
{
"INTROTEXT": "Here's what's going to happen! You're going to think about how disgusting broccoli is.",
"MANIP-TEXT-1": "Think about how disgusting broccoli is.",
"MANIP-TEXT-2": "It is so yucky!"
}
]
},
"exit-survey": {
"debriefing": {
"text": "Thank you for participating in this study! ",
"title": "Thank you!"
},
"id": "exit-survey",
"kind": "exp-lookit-exit-survey"
}
},
"sequence": [
"instruct-and-manip",
"exit-survey"
]
}
Notice that since both of the frames in the frameList were of the
same kind, we could define the kind in commonFrameProperties. We no
longer define id values for the frames, as they will be
automatically identified as instruct-and-manip-1 and
instruct-and-manip-2.
When the “instruct-and-manip” randomizer is evaluated, the Lookit experiment player will start with the frameList and add the key-value pairs in commonFrameProperties to each frame (not overwriting existing pairs):
[
{
"kind": "exp-lookit-text",
"blocks": [
{
"text": "Some introductory text about this study."
},
{
"text": "INTROTEXT"
}
],
"showPreviousButton": false
},
{
"kind": "exp-lookit-text",
"blocks": [
{
"text": "MANIP-TEXT-1"
},
{
"text": "MANIP-TEXT-2"
}
],
"showPreviousButton": true
}
]
Next, one of the two objects in parameterSets is selected randomly.
(By default, parameter sets are weighted equally, but
parameterSetWeights can be provided as an optional key in the
random-parameter-set frame.) If provided, parameterSetWeights
should be an array of relative weights for the parameter sets,
corresponding to the order they are listed. For instance, if we wanted
75% of participants to think about how tasty broccoli is, we could set
parameterSetWeights to [3, 1]. This allows uneven condition
assignment where needed to optimize power, as well as allowing
researchers to stop testing conditions that already have enough
participants as data collection proceeds.
Suppose that in this case the second parameter set is selected:
{
"INTROTEXT": "Here's what's going to happen! You're going to think about how disgusting broccoli is.",
"MANIP-TEXT-1": "Think about how disgusting broccoli is.",
"MANIP-TEXT-2": "It is so yucky!"
}
Now we return to the list of frames, and wherever any value matches one
of the keys in the parameterSet (even if that value is nested in
another object), it is replaced by the corresponding value from the
parameterSet, yielding the following final list of frames:
[
{
"kind": "exp-lookit-text",
"blocks": [
{
"text": "Some introductory text about this study."
},
{
"text": "Here's what's going to happen! You're going to think about how disgusting broccoli is."
}
],
"showPreviousButton": false
},
{
"kind": "exp-lookit-text",
"blocks": [
{
"text": "Think about how disgusting broccoli is."
},
{
"text": "It is so yucky!"
}
],
"showPreviousButton": true
}
]
Here is another example of how to use the “random-parameter-set” sampler in the context of different frame types (here, “exp-lookit-images-audio” and “exp-lookit-video”. In this case, the sampler will randomly pick one of the two parameterSets, displaying either 2 images of Zenna and then the bowl video, or 2 images of Remy and then the cup video. Note again how each frame does not have its own title within the frameList, and just begins with the definition of its kind.
{
"frames": {
"test-trials": {
"sampler": "random-parameter-set",
"kind": "choice",
"frameList": [
{
"kind": "exp-lookit-images-audio",
"images": [
{
"id": "happy",
"src": "FIRST_IMAGE_PLACEHOLDER",
"position": "left"
},
{
"id": "sad",
"src": "SECOND_IMAGE_PLACEHOLDER",
"position": "right"
}
]
},
{
"kind": "exp-lookit-video",
"video": {
"top": 10,
"left": 25,
"loop": false,
"width": 50,
"source": "VIDEO_PLACEHOLDER"
},
"autoProceed": true,
"doRecording": false,
"videoTypes": [
"mp4"
]
}
],
"commonFrameProperties": {
"baseDir": "https://www.mit.edu/~kimscott/placeholderstimuli/"
},
"parameterSets": [
{
"FIRST_IMAGE_PLACEHOLDER": "happy_zenna.jpg",
"SECOND_IMAGE_PLACEHOLDER": "sad_zenna.jpg",
"VIDEO_PLACEHOLDER": "cropped_bowl"
},
{
"FIRST_IMAGE_PLACEHOLDER": "happy_remy.jpg",
"SECOND_IMAGE_PLACEHOLDER": "sad_remy.jpg",
"VIDEO_PLACEHOLDER": "cropped_cup"
}
]
}
},
"sequence": [
"test-trials"
]
}
Parameters
- commonFrameProperties [Object]
Object describing common parameters to use in EVERY frame created by this randomizer. Parameter names and values are as described in the documentation for the frameType used.
- frameList [Array]
List of frames to be created by this randomizer. Each frame is an object with any necessary frame-specific properties specified. The
kindof frame can be specified either here (per frame) or incommonFrameProperties. If a property is defined for a given frame both in this frame list and incommonFrameProperties, the value in the frame list will take precedence.(E.g., you could include
'kind': 'normal-frame'incommmonFrameProperties, but for a single frame inframeList, include'kind': 'special-frame'.)Any property values within any of the frames in this list which match a property name in the selected
parameterSetwill be replaced by the correspondingparameterSetvalue. For example, suppose a frame inframeListis{ 'leftImage': 'LEFTIMAGE1', 'rightImage': 'frog.jpg', 'size': 'IMAGESIZE' }
and the row that has been selected randomly of parameterSets is
{ 'LEFTIMAGE1': 'toad.jpg', 'LEFTIMAGE2': 'dog.jpg', 'IMAGESIZE': 250 }
Then the frame would be transformed into:
{ 'leftImage': 'toad.jpg', 'rightImage': 'frog.jpg', 'size': 250 }
The same values may be applied across multiple frames. For instance, suppose frameList is
[ { 'leftImage': 'LEFTIMAGE1', 'rightImage': 'frog.jpg', 'size': 'IMAGESIZE' }, { 'leftImage': 'LEFTIMAGE2', 'rightImage': 'frog.jpg', 'size': 'IMAGESIZE' } ]
Then the corresponding processed frames would include the values
[ { 'leftImage': 'toad.jpg', 'rightImage': 'frog.jpg', 'size': 250 }, { 'leftImage': 'dog.jpg', 'rightImage': 'frog.jpg', 'size': 250 } ]
A property value like
IMAGESIZEmay be placed in a frame definition nested within another object (at any depth) or within a list and will still be replaced.You can also use selectors to randomly sample from or permute a list given in a
parameterSet. SupposeLISTVARis defined in aparameterSetasTHELIST, e.g. a list of potential stimuli. Within frames in yourframeList(and incommonFrameProperties), you can use any of the following:Select the Nth element (0-indexed) of
THELIST: (Will cause error ifN >= THELIST.length)'parameterName': 'LISTVAR#N'
Select (uniformly) a random element of
THELIST:'parameterName': 'LISTVAR#RAND'
Set
parameterNameto a random permutation ofTHELIST:'parameterName': 'LISTVAR#PERM'
Select the next element in a random permutation of
THELIST, which is used across all substitutions in this randomizer. This allows you, for instance, to provide a list of possible images in yourparameterSet, and use a different one each frame with the subset/order randomized per participant. If moreLISTVAR_UNIQparameters than elements ofTHELISTare used, we loop back around to the start of the permutation generated for this randomizer.'parameterName': 'LISTVAR#UNIQ'
- parameterSets [Array]
Array of parameter sets to randomly select from in order to determine the parameters for each frame in this session.
A single element of parameterSets will be applied to a given session.
- conditionForAdditionalSessions [String |
'random'] [Optional] How to select a parameterSet for a participant who has previously participated in this study. Must be one of
'random'(default),'persist', or'rotate'. Meanings:random: regardless of any previous sessions from this participant, select a parameterSet for this participant as usual (including using parameterSetWeights if provided). Default behavior.persist: Continue assigning the same participant to the same parameterSet for all sessions.rotate: The first time, assign parameterSet randomly (per parameterSetWeights if given); after that, each time the participant participates assign them to the next parameterSet in the list. Subtracts length of parameterSets until the ‘next’ index is in range.
The most recent session in which the
conditionsdata includes an element that looks like it was generated by this same randomizer (i.e., with key ending in-frameId, like-test-trials) will always be used for assignment.Only sessions with a completed consent frame are considered, so that participants are not rotated through conditions simply due to refreshing the setup page.
The “same” or “next” parameterSets are determined by the index of the previously-selected parameterSet. That is, if you were assigned to conditionNum 0 (index 0 in parameterSets) last time, you will be assigned to conditionNum 0 again this time if
conditionForAdditionalSessionsis"persist"and conditionNum 1 ifconditionForAdditionalSessionsis"rotate". So if you update the list ofparameterSetsin your study - e.g. to fix a bug or clarify wording - the new values will be used even for repeat participants. But be careful that you do not reorder them unless you intend to, say, swap all participants to the opposite condition on a specified date!If the previous index is now outside the range of the
parameterSetslist (e.g., you used to have 6 conditions, and the participant was previously in condition number 5, but then you changedparameterSetsto have only 3 elements) andconditionForAdditionalSessionsis"persist", then the participant is assigned to the last element ofparameterSets.
- parameterSetWeights [Array]
[Optional] Array of weights for parameter sets; elements correspond to elements of parameterSets. The probability of selecting an element
parameterSets[i]isparameterSetWeights[i]/sum(parameterSetWeights).If not provided, all
parameterSetsare weighted equally.This is intended to allow manual control of counterbalancing during data collection, e.g. to allow one condition to ‘catch up’ if it was randomly selected less often.
Instead of providing a single list of the same length as parameterSets, you may instead provide a list of objects specifying the weights to use within various age ranges, like this:
'parameterSetWeights': [ { 'minAge': 0, 'maxAge': 365, 'weights': [1, 0, 1] }, { 'minAge': 365, 'maxAge': 10000, 'weights': [0, 1, 0] }, ]
The child’s age in days will be computed, and the weights used will be based on the first element of
parameterSetWeightswhere the child falls between the min and max age. In the example above, children under one year old will be assigned to either the first or third condition; children over a year will be assigned to the second condition. This may be useful for researchers who need to balance condition assignment per age bracket. As you code data and realize you are set on 3-year-olds in condition A, for instance, you can stop assigning any more 3-year-olds to that condition.
Data collected
The information returned by this randomizer will be available in expData["conditions"]["THIS-RANDOMIZER-ID"]. The
randomizer ID will depend on its order in the study - for instance, 6-test-trials.
- conditionNum [Number]
the index of the parameterSet chosen
- parameterSet [Object]
the parameterSet chosen