random-parameter-set

Overview

Randomizer that provides flexible condition assignment and counterbalancing by allowing the user to specify an arbitrary sequence of frames to create. A set of parameters is randomly selected from a list of available parameterSets, and these parameters are substituted in to the parameters specified in the list of frames.

To use, define a frame with ‘kind’: ‘choice’ and ‘sampler’: ‘random-parameter-set’, as shown below, in addition to the parameters described under ‘properties’.

{
    ...
    "frames": {
        ...
        "test-trials": {
            "sampler": "random-parameter-set",
            "kind": "choice",
            ...
        }
    }
}

In addition, there are two special properties you need to define to use the random-parameter-set randomizer:

  • frameList: this is just what it sounds like: a list of all the frames that should be generated by this randomizer. Each frame is a JSON object just like you would use in the overall schema, with three differences:

    • Rather than giving each frame in the list its own name (e.g. “test_trial1”), the frameList automatically generates these names for each of the frames within it, so you can begin each frame with “kind”: “exp-lookit-text”, etc. rather than nesting it under another identifier (examples of this can be found below).

    • You can define default properties, to share across all of the frames generated by this randomizer, in the JSON object commonFrameProperties instead, as a convenience.

    • You can use placeholder strings for any of the properties in the frame; they will be replaced based on the values in the selected parameterSet.

  • parameterSets is a list of mappings from placeholder strings to actual values. When a participant starts your study, one of these sets will be randomly selected, and any parameter values in the frameList (including commonFrameProperties) that match any of the keys in this parameter set will be replaced.

Advanced options for choosing the parameterSet

You can determine the weights based on the child’s age, to maintain balanced conditions.) You can also keep kids in the same condition across all sessions they complete, or rotate them through conditions in order.

Examples

Let’s walk through an example of using this randomizer. Suppose we start with the following JSON document describing a study that includes instructions, an experimental manipulation asking participants to think about how delicious broccoli is, and an exit survey:

{
    "frames": {
       "instructions": {
           "id": "text-1",
           "blocks": [
               {
                   "text": "Some introductory text about this study."
               },
               {
                   "text": "Here's what's going to happen! You're going to think about how tasty broccoli is."
               }
           ],
           "showPreviousButton": false,
           "kind": "exp-lookit-text"
       },
       "manipulation": {
           "id": "text-2",
           "blocks": [
               {
                   "text": "Think about how delicious broccoli is."
               },
               {
                   "text": "It is so tasty!"
               }
           ],
           "showPreviousButton": true,
           "kind": "exp-lookit-text"
       },
       "exit-survey": {
            "debriefing": {
                "text": "Thank you for participating in this study! ",
                "title": "Thank you!"
            },
            "id": "exit-survey",
            "kind": "exp-lookit-exit-survey"
        }
    },
    "sequence": [
        "instructions",
        "manipulation",
        "exit-survey"
    ]
}

But what we really want to do is have some kids think about how tasty broccoli is, and others think about how yucky it is! We can use a random-parameter-set frame to replace both text frames:

{
    "frames": {
        "instruct-and-manip": {
            "sampler": "random-parameter-set",
            "kind": "choice",
            "id": "instruct-and-manip",
            "frameList": [
                {
                   "blocks": [
                       {
                           "text": "Some introductory text about this study."
                       },
                       {
                           "text": "INTROTEXT"
                       }
                   ],
                   "showPreviousButton": false
                },
                {
                   "blocks": [
                       {
                           "text": "MANIP-TEXT-1"
                       },
                       {
                           "text": "MANIP-TEXT-2"
                       }
                   ],
                   "showPreviousButton": true
               }
            ],
            "commonFrameProperties": {
                "kind": "exp-lookit-text"
            },
            "parameterSets": [
                {
                    "INTROTEXT": "Here's what's going to happen! You're going to think about how tasty broccoli is.",
                    "MANIP-TEXT-1": "Think about how delicious broccoli is.",
                    "MANIP-TEXT-2": "It is so tasty!"
                },
                {
                    "INTROTEXT": "Here's what's going to happen! You're going to think about how disgusting broccoli is.",
                    "MANIP-TEXT-1": "Think about how disgusting broccoli is.",
                    "MANIP-TEXT-2": "It is so yucky!"
                }
            ]
        },
       "exit-survey": {
            "debriefing": {
                "text": "Thank you for participating in this study! ",
                "title": "Thank you!"
            },
            "id": "exit-survey",
            "kind": "exp-lookit-exit-survey"
        }
    },
    "sequence": [
        "instruct-and-manip",
        "exit-survey"
    ]
}

Notice that since both of the frames in the frameList were of the same kind, we could define the kind in commonFrameProperties. We no longer define id values for the frames, as they will be automatically identified as instruct-and-manip-1 and instruct-and-manip-2.

When the “instruct-and-manip” randomizer is evaluated, the Lookit experiment player will start with the frameList and add the key-value pairs in commonFrameProperties to each frame (not overwriting existing pairs):

[
    {
        "kind": "exp-lookit-text",
        "blocks": [
            {
                "text": "Some introductory text about this study."
            },
            {
                "text": "INTROTEXT"
            }
        ],
        "showPreviousButton": false
    },
    {
        "kind": "exp-lookit-text",
        "blocks": [
            {
                "text": "MANIP-TEXT-1"
            },
            {
                "text": "MANIP-TEXT-2"
            }
        ],
        "showPreviousButton": true
    }
]

Next, one of the two objects in parameterSets is selected randomly. (By default, parameter sets are weighted equally, but parameterSetWeights can be provided as an optional key in the random-parameter-set frame. If provided, parameterSetWeights should be an array of relative weights for the parameter sets, corresponding to the order they are listed. For instance, if we wanted 75% of participants to think about how tasty broccoli is, we could set parameterSetWeights to [3, 1]. This allows uneven condition assignment where needed to optimize power, as well as allowing researchers to stop testing conditions that already have enough participants as data collection proceeds.

Suppose that in this case the second parameter set is selected:

{
"INTROTEXT": "Here's what's going to happen! You're going to think about how disgusting broccoli is.",
"MANIP-TEXT-1": "Think about how disgusting broccoli is.",
"MANIP-TEXT-2": "It is so yucky!"
}

Now we return to the list of frames, and wherever any value matches one of the keys in the parameterSet (even if that value is nested in another object), it is replaced by the corresponding value from the parameterSet, yielding the following final list of frames:

[
    {
        "kind": "exp-lookit-text",
        "blocks": [
            {
                "text": "Some introductory text about this study."
            },
            {
                "text": "Here's what's going to happen! You're going to think about how disgusting broccoli is."
            }
        ],
        "showPreviousButton": false
    },
    {
        "kind": "exp-lookit-text",
        "blocks": [
            {
                "text": "Think about how disgusting broccoli is."
            },
            {
                "text": "It is so yucky!"
            }
        ],
        "showPreviousButton": true
    }
]

Here is another example of how to use the “random-parameter-set” sampler in the context of different frame types (here, “exp-lookit-images-audio” and “exp-lookit-video”. In this case, the sampler will randomly pick one of the two parameterSets, displaying either 2 images of Zenna and then the bowl video, or 2 images of Remy and then the cup video. Note again how each frame does not have its own title within the frameList, and just begins with the definition of its kind.

{
   "frames": {
       "test-trials": {
           "sampler": "random-parameter-set",
           "kind": "choice",
           "frameList": [
               {
                   "kind": "exp-lookit-images-audio",
                   "images": [
                       {
                           "id": "happy",
                           "src": "FIRST_IMAGE_PLACEHOLDER",
                           "position": "left"
                       },
                       {
                           "id": "sad",
                           "src": "SECOND_IMAGE_PLACEHOLDER",
                           "position": "right"
                       }
                   ]
               },
               {
                   "kind": "exp-lookit-video",
                   "video": {
                       "top": 10,
                       "left": 25,
                       "loop": false,
                       "width": 50,
                       "source": "VIDEO_PLACEHOLDER"
                   },
                   "autoProceed": true,
                   "doRecording": false,
                   "videoTypes": [
                       "mp4"
                   ]
               }
           ],
           "commonFrameProperties": {
               "baseDir": "https://www.mit.edu/~kimscott/placeholderstimuli/"
           },
           "parameterSets": [
               {
                   "FIRST_IMAGE_PLACEHOLDER": "happy_zenna.jpg",
                   "SECOND_IMAGE_PLACEHOLDER": "sad_zenna.jpg",
                   "VIDEO_PLACEHOLDER": "cropped_bowl"
               },
               {
                   "FIRST_IMAGE_PLACEHOLDER": "happy_remy.jpg",
                   "SECOND_IMAGE_PLACEHOLDER": "sad_remy.jpg",
                   "VIDEO_PLACEHOLDER": "cropped_cup"
               }
           ]
       }
   },
   "sequence": [
       "test-trials"
   ]
}

Parameters

commonFrameProperties [Object]

Object describing common parameters to use in EVERY frame created by this randomizer. Parameter names and values are as described in the documentation for the frameType used.

frameList [Array]

List of frames to be created by this randomizer. Each frame is an object with any necessary frame-specific properties specified. The kind of frame can be specified either here (per frame) or in commonFrameProperties. If a property is defined for a given frame both in this frame list and in commonFrameProperties, the value in the frame list will take precedence.

(E.g., you could include 'kind': 'normal-frame' in commmonFrameProperties, but for a single frame in frameList, include 'kind': 'special-frame'.)

Any property values within any of the frames in this list which match a property name in the selected parameterSet will be replaced by the corresponding parameterSet value. For example, suppose a frame in frameList is

{
    'leftImage': 'LEFTIMAGE1',
    'rightImage': 'frog.jpg',
    'size': 'IMAGESIZE'
}

and the row that has been selected randomly of parameterSets is

{
    'LEFTIMAGE1': 'toad.jpg',
    'LEFTIMAGE2': 'dog.jpg',
    'IMAGESIZE': 250
}

Then the frame would be transformed into:

{
    'leftImage': 'toad.jpg',
     'rightImage': 'frog.jpg',
     'size': 250
}

The same values may be applied across multiple frames. For instance, suppose frameList is

[
    {
        'leftImage': 'LEFTIMAGE1',
        'rightImage': 'frog.jpg',
        'size': 'IMAGESIZE'
    },
    {
        'leftImage': 'LEFTIMAGE2',
        'rightImage': 'frog.jpg',
        'size': 'IMAGESIZE'
    }
]

Then the corresponding processed frames would include the values

[
    {
        'leftImage': 'toad.jpg',
        'rightImage': 'frog.jpg',
        'size': 250
    },
    {
        'leftImage': 'dog.jpg',
        'rightImage': 'frog.jpg',
        'size': 250
    }
]

A property value like IMAGESIZE may be placed in a frame definition nested within another object (at any depth) or within a list and will still be replaced.

You can also use selectors to randomly sample from or permute a list given in a parameterSet. Suppose LISTVAR is defined in a parameterSet as THELIST, e.g. a list of potential stimuli. Within frames in your frameList (and in commonFrameProperties), you can use any of the following:

  • Select the Nth element (0-indexed) of THELIST: (Will cause error if N >= THELIST.length)

    'parameterName': 'LISTVAR#N'
    
  • Select (uniformly) a random element of THELIST:

    'parameterName': 'LISTVAR#RAND'
    
  • Set parameterName to a random permutation of THELIST:

    'parameterName': 'LISTVAR#PERM'
    
  • Select the next element in a random permutation of THELIST, which is used across all substitutions in this randomizer. This allows you, for instance, to provide a list of possible images in your parameterSet, and use a different one each frame with the subset/order randomized per participant. If more LISTVAR_UNIQ parameters than elements of THELIST are used, we loop back around to the start of the permutation generated for this randomizer.

    'parameterName': 'LISTVAR#UNIQ'
    
parameterSets [Array]

Array of parameter sets to randomly select from in order to determine the parameters for each frame in this session.

A single element of parameterSets will be applied to a given session.

conditionForAdditionalSessions [String | 'random']

[Optional] How to select a parameterSet for a participant who has previously participated in this study. Must be one of 'random' (default), 'persist', or 'rotate'. Meanings:

  • random: regardless of any previous sessions from this participant, select a parameterSet for this participant as usual (including using parameterSetWeights if provided). Default behavior.

  • persist: Continue assigning the same participant to the same parameterSet for all sessions.

  • rotate: The first time, assign parameterSet randomly (per parameterSetWeights if given); after that, each time the participant participates assign them to the next parameterSet in the list. Subtracts length of parameterSets until the ‘next’ index is in range.

The most recent session in which the conditions data includes an element that looks like it was generated by this same randomizer (i.e., with key ending in -frameId, like -test-trials) will always be used for assignment.

Only sessions with a completed consent frame are considered, so that participants are not rotated through conditions simply due to refreshing the setup page.

The “same” or “next” parameterSets are determined by the index of the previously-selected parameterSet. That is, if you were assigned to conditionNum 0 (index 0 in parameterSets) last time, you will be assigned to conditionNum 0 again this time if conditionForAdditionalSessions is "persist" and conditionNum 1 if conditionForAdditionalSessions is "rotate". So if you update the list of parameterSets in your study - e.g. to fix a bug or clarify wording - the new values will be used even for repeat participants. But be careful that you do not reorder them unless you intend to, say, swap all participants to the opposite condition on a specified date!

If the previous index is now outside the range of the parameterSets list (e.g., you used to have 6 conditions, and the participant was previously in condition number 5, but then you changed parameterSets to have only 3 elements) and conditionForAdditionalSessions is "persist", then the participant is assigned to the last element of parameterSets.

parameterSetWeights [Array]

[Optional] Array of weights for parameter sets; elements correspond to elements of parameterSets. The probability of selecting an element parameterSets[i] is parameterSetWeights[i]/sum(parameterSetWeights).

If not provided, all parameterSets are weighted equally.

This is intended to allow manual control of counterbalancing during data collection, e.g. to allow one condition to ‘catch up’ if it was randomly selected less often.

Instead of providing a single list of the same length as parameterSets, you may instead provide a list of objects specifying the weights to use within various age ranges, like this:

'parameterSetWeights': [
    {
        'minAge': 0,
        'maxAge': 365,
        'weights': [1, 0, 1]
    },
    {
        'minAge': 365,
        'maxAge': 10000,
        'weights': [0, 1, 0]
    },
]

The child’s age in days will be computed, and the weights used will be based on the first element of parameterSetWeights where the child falls between the min and max age. In the example above, children under one year old will be assigned to either the first or third condition; children over a year will be assigned to the second condition. This may be useful for researchers who need to balance condition assignment per age bracket. As you code data and realize you are set on 3-year-olds in condition A, for instance, you can stop assigning any more 3-year-olds to that condition.

Data collected

The information returned by this randomizer will be available in expData["conditions"]["THIS-RANDOMIZER-ID"]. The randomizer ID will depend on its order in the study - for instance, 6-test-trials.

conditionNum [Number]

the index of the parameterSet chosen

parameterSet [Object]

the parameterSet chosen