Study approval process

Remember, before your study can go live on Lookit, there are three things you’ll need to do:

  1. Sign and return the access agreement. This covers your entire lab. You only have to do this once.
  2. Get IRB approval at your own institution.
  3. Get peer feedback on the study, and then submit it to Lookit for approval. This is the part we’ll discuss here!

Why is there a study review process?

Because Lookit is a shared platform with a shared reputation, it benefits all of us to uphold high standards for all Lookit studies. Families who have a good experience - the study is fun, it runs smoothly, the instructions are easy to follow, they get a gift card promptly or get quick friendly answers to their questions, their child isn’t upset by the stimuli - will come back. Positive media coverage likewise benefits all of us.

To make sure all studies meet community standards, we require two forms of review before studies are posted: at least one other researcher outside your lab will provide peer feedback, and Lookit staff will review as well. While this may feel like an “extra step” compared with in-lab research, it’s an opportunity to improve both the participant experience and the science - saving you time later. So far, researchers who were frustrated by the extra steps have been convinced once they see how much their studies improve in the process.

If you don’t have IRB approval or an access agreement yet

You can still go ahead with steps 1 and 2 below so you’re ready to go as soon as possible! The “Lab” your study is associated with on Lookit will have to be approved to test before you’ll be able to submit in Step 3, which means you’ll need the access agreement set at that point.

Step 1: study preparation and self-review

First, get your study working smoothly, the way you want it to, and make any revisions from within-lab feedback. Please work through the self-review checklist before requesting peer review!

Step 2: peer review

Next, post a preview link for your study on the Lookit Slack #researchers channel to gather some feedback from researchers outside your group. This is like getting a bunch of fresh eyes at a lab meeting made up of researchers who work online!

Peer reviewers can reference the peer review checklist and will be looking for things like:

  • Typos
  • Overall flow of the study - are there any abrupt transitions? do they feel like they know what to expect?
  • Places where your instructions are confusing, overwhelming, condescending, etc.
  • Cases where your stimuli or prompts might be unintentionally confusing, upsetting, or unnecessarily non-inclusive (e.g., you’ve assumed that each child has a mother and father at home, that a child who’s experienced the birth of a sibling still has that sibling, etc.; you’ve prompted children to describe something bad that happened to them assuming they’re going to think about stuff like their popsicle falling)
  • Whether your study description accurately reflects their experience
  • Experimental design issues that happen to occur to them (this isn’t primarily a scientific review)

With this feedback you can revise and improve your study, often the instructions and parent-facing text in particular.

You should also expect to return the favor and review other labs’ studies, as part of participating in the platform.

Step 3: Lookit submission and approval

When your study is ready and you’ve responded to the peer feedback, you can submit your study for Lookit approval so it can go live. This is the point where you’ll need to have your access agreement set up.

When you submit the study, you will be prompted to note any non-standard elements that require specific approval per the Terms of Use (e.g., integration of additional about participants from another source) as well as what changes you made based on peer feedback.

At this point, Lookit staff will review your study, focusing primarily on the participant experience. This is also when any custom code you’re using will be reviewed for security or functionality issues.

Initial internal review can generally be completed within a week. Revisions may be requested before the study can be approved to run. To minimize the number of rounds of review needed, researchers are strongly encouraged to polish their studies as much as possible before submitting - please don’t use us as a proofreading service!