CHS/Lookit
  • CHS
  • Lookit EFP
  • jsPsych

Overview

  • Getting started guide
  • Features
  • About
  • Progress updates
  • FAQ
  • Resources
  • Publications

Using CHS

  • Registration and login
  • Labs on CHS
  • Managing studies
  • Study permissions
  • Setting study details
  • Setting the study design

Lookit studies

  • Lookit protocol specification
  • Updating the Lookit experiment runner
  • Preparing stimuli
  • Style guide
  • Timing accuracy in Lookit studies
  • Releases

jsPsych studies

  • jsPsych studies on CHS
  • Limitations

Reviewing your study

  • Study approval process
  • Basic review requirements
  • Self review checklist
  • Peer review checklist

Collecting data

  • Coding consent
  • Downloading data
  • Day-to-day study operation
  • Preventing spam
  • Participant recruitment
  • Using the API

Tutorial

  • 1. Accessing CHS site and community
  • 2. Contributing to the codebase
  • 3. Create a study
  • 4. Managing study data
  • 5. Finishing steps

Community and procedures

  • Community involvement
  • Working Groups
  • IRB and legal info
    • Overview
    • Access agreement
      • What it is
      • How to get it signed
      • Correspondence between access agreements and CHS Labs
      • If your institution requires changes
      • Sending it to CHS
    • Ethics approval for CHS studies
      • How consent and assent work
    • Responsibilities of researchers
    • Privacy policy
    • Sub-processors and information about GDPR compliance/DPAs

Contributing

  • Editing this documentation
  • Using Github issues
  • Overview of CHS architecture
  • Developing new frames
  • Contributor Guidelines
  • Installation: lookit-api (Django project)
  • Installation: ember-lookit-frameplayer (Ember app)
  • Django app implementation notes
CHS/Lookit
  • IRB and legal info
  • Edit on GitHub

IRB and legal info

Overview

Before you can run a study on CHS/Lookit, there are three things you’ll need to do:

  1. Sign and return the Lookit access agreement (details below). This covers your entire lab. You only have to do this once.

  2. Get IRB approval at your own institution (details below).

  3. Get peer feedback on the study, and then have it approved by CHS.

Access agreement

What it is

In order to use CHS/Lookit for data collection (including posting studies that run on other platforms/video chat studies), the principal investigator (PI) of a research group must first sign an institutional agreement with MIT that lays out each institution’s rights and responsibilities. This may require review by a legal or contracts office or an office of sponsored programs depending on the institution.

The access agreement with a PI at a given institution covers all studies conducted on CHS where the approved IRB protocol at that institution names that person as the PI. (If that institution cedes to another institution for approval, that’s still okay.)

The agreement itself is short and closely based on Databrary’s agreement (with permission); the meat is in the Terms of Use you are agreeing to. Researchers should actually read the terms carefully: these are real content written by a real person and contains non-obvious information about ethical/privacy issues you may not have encountered in the lab. Note that we do impose standards that may in some cases be more stringent than your own IRB’s.

To make sure we’re all on the same page, we require that at least one researcher associated with each research group using CHS complete this quiz about the Terms of Use.

How to get it signed

If you are planning a CHS study, getting the access agreement signed should be your first step because it takes a highly variable amount of time (record so far is under two weeks, but it can take a few months). The access agreement doesn’t require any customization on your part - you just need to send it off to someone at your institution who’s authorized to sign on behalf of the institution.

First, check whether your institution already has an access agreement set up. You can check the list here or email childrenhelpingscience@gmail.com. If so, you’ll be able to just add yourself (newer form of agreement) or get in touch with the person who signed off previously (older form of agreement). If there were any modifications to the standard agreement made for your institution, you’ll be able to just use the modified form instead of having to re-negotiate the same changes.

If there isn’t an existing agreement, you’ll need to figure out which office to send it to for approval. Unfortunately the appropriate office varies by institution. If you’ve previously set up to use Databrary, try contacting the office that signed off on that. As examples, the following offices have approved the access agreement: Contracts, VP Finance, Partnerships, University/Industry Liason, Department head, Provost, Sponsored programs.

In general, you do NOT need to have an IRB protocol to use CHS before the access agreement can be signed. In some cases, though, the office that signs the agreement may ask that you obtain ethical approval first.

The one exception to this process is that researchers at MIT, completing studies approved by MIT’s IRB, do not require the signature of an MIT authorized signer. However, the MIT PI still needs to sign and return the agreement.

Correspondence between access agreements and CHS Labs

In general, each Authorized Investigator on an access agreement should correspond to exactly one Lab on CHS, which makes it far more straightforward to ensure that everyone’s actually covered.

Each Lab must be associated with at least one agreement. (E.g., suppose Jill is a PI at Science University and has signed an access agreement, and Joe is another PI at Science University who has not. We cannot approve Joe’s Lab on CHS to test - even if he tells us that his work will always actually be covered under Jill’s IRB protocol.)

However, if multiple PIs at your institution are closely affiliated (e.g. a lab jointly headed by two professors, or students all serve as PIs on IRB protocols in your department) please go ahead and create a lab listing multiple PIs and just let us know. We’d need an access agreement for each PI, and would expect all studies conducted by this lab to be covered by an IRB protocol listing at least one of those PIs.

If your institution requires changes

We have generally been able to accommodate requests for minor changes to the agreement as required by universities’ legal departments (e.g. when they are legally unable to agree to MA governing law, or to allow UK ethics committee approval in addition to “US-equivalent” IRBs).

If it looks like some clause would be a deal-breaker for your institution, please get in touch with CHS staff (childrenhelpingscience@gmail.com) and general counsel lawyer Jason Baletsa (jbaletsa at mit.edu) for advice.

Please note that approval of changes happens on the MIT General Counsel office’s timeline, which is not under CHS’s control.

Sending it to CHS

Once you have the user agreement signed by the PI and institution’s authorized signer, please:

  • Make sure you or someone in your group has completed this quick quiz about the Terms of Use

  • Send the PDF of the signed agreement to childrenhelpingscience@gmail.com, noting that you have also completed the quiz

Ethics approval for CHS studies

A researcher using CHS applies to their own institution’s IRB for approval. Except for our involvement in a small set of initial studies, MIT is not considered to be engaged in human subjects research by running CHS.

You can find resources for writing your IRB protocol here. You will likely need to give a brief overview of how data collection on CHS works and what protections are included for participant data.

CHS may request a copy of the approved protocol at the time a study is submitted to run on CHS. (In general we don’t - we aren’t looking to get into that much paperwork. This is mostly if we have any concerns or you’re getting permission to do something unusual.)

How consent and assent work

The consent process on CHS is fairly standardized by design, to reduce the mental load on families. Parents consent by recording a verbal statement of consent after reviewing a document.

Information about what the consent page looks like to parents, and how to customize it, is available in the Lookit EFP documentation here and in the CHS jsPsych documentation here. You can see the available templates for the consent document here. Essentially you will supply pieces of text that will be substituted into the template.

If you are testing older children and need to collect assent in addition, you will use the Lookit assent page, which is less standardized at this point.

When applying for IRB approval, you may need to present the exact text of the consent document that will be presented, which you can do by substituting values into the templates linked above. You can even start setting up your study (copy one of the tutorial/example studies and customize the consent text) and preview it to get screenshots or the PDF download.

If you anticipate running follow-up studies (which will have at least different titles and durations), see if you can show your IRB the template and note that certain pieces may vary, giving them examples. They may be ok with you saying, for instance, that it’s going to look exactly like this except that (a) your first study is called “Do Babies Love Cats?” but other studies may be called “Do Babies Love Dogs?”, “Do Babies Love Rabbits?”, etc. and (b) your first study is 15 minutes long so that’s what the consent form says, but ohter studies may be 5 - 25 minutes long, and their durations will be accurately listed in the consent form. Keeping the procedures and purpose information relatively general to accommodate a broad protocol is ok if you need to!

Please keep the text you insert as easy to read as you possibly can. Do not include boilerplate text that’s in your five-page in-person consent just because it’s there and you’re not sure if your IRB will let you remove it - ask them! Do not include information that is covered elsewhere in the template just because you usually word it slightly differently; that’s confusing.

Researchers must use these standard consent pages, barring extraordinary legal barriers at their institutions (we have not encountered anyone not able to approve a protocol). You may NOT use your existing consent form instead or in addition just because you don’t want to submit an amendment to your protocol (sorry).

You will likely need to apply for a waiver of the usual requirement to collect written consent from participants so that you can rely on videorecorded statements. (The statements parents make during Lookit studies are better evidence of true informed consent than commonly-approved checkboxes for online studies, and so far this has not been an issue.) You can see an example here.

Responsibilities of researchers

  • Everyone:

    • Protect your CHS account credentials. You are responsible for any access to participant data on CHS via your account. You should choose a strong password, change it regularly, and never share it. (Share access by adding another researcher to your Lab - not by sharing a lab account password.)

    • Promptly report any breach or potential breach of participant information that you become aware of, both to CHS and to your local institution.

  • PIs:

    • You are responsible for the actions of anyone who’s conducting research on CHS under your access agreement. Ensure that anyone added to your Lab on CHS understands any rules in the Terms of Use that apply to what they’ll be doing. (E.g., make sure an RA tasked with approving consent and contacting families with compensation understands when it’s ok to withhold compensation, when to approve/reject consent, etc. Make sure someone helping with analysis understands which fields must be omitted from published data.) You may choose to create your own training materials based on the tasks they’ll actually be doing - please share these if so!

    • Ensure that people who are no longer in your lab are removed from your Lab promptly.

  • Study admins:

    • Ensure you have current IRB approval for your study before beginning ANY data collection on CHS. Promptly pause the study if approval lapses.

    • Ensure that anyone with access to participant data for this study is listed on the appropriate IRB protocol. Promptly remove permissions for anyone who is removed from the IRB.

    • Assign minimal appropriate roles for each person who needs access to your study.

Privacy policy

Both researchers and participants are covered by the privacy policy.

Sub-processors and information about GDPR compliance/DPAs

  • AWS S3 (data storage - participant video): https://aws.amazon.com/service-terms/

  • GCP (data storage, databases; temporary data storage, video .zip downloads; platform and study hosting/deployment): https://cloud.google.com/security/gdpr/resource-center/contracts-and-terms

  • Sentry (error reporting; no identifiable information): https://sentry.io/security/#hipaa-and-hitech

Only applies to studies using an older Lookit experiment runner (versions released before Jan 30 2024):

  • Pipe (video streaming; no copies stored): https://addpipe.com/gdpr

Previous Next

© Copyright 2017, Center for Open Science and MIT.

Built with Sphinx using a theme provided by Read the Docs.