Terracotta—one of the Digital Learning Platforms (DLPs) within SEERNet—is a tool designed to facilitate experimental education research within the Canvas learning management system (LMS). It allows educators and researchers to test how different instructional content, contexts, and learning activities impact student outcomes—all within their existing LMS workflows.
Aligned with SEERNet’s goal to advance education research by developing and enhancing DLPs-as-research-infrastructure, Terracotta continues to innovate and expand the capacity to enable educators and researchers to conduct high-quality experiments in K-12 and higher education spaces. The latest update now seamlessly integrates web-based applications, such as Qualtrics surveys, jsPsych experiments, and custom interactive web activities. These new integrations allow researchers to:
These integrations allow researchers to deploy a wider variety of experimental treatments and data collection activities within Terracotta, where students complete them as graded LMS assignments.
“We’re thrilled to introduce a transformative update to Terracotta that significantly enhances our ability to conduct research in authentic learning environments. This latest release marks a major milestone in our mission to break down barriers to rigorous and responsible education research” – Ben Motz, Assistant Professor of Psychology and Brain Sciences, Indiana University
Educators and researchers are already leveraging these new features for innovative studies. Drs. Akira Miyake, Michael Kane, Hannah Snyder and Matthew Bernacki, one of the SEERNet research teams recently funded by the Institute of Education Sciences (IES), will use Terracotta to test the efficacy of interventions designed to reduce academic procrastination. Although it is often condoned, academic procrastination is associated with various negative outcomes, such as worse grades, higher drop-out rates, higher academic dishonesty, and worse mental health and well-being. For this study, Terracotta will be used to randomly assign students from large lecture-based undergraduate psychology, biology, and chemistry classes to receive the interventions or to the control group within their Canvas course sites, allowing for a sophisticated 2×2 factorial design. Researchers will also use Terracotta’s embedded informed consent process, which shows up as an assignment in Canvas, to make sure that only students who volunteer to participate will be included in the study.
The outcomes of interest include the completion timing of, and performance on, course assignments visible in Canvas gradebooks. Terracotta’s new Qualtrics Integration will allow researchers to survey students about additional outcomes, such as reduction in stress and improvement in students’ psychological well-being after exposure to the interventions. Researchers can also use the survey integration to collect more nuanced explanatory information, such as why assignments or quizzes were turned in when they were.
Importantly, and connected to the SEER principles, the Qualtrics Integration will allow researchers to collect self-reported individual learner characteristics that will address “for whom” the interventions work. Specifically, the survey integration will allow researchers to collect data (from prior to random assignment) on student experiences and circumstances that might moderate outcomes, such as whether they have full or part-time jobs in addition to their school work, whether they have a full course load, or whether they have other personal/family obligations. The surveys will also ask about students’ beliefs, goals, and behaviors, including study habits or other skills they use to complete their coursework. Terracotta Integrations thus allow researchers to collect more precise information about students’ background and motivation that will more richly describe the student experience. Through the surveys, researchers can also collect information about implementation and use of the techniques being taught in the intervention.
These outcome data and student survey data can be further paired with data on learners’ procrastination behaviors, which can be extracted from the Canvas Live Events log, using a tool called CLICKSTREAM. The team will engineer features that describe the shape of learners’ procrastination behaviors and examine how the frequency, timing, and extent of procrastination is influenced by different intervention approaches that target stress-related and behavior-related drivers of procrastination. These robust datasets will enable the researchers to dig into the “black box” of the process to more fully understand the mechanisms that connect the interventions and the outcomes.
Another thing to note is the scale of this work: the research team is working across 7 universities. Terracotta will support the random assignment, the consistent administration of experimental manipulations, and rich, continuous data collection at scale and across contexts.
Terracotta Integrations supports these expanded capabilities while maintaining student privacy and data security. The new integration system ensures that external tools, like Qualtrics, never access student identities. Additionally, the Terracotta team has implemented a secure, single-use token system to preserve the integrity of individual student submissions.
Including survey and other implementation data in educational experiments is nothing new. But working within common, familiar tools that enable:
all within an authentic learning environment in which students are actively engaging—that is innovation to be excited about!
“We’re all excited to conduct this large-scale academic procrastination intervention project using Terracotta. The field currently lacks well-validated interventions for reducing academic procrastination that can be readily implemented in authentic classroom settings—something that interested instructors can easily administer without extensive changes to their instructions. Part of the challenge is the difficulty of conducting a rigorous intervention study in classroom settings, with large sample sizes and sufficient statistical power. Terracotta helps us overcome this challenge and conduct a rigorous randomized control study across multiple classrooms and multiple universities.” – Akira Miyake, Professor of Psychology, University of Colorado Boulder.
The Terracotta team—along with the broader SEERNet network—is committed to expanding the field of who conducts educational research. The team has worked to broaden these opportunities by strategically integrating widely used web-based applications. Additionally, they have worked to build and facilitate processes that allow educators, who may or may not be formally trained education researchers/psychologists, to drive these studies in their own courses. To support this, the Terracotta team has developed accessible step-by-step documentation, integration guides, and accompanying videos to help educators integrate these new tools into their research.
Curious to see how it works? Check out this quick screen share demo showcasing how to add a Qualtrics survey as a treatment in a Terracotta experiment. And visit the Terracotta Knowledge Base to review the following detailed guides:
The Terracotta team hopes these expanded capabilities inspire new and creative research approaches to student learning. Whether you’re interested in sophisticated survey designs, cognitive psychology experiments, or interactive educational tools, Terracotta now offers even greater flexibility.
This project is supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305N210034 to Digital Promise. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.