OpenStax Kinetic (currently in its Alpha version) enables research on a wide spectrum of learner characteristics (including behavior, performance, and psychosocial constructs) and their impact on learning and retention for postsecondary adult learners. Researchers can design their studies using pre- and post-intervention assessments, collection of self-report or assessment measures from learners, and/or longitudinal analyses over multiple time scales. Researchers can use data from their own studies, as well as from the Kinetic “library” of learner characteristics (https://bit.ly/kinetic-learner-characteristics-library). The Alpha version will facilitate researchers to deploy learning studies that can be designed and delivered via Qualtrics, while subsequent Kinetic versions will allow linking of these researcher-administered measures and studies to existing learner usage data of OpenStax materials and other external learner data.
Post-secondary adult formal and informal learners in the US; majority using OpenStax textbooks
Kinetic encourages researchers to pre-register their studies on the Open Science Framework (https://osf.io/), and subsequent versions on Kinetic will seek to integrate preregistration within the Kinetic researcher experience.
Kinetic is in the process of convening a Diversity, Equity, and Inclusion (DEI) as well as a Fairness, Ethics, Accountability, and Transparency (FEAT) advisory panel to build processes and protocols for vetting research studies. Additionally, the OpenStax research team is working to create processes and automated tools for sustainably vetting Kinetic studies.
Currently, the Kinetic IRB is an exempt umbrella protocol where the Rice University IRB serves as the institutional IRB of record for all Kinetic studies. We are however investigating ways of effectively integrating the IRB process with the Kinetic researcher experience. The Rice IRB guidelines are also included as part of the Kinetic Terms of Use to ensure that research conducted on Kinetic is aligned with our exempt protocol and standard instructional practice. It’s important to reference that any study plans that require greater scrutiny will need to submit an IRB protocol for review at their home institution with a subsequent approval from the Rice IRB.
The Kinetic team is using a multi-pronged approach to recruit learners and amplify participation on Kinetic. The following are some of the highlighted recruitment initiatives currently:
On Kinetic Alpha, researchers can conduct rapid-cycle randomized control trials (RCTs) where the research question of interest can be answered with immediate measures of outcomes across a large sample of learners who are randomly (or pseudo-randomly) assigned to control and treatment conditions (e.g., the impact of concept mapping vs. passive re-reading on retention of material); A/B/N testing or value-added research where two (or more) conditions are contrasted with one condition (treatment) being different from the other (control) in one respect or having one enhancement. Kinetic Alpha also enables longitudinal research by inviting learners who participated in a multi-part Kinetic study to return for later sessions, replication studies to validate whether findings obtained in more controlled conditions persist in an authentic learning environment; and efficacy studies to assess the impact of learning interventions in ecologically valid environments.
Once Kinetic is integrated into the entire OpenStax ecosystem, researchers will be able to conduct more complex study designs, including cross-domain research to assess constructs such as learning transfer (e.g., whether learners utilize a specific intervention across subject domains, and how that affects immediate and delayed learning outcomes); incorporate learner interaction with different OpenStax systems; investigate custom plug-in learning tools; and finally, make the Kinetic infrastructure ready for investigating the impact of the use of different AI tools in learning and education.
Kinetic Alpha accommodates studies that can be administered in Qualtrics. Try a sample Kinetic study at https://openstax.org/kinetic and/or learn more about our work https://openstax.org/research.
Over subsequent years, we will expand capabilities to support researchers who build their own custom tasks outside of Qualtrics to plug-in to the Kinetic system. Finally, we are in the process of evaluating what capabilities Kinetic needs to support AI in education research.
Kinetic provides a means for researchers to better understand their learners, by virtue of our extensive learner characteristics library available to researchers to use as part of their work. The learner characteristics library to includes a broad range of adult learner characteristics (including demographics, psychosocial constructs such as goal orientation and vocational interest); learner engagement and participation on the platform (e.g., studies completed, time spent, session activities); and learner performance (on research tasks that we deploy on the platform). In later versions of Kinetic, researchers will have the capability to merge learner use of and engagement with online textbooks and other OpenStax offerings with their own data sets. We currently have over 25 measures of learner characteristics in our individual differences library, including personality (Big 5), vocational interest (RIASEC), self-efficacy, goal orientation, reading comprehension, and resilience.
The Kinetic Alpha platform accommodates measures that can be administered in Qualtrics. Eventually, OpenStax will have data sharing agreements with institutional partners to acquire long-term outcomes (e.g., GPA, course grades) and is working on a pipeline to securely and easily collect the data from partnering institutions.
Kinetic Alpha for researchers will feature a minimal viable version (MVP) of secure data enclaves that will permit researchers to analyze learner records without ever seeing individual data points. Additionally, Kinetic will build capabilities to support setting inclusion and exclusion criteria for research tasks. Overall, this approach will significantly reduce privacy risks while also making the breadth of identified data available to support robust learning science research. As part of the data enclave interface, Kinetic will also provide code templates in R (followed by Python) for researchers to modify for their own projects.
This project is supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305N210034 to Digital Promise. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.