Kinetic will enable research on a wide spectrum of postsecondary learner outcomes related to behavior, performance, and psychosocial constructs. Depending upon the version of Kinetic, its capabilities, and the exact outcomes of interest, researchers might design their studies using pre- and post-intervention assessments, collection of self-report or external data from students, and/or longitudinal analyses over multiple time scales using student records in OpenStax products. The alpha version will allow researchers to administer any measure that can be delivered via Qualtrics, while the beta version will allow linking of these researcher-administered measures to existing learner outcomes in OpenStax materials.
Post-secondary adult formal and informal learners in the US; majority using OpenStax textbooks
Kinetic will encourage researchers to pre-register their studies on OSF.io and include this information on the platform. Kinetic is developing processes to vet potential studies with an internal reviewer supported by automated tools.
We are currently in conversation with Rice IRB to determine the IRB process on Kinetic. As of now, researchers will submit their protocol to their home institution and the home institution IRB will connect with Rice IRB in the back end. This process will be iteratively refined to be smoother and easier to scale.
In order to supplement recruitment of student participants from existing OpenStax products to Kinetic, they have partnered with 21 postsecondary institutions that represent 40 campuses, 78% of which are minority-serving institutions. In total, the collaborating institutions reach over 417,000 postsecondary students, of which 33% are eligible for Pell grants. These partnerships will enable Kinetic to implement institution-wide recruitment of students. In contrast to individual student recruitment directly from OpenStax products, institution-wide recruitment will minimize the possibility of self-selection bias.
In Alpha, researchers will be able to conduct rapid-cycle randomized control trials (RCTs) where the research question of interest can be answered with immediate measures of outcomes across a large sample of students who are randomly (or pseudo-randomly) assigned to control and treatment conditions (e.g., the impact of concept mapping vs. passive re-reading on retention of material); A/B/N testing or value-added research where two (or more) conditions are contrasted with one condition (treatment) being different from the other (control) in one respect or having one enhancement.
In Beta, thanks to its integration into the entire OpenStax ecosystem, researchers will be able to additionally conduct longitudinal research that tracks student learning behavior and related outcomes over a specific timescale by inviting students who participated in a Kinetic study to return for later sessions. Potential study designs include cross-domain research to assess constructs such as learning transfer (e.g., whether students utilize a specific intervention across subject domains, and how that affects immediate and delayed learning outcomes); replication studies to validate whether findings obtained in laboratory conditions persist; and efficacy studies to assess the impact of learning interventions in ecologically valid environments.
The initial Kinetic platform will accommodate measures that can be administered in Qualtrics. See examples of interventions here.
Kinetic, by virtue of working with higher ed students, will have the ability to collect: learner characteristics (including demographics, psychosocial constructs such as goal orientation, vocational interest); learner engagement and participation on the platform (e.g., consent, studies completed, time spent, session activities); learner performance (on research tasks that we deploy on the platform); and in later versions of Kinetic, the capability for researchers to merge student use of and engagement with OpenStax textbooks. We currently have 21 measures of learner characteristics in our individual differences library including personality (Big 5), vocational interest (RIASEC), self-efficacy, goal orientation, and resilience.
The initial Kinetic platform will accommodate measures that can be administered in Qualtrics. Eventually, OpenStax will have data sharing agreements with institutional partners to acquire prior achievement information (e.g., GPA) and is working on a pipeline to securely and easily get the data from the institutions.
Kinetic will utilize a secure data enclave that will permit researchers to not only analyze student records but also to set inclusion and exclusion criteria for their research tasks. Overall, this approach will significantly reduce privacy risks while also making the breadth of identified data available to support research. Kinetic does not yet support automated statistical analyses of data, but they plan to provide code templates for researchers to modify for their own projects.
This project is supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305N210034 to Digital Promise. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.