The National Science Foundation has awarded Indiana University and a leading educational technology company, Instructure, a planning grant to develop TOPSAIL – Testing Outcomes Privately and Securely for Artificial Intelligence in Learning – a groundbreaking testbed for research on the educational effectiveness of AI learning technologies.
In education, like in many domains, AI tools are emerging rapidly. Some of these tools show strong potential to improve student learning and success, but the speed of innovation has outpaced the research community's ability to evaluate them. TOPSAIL is a collaboration between researchers, schools, and ed tech innovators, that aims to make it faster and easier to evaluate new AI tools.
The two-year planning project, led by Principal Investigator Benjamin A. Motz in the IU College of Arts and Sciences' Department of Psychological and Brain Sciences, represents the first effort to design shared research infrastructure for evaluation of AI tools in contemporary learning environments. This initiative hopes to establish a new paradigm for evidence-based adoption of AI technologies in education, grounded in rigorous psychological science and learning theory.
"Testbeds enable rigorous and replicable research in situ, and they provide guardrails that protect end users’ rights and safety,” said Motz. “TOPSAIL will be a living laboratory with the goal of accelerating research and discovery in authentic learning environments."
Collaborative Approach to Urgent Challenges
TOPSAIL will be built in Canvas, Instructure's widely-used learning management system, and will leverage Terracotta, an experimental research plugin to Canvas that supports ethical research practices. Canvas serves millions of students nationwide, providing an ideal platform for real-world testing of AI innovations. The testbed will enable researchers to evaluate AI tools across diverse educational contexts while implementing robust privacy protections and experimental controls.
"As learning technologies, especially those with AI integrations, evolve at a rapid pace, the need for rigorous research has never been greater,” said Dr. Mary Styers, Co-Principal Investigator and Director of Research for LearnPlatform by Instructure. “This project is a crucial step toward creating the kind of evidence-based supports that educators and students deserve, while also implementing the guardrails necessary to ensure student safety and privacy,"
The project exemplifies both institutions' commitment to AI leadership and innovation. The team’s expertise in the science of learning and research infrastructure, combined with Instructure's educational platform reach, creates a unique foundation for improving evaluation methods.
The TOPSAIL team includes experts in the science of learning, educational technology, AI, and cyberinfrastructure. It has established partnerships with three K-12 school districts, recognizing that effective testbed design requires deep collaboration with local educational agencies. Through on-site focus groups and multi-stakeholder workshops, the project will co-create design requirements that address concerns about AI safety, privacy, and implementation.
Dr. Matthew Callison, Director of Innovation and Strategic Partnerships for the South Fayette Township School District in Pennsylvania, who is a key collaborator in the project, noted, “What excites me most about TOPSAIL is its collaborative approach—it creates the infrastructure for districts like ours to work alongside researchers and technology companies to ensure AI tools are truly serving our educational mission. TOPSAIL represents exactly what our field needs: a systematic way to pilot AI tools responsibly, gather meaningful data, and make informed decisions about scaling. It's research that directly informs practice while maintaining the rigorous standards our students and families deserve."
Transforming Applied Research Methodology
Testbeds have proven transformative in domains ranging from cybersecurity to autonomous vehicles, providing realistic environments where innovations can be safely piloted before widespread deployment. TOPSAIL will bring this proven methodology to education research for the first time, creating infrastructure that enables research teams to conduct pilot evaluations in authentic classrooms responsibly, rigorously, and rapidly.
The planning grant will establish the technical and ethical frameworks necessary for full testbed implementation. Through extensive stakeholder engagement, the project will develop protocols that enable research opportunities within the strict boundaries of student privacy and safety. The resulting design requirements will guide the development of a comprehensive testbed system that can serve researchers nationwide.
"This project will fundamentally change how we approach AI innovations in learning technologies," said Motz. "By creating shared infrastructure for rigorous testing, we're shifting the conversation about AI-enabled learning tools from ‘can this work in principle?’ to ‘does this work in practice?’"
The TOPSAIL testbed will serve as a model for research infrastructure development, demonstrating how academic institutions and industry partners can collaborate to address complex societal challenges through innovative solutions.
TOPSAIL is led by Indiana University in collaboration with Instructure, Arizona State University, and three K-12 school district partners.
The College of Arts