Developing an International Large-Scale AI Tool for Educational Assessment and Personalised Learning

Instead of comparing students against one another, the model captures each learner’s individual growth over time. By delivering personalised diagnostic and robust analytical capabilities, it supports both formative assessments and summative assessments.

In addition, the model’s capacity for in-depth analysis at both national and school levels allows countries and economies to extract richer insights from diverse data sources, including national and international assessments. Crucially, the project offers practical strategies to tackle a common challenge in education: how to transform assessment data into targeted interventions that improve student proficiency.

Furthermore, this new assessment technology enables countries and economies to develop or enhance their own examinations, accurately measuring even complex competencies that they wish to assess. By capturing such nuanced information, the model stands to fundamentally transform how learning is assessed and supported worldwide.

The primary objective is to leverage AI technologies and psychometric methods to provide reliable and valid assessments of test-takers’ competencies. By offering instantaneous assessments across a broad spectrum of skills, the model enables educators and policymakers to make swift, informed decisions that effectively support student progress. In addition, the project aims to facilitate highly granular data analysis at both national and school levels. By drawing on datasets from national assessments and international assessments, educational institutions can gain more nuanced, actionable interpretations of their data, which ultimately drive targeted improvements in student outcomes.

Another goal of this project is to offer clear, evidence-based insights that address a common challenge in education: converting assessment data into practical strategies for enhancing test-taker's proficiency. By focusing on actionable insights, the model empowers educators, administrators, and policymakers to design targeted interventions that directly address students’ learning needs.

The initiative also seeks to promote personalised learning environments by providing teachers with detailed diagnostic information. As a result, instruction can be differentiated and tailored to each learner’s progression, helping students remain engaged and reach their full potential.

Finally, the project aims to explore policy guidelines and ethical considerations surrounding AI-based assessments. This includes developing best practices and recommendations that ensure data privacy, fairness, and transparency, thereby creating an ethical framework that advances innovation while safeguarding the well-being of students and educators. 

The Collective Intelligence Model for Education (CIME)

The project aims to develop an international large-scale AI assessment model by assembling experts in educational measurement, natural language processing (NLP), and educational technology. This specialised team will establish robust theoretical and technical foundations to effectively integrate large-scale language models (LLMs) with psychometric modelling. Unlike general generative AI models, this controlled, framework-based approach ensures reliable evaluation of both written and oral responses, significantly enhancing assessment accuracy, fairness, and reliability. Key assessment criteria emphasized include cross-lingual consistency, equitable treatment across diverse student backgrounds, and invariance across different AI models. The project's methodologies, findings, and model capabilities will be thoroughly documented in a detailed technical report.

Validating and Enhancing CIME for Summative Assessments

Building upon the developed AI model, this phase applies the model to existing large-scale assessment datasets to validate its effectiveness and refine its application in summative assessments. Participating countries and economies with relevant educational data, especially open-ended assessment responses, will collaborate to validate and enhance the AI model's performance. Through ongoing dialogue with curriculum developers, policymakers, and educational experts, best practices will be identified for integrating AI-generated insights into policy decisions, curriculum improvements, and assessment strategies. This collaborative effort will continuously strengthen the AI model, providing actionable insights that drive informed educational policy and practice improvements. 

Adapting and Applying CIME for Formative Assessments in Schools

In this task, the previously developed AI model will be specifically adapted for formative assessments within school contexts. Countries and municipalities are invited to provide formative assessment data, such as student-written or oral responses, for group-level analysis without identifying individual students, ensuring compliance with privacy standards. The AI model's diagnostic outputs will offer educators timely, actionable feedback, enhancing teaching practices in real-time. Active collaboration with participants will refine the model based on continuous feedback, aligning it closely with educators' practical needs. Importantly, personal student data will not be stored or used for training, with assessments guided strictly by predefined expert rules. The project's outcomes will include widely disseminated best practices to guide the effective and responsible implementation of AI-driven formative assessments, significantly enhancing their adaptability and relevance in educational settings.

Ensuring Safe, Ethical AI Use and Supporting Teachers Effectively

This task involves collaborative discussions with participants from previous activities to ensure the safe, ethical, and effective use of the developed AI assessment model in educational settings. These conversations will address practical concerns such as data protection, intellectual property rights, privacy compliance, and ethical fairness, aiming to minimize biases and potential risks in AI-driven assessments. A central focus will be exploring ways to provide actionable, relevant AI-generated insights to teachers without overwhelming them, enhancing their classroom instruction effectively. Real-world experiences and feedback from prior activities will inform these discussions, resulting in concrete recommendations and best practices documented in the project's final report. This comprehensive resource will guide responsible AI implementation and effectively support teachers in schools. 

For any information about the project, please contact via email Tomoya Okubo (Project lead).