Generative AI in Education

Large Language Models (LLMs) are increasingly used to create educational content such as quizzes. While generation quality has improved, there is no standardized, reproducible benchmark for evaluating assessments against pedagogically relevant criteria such as difficulty, fidelity to the source material, coverage, and distractor quality.

This thesis constructs and validates a modular benchmarking framework to systematically and reproducibly evaluate quizzes using multiple LLMs as judges. By providing structured, rubric-driven scores and logging all evaluation details, the framework quantifies variance, supports robust aggregation, and produces actionable benchmarking reports. This system provides a systematic, reproducible approach to evaluating quizzes against pedagogically relevant criteria.

Developing high-quality quiz questions within Artemis currently necessitates significant manual effort, deep domain expertise, and strict alignment with course contents. Consequently, editors face challenges in maintaining robust question pools, often resulting in limited practice material for students. This thesis proposes integrating generative artificial intelligence (AI) into the Artemis platform to streamline the quiz creation lifecycle.

The proposed solution establishes a human-in-the-loop workflow where editors define specific constraints, such as topic and question count, to produce structured initial drafts. Furthermore, generation quality is enhanced by utilizing internal platform data like course content and learning competencies. Finally, a dedicated refinement layer empowers editors to iteratively adjust drafts via natural language instructions prior to final approval.

In this thesis, the goal is to enhance the contextual awareness of a GPT-based educational chatbot, named IRIS, on the Artemis Learning Platform by incorporating lecture content. To do this, the lecture slides should be embedded into a vector database, and the chatbot should be able to retrieve the most relevant slides based on the user’s query in order to provide the most relevant answer.

The challenges encountered by students during the completion of exercises necessitate the implementation of a proactive assistance mechanism within Artemis. This could potentially be achieved through the integration of generative AI technologies such as ChatGPT. The objective of this thesis is to augment Artemis with the capability to provide automatic and proactive assistance to students when they encounter difficulties. The effectiveness and impact of this approach on the learning experience will be evaluated through a comprehensive assessment.

The goal of this thesis is to evaluate the effectiveness of IRIS, a GPT-based chatbot for higher education. The chatbot is integrated into the Artemis learning platform and is designed to provide assistance to students when they encounter difficulties. In addition, the chatbot is capable of answering questions related to the course content. Instructors can benefit from IRIS through assistance in exercise generation.

In this thesis, the effectiveness of IRIS will be evaluated through a combination of quantitative and qualitative methods. The quantitative evaluation will be conducted through a comprehensive assessment of the chatbot’s performance both in real course and experimental settings. The qualitative evaluation will be conducted through a survey of students and instructors and expert interviews. The results of the evaluation will be used to identify the strengths and weaknesses of the chatbot and to provide recommendations for future improvements.

Competency-based educational recommender systems enable personalized learning, but the absence of standardized benchmark datasets prevents systematic algorithmic comparison. This thesis develops a platform for collaborative competency relationship data collection with four objectives: a generalized database model, authentication systems for credential verification, administrative interfaces for dataset management and export, and production deployment infrastructure. The system enables systematic collection of high-quality competency mapping data for creating reliable benchmarks.

This thesis focuses on transitioning an experimental prototype to a production-ready Open Educational Resources (OER) platform. LEARN-Hub aims to distribute standardized teaching materials for computer science (CS) education. The existing system requires increased maintenance effort and offers limited support for processing teaching materials at scale.

The work migrates the server from Flask to Spring Boot to align the platform with institutional standards. It designs a pipeline to transform unstructured PDFs into structured data for consistent visualization. In addition, the work refines the React client to adhere to established usability heuristics and to ensure coherent presentation of the generated content.