Generative AI in Education

Large Language Models (LLMs) are increasingly used to create educational content such as quizzes. While generation quality has improved, there is no standardized, reproducible benchmark for evaluating assessments against pedagogically relevant criteria such as difficulty, fidelity to the source material, coverage, and distractor quality.

This thesis constructs and validates a modular benchmarking framework to systematically and reproducibly evaluate quizzes using multiple LLMs as judges. By providing structured, rubric-driven scores and logging all evaluation details, the framework quantifies variance, supports robust aggregation, and produces actionable benchmarking reports. This system provides a systematic, reproducible approach to evaluating quizzes against pedagogically relevant criteria.

Developing high-quality quiz questions within Artemis currently necessitates significant manual effort, deep domain expertise, and strict alignment with course contents. Consequently, editors face challenges in maintaining robust question pools, often resulting in limited practice material for students. This thesis proposes integrating generative artificial intelligence (AI) into the Artemis platform to streamline the quiz creation lifecycle.

The proposed solution establishes a human-in-the-loop workflow where editors define specific constraints, such as topic and question count, to produce structured initial drafts. Furthermore, generation quality is enhanced by utilizing internal platform data like course content and learning competencies. Finally, a dedicated refinement layer empowers editors to iteratively adjust drafts via natural language instructions prior to final approval.

Artemis and Athena help instructors create and assess digital learning activities in large courses, including programming exercises, quizzes, and modeling tasks. This thesis focuses on two practical challenges. First, exercise creators spend a lot of time writing grading instructions from scratch. Second, students can find Athena feedback hard to understand when the interface shows little code context and does not clearly mark feedback as positive or negative.

To address these challenges, the thesis adds LLM support that creates a first draft of grading instructions during exercise creation. It also improves how Artemis shows Athena feedback for programming exercises by adding scores and clearer code context. Together, these changes can reduce manual authoring effort and make feedback easier for students to understand.

In this thesis, the goal is to enhance the contextual awareness of a GPT-based educational chatbot, named IRIS, on the Artemis Learning Platform by incorporating lecture content. To do this, the lecture slides should be embedded into a vector database, and the chatbot should be able to retrieve the most relevant slides based on the user’s query in order to provide the most relevant answer.

The challenges encountered by students during the completion of exercises necessitate the implementation of a proactive assistance mechanism within Artemis. This could potentially be achieved through the integration of generative AI technologies such as ChatGPT. The objective of this thesis is to augment Artemis with the capability to provide automatic and proactive assistance to students when they encounter difficulties. The effectiveness and impact of this approach on the learning experience will be evaluated through a comprehensive assessment.

The goal of this thesis is to evaluate the effectiveness of IRIS, a GPT-based chatbot for higher education. The chatbot is integrated into the Artemis learning platform and is designed to provide assistance to students when they encounter difficulties. In addition, the chatbot is capable of answering questions related to the course content. Instructors can benefit from IRIS through assistance in exercise generation.

In this thesis, the effectiveness of IRIS will be evaluated through a combination of quantitative and qualitative methods. The quantitative evaluation will be conducted through a comprehensive assessment of the chatbot’s performance both in real course and experimental settings. The qualitative evaluation will be conducted through a survey of students and instructors and expert interviews. The results of the evaluation will be used to identify the strengths and weaknesses of the chatbot and to provide recommendations for future improvements.

Competency-based educational recommender systems enable personalized learning, but the absence of standardized benchmark datasets prevents systematic algorithmic comparison. This thesis develops a platform for collaborative competency relationship data collection with four objectives: a generalized database model, authentication systems for credential verification, administrative interfaces for dataset management and export, and production deployment infrastructure. The system enables systematic collection of high-quality competency mapping data for creating reliable benchmarks.

This thesis focuses on transitioning an experimental prototype to a production-ready Open Educational Resources (OER) platform. LEARN-Hub aims to distribute standardized teaching materials for computer science (CS) education. The existing system requires increased maintenance effort and offers limited support for processing teaching materials at scale.

The work migrates the server from Flask to Spring Boot to align the platform with institutional standards. It designs a pipeline to transform unstructured PDFs into structured data for consistent visualization. In addition, the work refines the React client to adhere to established usability heuristics and to ensure coherent presentation of the generated content.

Although computer science enrollments have recently declined, student numbers remain high and still require scalable teaching tools. This project enhances Artemis and its Artificial Intelligence (AI) subsystem, Athena, to automate feedback. Currently, fragmented interfaces create excessive instructor workloads and a disjointed student experience. These structural issues hinder the adoption of automated tools and reduce the educational value of the feedback provided.

This thesis unifies the assessment ecosystem by generalizing logic across exercise types. Refactoring the architecture into a single service layer eliminates technical debt and ensures data synchronization. A redesigned interface reduces workflow complexity for instructors, while integrated feedback allows students to engage directly within their submissions. This research improves usability, supports adoption, and enhances learning outcomes by providing a consistent AI-Feedback interface design for instructors and students.

At the Applied Education Technologies (AET) research group of the Technical University of Munich (TUM), both the applications it builds and the developers in its courses need access to large language models (LLMs). Logos provides this access, but represents users as a flat list of equally privileged Application Programming Interface (API) keys. Instructors have to create every user and key manually, cannot attribute requests to users or teams, and cannot cap capacity through budgets and rate limits. Existing gateways offer capacity control, but do not provision course teams from university-managed groups.

Institutional deployments of large language models (LLMs) face a fundamental tension between growing inference demand and constrained local Graphics Processing Unit (GPU) hardware. Without a dedicated orchestration layer, multiple applications sharing on-premise inference capacity must independently manage model selection and memory allocation, making coordinated hardware utilization impossible.

This guided research evaluates whether Logos, an open-source LLM orchestration layer developed at the Applied Education Technologies (AET) research group at TUM, provides measurable throughput benefits over direct access to LLM serving frameworks such as vLLM. The study adapts existing benchmarks for such frameworks to a multi-application, multi-model scenario and compares three conditions: Logos-orchestrated vLLM, direct vLLM access, and Ollama, across two request datasets and two inference nodes (dedicated GPU hosts). The results provide empirical evidence for the design choices in Logos and establish a reusable benchmark methodology for shared LLM infrastructure.

The Logos platform serves as a central gateway between users and AI models within a shared GPU infrastructure, hosting and serving models through vLLM. Currently, deploying new models requires manual intervention by the administrator, who must verify compatibility, resolve errors, and accept licensing agreements on behalf of all users. Users do not have visibility into the status of a model request, leaving them unable to make informed decisions about whether to wait, switch to an available model, or request an alternative.