Institutional deployments of large language models (LLMs) face a fundamental tension between growing inference demand and constrained local Graphics Processing Unit (GPU) hardware. Without a dedicated orchestration layer, multiple applications sharing on-premise inference capacity must independently manage model selection and memory allocation, making coordinated hardware utilization impossible.
This guided research evaluates whether Logos, an open-source LLM orchestration layer developed at the Applied Education Technologies (AET) research group at TUM, provides measurable throughput benefits over direct access to LLM serving frameworks such as vLLM. The study adapts existing benchmarks for such frameworks to a multi-application, multi-model scenario and compares three conditions: Logos-orchestrated vLLM, direct vLLM access, and Ollama, across two request datasets and two inference nodes (dedicated GPU hosts). The results provide empirical evidence for the design choices in Logos and establish a reusable benchmark methodology for shared LLM infrastructure.